feat: enforce schema-backed agent outputs

.env.example

@@ -77,7 +77,7 @@ FORGE_REQUIRE_PROJECT_CONFIG=true
 # variables; connection definitions contain no secrets.
 GOOGLE_CLOUD_PROJECT=your-gcp-project-id
 GOOGLE_CLOUD_LOCATION=global
-MODEL_CONNECTIONS={"vertex-prod":{"backend":"vertex-ai","project":"your-gcp-project-id","location":"global","allowed_models":["gemini-3.5-pro","gemini-3.5-flash"],"capabilities":["tools"]}}
+MODEL_CONNECTIONS={"vertex-prod":{"backend":"vertex-ai","project":"your-gcp-project-id","location":"global","allowed_models":["gemini-3.5-pro","gemini-3.5-flash"],"capabilities":["structured_output","tools"]}}
 MODEL_DEFAULT={"connection":"vertex-prod","model":"gemini-3.5-flash"}
 # Optional per-stage override:
 # MODEL_POLICY={"generate_prd":{"connection":"vertex-prod","model":"gemini-3.5-pro"}}

README.md

@@ -56,9 +56,13 @@ The built-in model factory supports direct Anthropic API credentials and Google
 
 Forge is not just an agent with a large prompt or a folder of skills. It is a stateful delivery workflow that decides what should happen next, when to pause, which artifact needs review, which repository should be changed, and how to recover when something fails.
 
-- **Workflow first, agents second**: LangGraph coordinates the lifecycle from ticket intake to PR review. Agents perform bounded stage work; the workflow owns routing, checkpoints, retries, approvals, and handoffs.
+- **Workflow first, agents second**: Forge-owned, versioned definitions coordinate the lifecycle
+  from ticket intake to PR review. LangGraph executes those definitions; typed stations perform
+  bounded work without owning routing, checkpoints, approvals, or handoffs.
 - **Cross-repo by design**: Forge can plan features and bugs across services, clients, infrastructure, and documentation repos, then split the work into repo-scoped units that can be implemented and reviewed independently.
-- **Controlled write boundaries**: Agents do not directly mutate Jira, GitHub, or production repositories. Implementation agents write only inside their local/container workspace; Forge's integration layer performs external updates such as Jira comments, labels, branch pushes, and PR creation at explicit workflow steps.
+- **Controlled write boundaries**: Agents do not directly mutate Jira, GitHub, or production
+  repositories. Forge journals required external effects before execution and retains attempt and
+  provider evidence for recovery and operator replay.
 - **Native engineering loop**: Forge works through Jira tickets, Jira comments, Jira labels, GitHub PRs, GitHub reviews, and CI webhooks instead of forcing teams into a separate agent UI.
 - **Traceable by default**: Work is reflected back into Jira and GitHub as comments, labels, PRs, review updates, CI decisions, and post-merge summaries, so teams can follow why the workflow moved or paused.
 - **Project visibility**: Prometheus metrics, Langfuse traces, and Grafana dashboards expose workflow throughput, step latency, ticket execution cost, model usage, CI behavior, and observability health by project, ticket type, workflow step, and Jira issue.
@@ -170,19 +174,24 @@ This lets Forge follow local engineering conventions without forking the orchest
 
 ## Architecture
 
-Forge is event-driven:
+Forge is event-driven and checkpointed:
 
 ```text
-Jira + GitHub Webhooks
+Jira + GitHub Webhooks / Poller
   -> FastAPI Gateway
   -> Redis Streams Queue
-  -> LangGraph Workflow
-  -> Host Orchestrator Agent
-  -> Container Agent for Implementation
-  -> Jira + GitHub Updates
+  -> Observation Reconciliation
+  -> Pinned Versioned Workflow
+  -> Typed Stations
+  -> Durable External Effects
+  -> Jira + GitHub
 ```
 
-Jira and GitHub send webhooks to Forge. Forge queues events, resumes the right workflow state, runs the next node, and posts the result back to Jira or GitHub. Planning runs through the host orchestrator. Code implementation runs in short-lived containers. Agents generate artifacts and local code changes; Forge's workflow and integration layer decide when those outputs become Jira updates, branch pushes, or pull requests.
+Webhook and poller deliveries normalize to the same observation contract. Forge deduplicates and
+orders provider revisions, interprets accepted evidence through the instance's pinned workflow
+definition, and invokes typed stations. Planning agents run on the host and implementation agents
+run in short-lived containers. Jira and source-control mutations cross a durable effect journal;
+operators can inspect the combined process, observation, station, and effect timeline.
 
 ## Quick Start
 
@@ -218,6 +227,10 @@ See [Getting Started](https://Forge-sdlc.github.io/forge/getting-started/) for t
 - [Task Workflow](https://Forge-sdlc.github.io/forge/guide/task-workflow/): Understand standalone Task and Epic implementation.
 - [PR Commands](https://Forge-sdlc.github.io/forge/guide/pr-commands/): Rebase PRs and handle CI gate skips.
 - [Configuration Reference](https://Forge-sdlc.github.io/forge/reference/config/): Environment variables and project configuration.
+- [Architecture](https://Forge-sdlc.github.io/forge/architecture/): Versioned workflows,
+  reconciliation, typed stations, durable effects, and execution inspection.
+- [Declarative Workflows](https://Forge-sdlc.github.io/forge/reference/declarative-workflows/):
+  Author and govern constrained project workflow definitions.
 - [Skills System](https://Forge-sdlc.github.io/forge/skills/): Customize Forge for your team and stack.
 - [Developer Guide](https://Forge-sdlc.github.io/forge/developer-guide/): Local testing, debugging, Prometheus metrics, Langfuse tracing, and Grafana dashboards.
 

docs/architecture/index.md

@@ -1,11 +1,13 @@
-# Forge Architecture
+# Forge architecture
 
-Architecture reference for Forge, an AI-powered SDLC orchestrator. Covers system structure, runtime topology, state management, failure modes, security boundaries, and key design decisions.
+Architecture reference for Forge's versioned workflow control plane, typed stations, reconciliation,
+durable effects, execution inspection, and model-output boundaries.
 
 For workflow details, see the [Feature](../guide/feature-workflow.md), [Bug](../guide/bug-workflow.md), and [Task](../guide/task-workflow.md) guides. For API reference, see the OpenAPI spec at `/docs` when the gateway is running.
 
 | Part | Contents |
 |------|----------|
-| [System & Components](overview.md) | System context, external actors, component responsibilities |
-| [Internals](internals.md) | Runtime topology, state and concurrency, failure recovery, security |
+| [System and components](overview.md) | Control-plane structure and component responsibilities |
+| [Runtime internals](internals.md) | State authority, reconciliation, stations, effects, and security |
 | [Reference](reference.md) | Architectural decisions, known limitations, workflow lifecycles |
+| [Structured model output](structured-output.md) | Typed model responses and provider fallback behavior |

docs/architecture/internals.md

@@ -1,63 +1,68 @@
-# Internals
+# Runtime internals
 
-## Runtime Topology
+## State and correctness boundaries
 
-Forge runs as two process types plus Redis:
+Forge deliberately keeps four kinds of durable state separate:
 
-- **Gateway**: Single FastAPI/Uvicorn process. Stateless; can be load-balanced.
-- **Worker(s)**: One or more `OrchestratorWorker` processes. Each joins the Redis consumer group. **Must run on a host with Podman installed.** Each worker handles up to 20 concurrent tasks (configurable via `QUEUE_MAX_CONCURRENT_TASKS`).
-- **Redis**: Single-instance server. No built-in HA; must be provided externally if required.
+| Record | Authority | Purpose |
+| --- | --- | --- |
+| Observation ledger | External resource revisions | Deduplicate, order, and classify webhook and poller evidence |
+| Workflow checkpoint | Forge process instance | Pin the definition and retain process position and station state |
+| Effect journal | Forge external-write intent | Make mutations recoverable and idempotent across crashes |
+| Execution timeline | Operational history | Explain observations, transitions, attempts, effects, and operator actions |
 
-Gateway and Worker communicate only through Redis and can be deployed on separate hosts. Horizontal Worker scaling has a limitation: per-ticket event serialization uses an in-process `asyncio.Lock`, not a distributed lock (see [Known Limitations](reference.md#known-limitations)).
+External facts do not directly overwrite process position. An accepted observation is interpreted
+as a command, validated, and applied through the selected workflow's transition policy. Conversely,
+a checkpoint does not claim ownership of Jira issue content, pull-request state, or CI results; a
+new provider revision can cause those facts to be reconciled and re-evaluated.
 
-## State and Event Processing
+## Delivery and concurrency
 
-**Delivery guarantee:** At-least-once. Messages are acknowledged (`XACK`) only after successful processing. The system does not provide exactly-once semantics.
+Queue delivery is at least once. The observation `delivery_identity` makes equivalent webhook and
+poller deliveries converge before command handling. The ledger uses monotonic provider revisions
+and records duplicate, stale, conflict, and accepted decisions. Where a provider supplies no stable
+revision, Forge requires a stable event identity and reports ambiguity instead of guessing.
 
-**Checkpointing:** LangGraph workflow state is persisted via `AsyncRedisSaver`, keyed by Jira ticket key (e.g., `AISOS-123`). Checkpoints are written after each graph node completes. When a new event arrives for an existing ticket, the workflow resumes from its last checkpoint.
+Workflow state is persisted through LangGraph's Redis checkpointer. Definitions are pinned by
+revision and digest, so publication or activation of a newer revision cannot silently change an
+in-flight run. Compatibility analysis and explicit migration mappings govern intentional moves.
 
-**Idempotency:** The worker records normalized webhook/poller observations in
-the reconciliation ledger before command interpretation. Duplicate, stale,
-and conflicting observations do not re-enter the workflow. External writes
-still use durable effect identities; provider operations such as Jira comment
-posting must remain idempotent across crash recovery.
+## Station execution
 
-**Consistency boundary:** Workflow mutations are persisted as stable effect intents before
-provider execution. Provider-specific recovery evidence and idempotent ref updates close
-the crash window between provider success and checkpoint acknowledgement. Workflow state
-and effect state remain separate durable records, joined by the workflow run identity.
+A graph node projects the permitted checkpoint fields into a versioned station request. The station
+returns a typed outcome; a reducer validates and applies only the fields that station owns. The same
+request can run through the local station runner without Redis, LangGraph, or provider clients,
+except where the station's declared capability explicitly requires an adapter.
 
-## Failure and Recovery
+Agent operations resolve a model connection through stage policy and declared capabilities such as
+`tools` and `structured_output`. Structured stages preserve the full Deep Agent tool loop, validate
+the final object, and retry with a tool-based schema strategy when native structured output fails.
 
-| Component | Failure impact | Recovery |
-|-----------|---------------|----------|
-| Gateway | Incoming webhooks dropped | Jira/GitHub retry delivery per their own policies |
-| Worker | In-flight messages stay in Redis PEL | Restart consumes new messages; PEL requires manual `XCLAIM` |
-| Redis | Complete system outage; all state at risk | Configure Redis persistence (RDB/AOF) externally |
-| LLM provider | Planning/code generation fails | Retried up to 3 times, then moved to dead-letter queue |
-| Container | Non-zero exit captured by orchestrator | Retry mechanism determines re-attempt |
+## External effects and recovery
 
-**Retry policy:** Up to 3 attempts with exponential backoff (30s initial, 2x multiplier, capped at 1 hour). Failed messages go to a dead-letter queue for manual investigation.
+Required external mutations are stable `EffectCommand` values. Forge records an intent before
+calling Jira or source control, leases execution, and stores attempt history and provider evidence.
+Reprocessing the same logical action reuses its idempotency identity. Indeterminate and failed
+effects are visible through the operator API and can be replayed without rerunning the whole station.
 
-**Blocked workflows:** The `forge:blocked` label is applied to Jira tickets in error state. Adding `forge:retry` triggers re-entry at the failed step.
+The operator endpoints expose:
 
-**Approval gates:** Workflows pause indefinitely at human review gates. There is no automatic timeout or escalation.
+- `GET /api/v1/workflows/{ticket_key}/execution`
+- `GET /api/v1/workflows/{ticket_key}/execution/timeline`
+- `GET /api/v1/effects/workflow/{run_id}`
+- `POST /api/v1/effects/{idempotency_key}/replay`
 
-## Security Boundaries
+These views do not advance workflow state.
 
-**Webhook authentication:** HMAC-SHA256 validation via `hmac.compare_digest()`. Validation is conditional: it only runs when secrets are configured (`JIRA_WEBHOOK_SECRET`, `GITHUB_WEBHOOK_SECRET`). **Always configure secrets in production.**
+## Security boundaries
 
-**Credential distribution:**
-
-| Credential | Worker | Container |
-|------------|--------|-----------|
-| Redis | Yes | No |
-| Jira API token | Yes | No |
-| GitHub App credentials | Yes | No |
-| LLM provider (API key or Vertex AI) | Yes | Yes |
-| Langfuse | Yes | Yes (when enabled) |
-| Git identity | No | Yes |
-
-Containers do not receive Jira, GitHub, or Redis credentials. All external platform operations are performed by the orchestrator after the container exits.
-
-**Container isolation:** Rootless Podman with configurable network mode (`slirp4netns` default), memory limit (4GB), CPU limit (2 cores), and 30-minute timeout. Workspace mounted read-write at `/workspace`; task file read-only at `/task.json`.
+- Webhook signatures are validated when the corresponding secret is configured; production must
+  configure both Jira and source-control secrets.
+- Operator execution/effect routes require their configured bearer token and fail closed when the
+  token is absent.
+- Rootless Podman constrains implementation execution with configured CPU, memory, network, and
+  timeout limits.
+- Containers do not receive Jira, Redis, or source-control credentials. Those writes pass through
+  the host-side durable effect boundary.
+- Custom workflow definitions select registered capabilities; they cannot embed credentials,
+  provider-specific calls, arbitrary HTTP, shell code, or Python imports.