From e9a6ef554d17145af79d332f138d86aed3f87f16 Mon Sep 17 00:00:00 2001 From: Mahmoud Mabrouk Date: Wed, 24 Jun 2026 23:01:22 +0200 Subject: [PATCH 1/3] docs(agent): design EmbedRef tools (tools-as-workflows) Add a plan-feature workspace under docs/design/agent-workflows/projects/embedref-tools/ for allowing an @ag.embed reference in the agent config tools field, the way skills already supports it. Researches the generic embed resolver, the tool taxonomy, and the _agenta.* platform catalog; proposes a callback-executor workflow tool variant plus an embed-as-content stepping stone. Design only, spun from PR #4821 review comment 3469653315. Claude-Session: https://claude.ai/code/session_01GYo3UEfvsZpncagqb28Mbc --- .../projects/embedref-tools/README.md | 45 +++++ .../projects/embedref-tools/context.md | 73 +++++++ .../projects/embedref-tools/plan.md | 183 +++++++++++++++++ .../projects/embedref-tools/research.md | 187 ++++++++++++++++++ .../projects/embedref-tools/status.md | 81 ++++++++ 5 files changed, 569 insertions(+) create mode 100644 docs/design/agent-workflows/projects/embedref-tools/README.md create mode 100644 docs/design/agent-workflows/projects/embedref-tools/context.md create mode 100644 docs/design/agent-workflows/projects/embedref-tools/plan.md create mode 100644 docs/design/agent-workflows/projects/embedref-tools/research.md create mode 100644 docs/design/agent-workflows/projects/embedref-tools/status.md diff --git a/docs/design/agent-workflows/projects/embedref-tools/README.md b/docs/design/agent-workflows/projects/embedref-tools/README.md new file mode 100644 index 0000000000..0ab617b587 --- /dev/null +++ b/docs/design/agent-workflows/projects/embedref-tools/README.md @@ -0,0 +1,45 @@ +# EmbedRef tools (tools-as-workflows) + +Index for the design workspace that lets the agent config `tools` field accept an +`@ag.embed` reference, the same way `skills` already does. An embed in `tools` lets an +author write a tool **as a workflow** and have the backend inline it into a runnable tool +spec before the runner ever sees it. + +Spun out of PR #4821 review comment +[3469653315](https://github.com/Agenta-AI/agenta/pull/4821#discussion_r3469653315) on +`interfaces/public-edge/agent-config-schema.md`: *"we should also allow here embedref like +skills. these would allow creating tools as workflows and embedding them."* + +This is a **design-only** workspace. No code is changed by this PR. It is POC / +pre-production: no back-compat is required. + +## Files + +- [context.md](context.md) — why this exists, goals, non-goals, the reviewer's ask, and + the one hard difference from skills (a tool must be *invoked*, a skill is just content). +- [research.md](research.md) — how `skills` embedding actually works today (the generic + `@ag.embed` resolver, the `ResolverMiddleware`, the `_agenta.*` platform catalog), the + tool taxonomy (type/executor model), and the exact seams to mirror, with file paths. +- [plan.md](plan.md) — the proposed change: the embed-ref schema arm, the inlined tool + shape, the new `workflow` tool variant and its `callback` executor, the server-side + execute endpoint, the wire, tests, and rollout. +- [status.md](status.md) — current state, the key decision, and the open questions for the + user. + +## One-paragraph answer to the reviewer + +Yes, and the embedding half is almost free. The `@ag.embed` resolver is **already generic**: +the `ResolverMiddleware` walks the whole `parameters` tree (lists included) and inlines every +embed server-side *before* `AgentConfig.from_params` parses the config or `resolve_tools` +runs, so an `@ag.embed` placed inside `tools[i]` already resolves with **zero resolver +changes**. The real design work is three smaller things: (1) make the strict +`AgentConfigSchema.tools` accept the embed-ref arm (mirroring `_SkillEmbedRefSchema`), so a +referenced tool validates in the playground; (2) decide **what shape** the embedded workflow +inlines into — the cleanest answer is a new `type: "workflow"` tool variant that the inline +substitutes into; and (3) decide **how that becomes callable** — a workflow tool is +server-executed, so it fits the existing `callback` executor exactly like a `gateway` tool: +it resolves to a `CallbackToolSpec` whose `call_ref` encodes the workflow identity, and a +server-side execute endpoint runs the referenced workflow revision and returns the result. +The runner needs **no new `kind`**. See [the key decision](status.md#key-decision) for the +two embedding flavors (embed-as-content vs reference-as-tool) and why the design recommends +the second. diff --git a/docs/design/agent-workflows/projects/embedref-tools/context.md b/docs/design/agent-workflows/projects/embedref-tools/context.md new file mode 100644 index 0000000000..0d9eb4a76a --- /dev/null +++ b/docs/design/agent-workflows/projects/embedref-tools/context.md @@ -0,0 +1,73 @@ +# Context + +## Why this exists + +The agent config has two list fields that an author commits: `tools` and `skills`. They are +not symmetric today. + +- **`skills`** accepts `(SkillConfig | EmbedRef)[]`. An author can write a skill inline as a + `SkillConfig`, OR drop an `@ag.embed` reference to a workflow and the backend inlines that + workflow's content into a concrete `SkillConfig` before the runner sees it. The default + config ships exactly such an embed (the `_agenta.agenta-getting-started` platform skill). +- **`tools`** accepts only the four concrete variants `ToolConfig = builtin | gateway | code + | client`. There is no embed-ref arm, so a tool cannot be authored as a workflow and reused + by reference. + +PR #4821 review comment +[3469653315](https://github.com/Agenta-AI/agenta/pull/4821#discussion_r3469653315) asks to +close that gap: + +> we should also allow here embedref like skills. these would allow creating tools as +> workflows and embedding them. + +This unlocks **tools-as-workflows**: a tool is authored once as a workflow revision (with its +own versioning, history, and editing surface), then referenced from any agent config by an +embed. The agent author does not re-declare the tool's body; they point at it. + +## Goals + +- Make `tools` accept an `@ag.embed` reference, mirroring the `skills` shape. +- Define what the embedded workflow inlines into (which tool variant / shape). +- Define how an embedded-workflow tool becomes a **callable** tool at run time, within the + existing three-axis tool taxonomy (executor / approval / render) and the resolved-spec + model — without a new runner `kind` if avoidable. +- Define the resolution path end to end: schema arm -> embed resolution (already generic) -> + parse -> tool resolution -> `/run` wire -> runner dispatch -> server-side execute. +- Keep secrets and connection auth server-side, the same safety property gateway tools have. + +## Non-goals + +- Back-compat. This is POC / pre-production; we may change the union and the wire freely. +- Building the workflow-authoring UI for tools (the editor that produces the referenced + workflow revision). This design assumes a workflow revision exists; producing it is a + separate surface. +- Changing the generic embed resolver. It already walks `tools[]`; this design relies on + that, it does not modify it. +- A new vault or connection concept. Embedded-workflow tools reuse the existing named-secret + and connection resolution. +- MCP. `mcp_servers` is a sibling field with its own deferral; out of scope here. + +## The reviewer's ask, restated + +Add an embed-ref arm to `tools` so a tool can be created as a workflow and embedded. Two +mechanisms must meet: the **embedding** mechanism (already exists, generic) and the +**tool-ness** mechanism (the embedded thing has to end up as a tool the agent can call). + +## The one hard difference from skills + +A skill and a tool are inlined the same way, but they *do* different things, and that is the +whole design problem: + +- A **skill** is passive content. Once inlined into a `SkillConfig`, it is just markdown + + files laid into the workspace. Nothing executes it; the model reads it. So skill embedding + needed no executor design at all — inline the content, done. +- A **tool** is active. Once inlined, the agent must be able to **call** it: select it by + name, pass arguments, and get a result. So "inline the workflow into a tool" raises a + question skills never had to answer: *what runs when the model calls this tool, and where?* + +The answer this design lands on: an embedded-workflow tool resolves to the existing +`callback` executor (the same one gateway tools use). When the model calls it, the runner +posts the call back to an Agenta service endpoint, which **invokes the referenced workflow +revision** and returns the result. Execution stays server-side; the runner only relays. See +[research.md](research.md) for why `callback` is the right fit and +[plan.md](plan.md) for the concrete shape. diff --git a/docs/design/agent-workflows/projects/embedref-tools/plan.md b/docs/design/agent-workflows/projects/embedref-tools/plan.md new file mode 100644 index 0000000000..b15179bb79 --- /dev/null +++ b/docs/design/agent-workflows/projects/embedref-tools/plan.md @@ -0,0 +1,183 @@ +# Plan + +The proposed change to let `tools` accept an `@ag.embed` reference and turn an embedded +workflow into a callable tool. POC / pre-production: no back-compat. + +The plan has two layers, because "embedref like skills" can mean two genuinely different +things. The recommended design is **Option B** (reference-as-tool); **Option A** +(embed-as-content) is the cheaper first step and is fully compatible as a stepping stone. +Both are detailed; the open questions in [status.md](status.md) decide which we build first. + +## Option A — Embed-as-content (cheapest, reuses everything) + +An embed in `tools` inlines a **concrete, already-supported tool config** that an author +committed inside a workflow's parameters. The workflow is just a reusable container for a +tool declaration. + +How it works: + +1. Schema: add the embed-ref arm to `tools` (the only required code change shared by both + options). In `sdks/python/agenta/sdk/utils/types.py`, add `_ToolEmbedRefSchema` (copy of + `_SkillEmbedRefSchema`) and make `AgentConfigSchema.tools` a + `List[Union[ToolConfig, _ToolEmbedRefSchema]]`. +2. Authoring: a tool workflow stores a tool config under `data.parameters.tool`, e.g. + `{"tool": {"type": "gateway", "provider": "composio", ...}}`. +3. Embed: the agent config references it: + ```jsonc + { "@ag.embed": { "@ag.references": { "workflow": { "slug": "my-github-issue-tool" } }, + "@ag.selector": { "path": "parameters.tool" } } } + ``` +4. Resolution: the **generic resolver inlines it** into a concrete `gateway`/`code`/`client` + config at `tools[i]`. `resolve_tools` then resolves it on the existing path. **No new tool + variant, no new executor, no wire change, no runner change.** + +What it buys: reuse and versioning of a tool declaration across agents. What it does not buy: +a tool whose *body is itself a workflow* — it can only inline tool types we already run. + +This is the smallest possible "embedref like skills" and is almost free (one schema arm). + +## Option B — Reference-as-tool (the real "tools as workflows") + +The embedded workflow is **not** a tool config; it is a workflow that *becomes* a tool. When +the model calls the tool, Agenta **invokes that workflow revision** with the model's arguments +and returns its output as the tool result. This is what "creating tools as workflows" most +naturally means: any workflow (a prompt, a chain, an evaluator-style step) is exposed to the +agent as a single callable tool. + +### B1 — New tool variant `workflow` + +In `sdks/python/agenta/sdk/agents/tools/models.py`, add to the discriminated union: + +```python +class WorkflowToolConfig(ToolConfigBase): + type: Literal["workflow"] = "workflow" + name: str = Field(min_length=1) # the tool name the model calls + description: Optional[str] = None # what the tool does (for the model) + input_schema: Dict[str, Any] = Field(default_factory=_empty_object_schema) + # the referenced workflow (filled by the embed inline, or authored directly): + workflow_slug: str = Field(min_length=1) + workflow_version: Optional[str] = None # None = latest revision +``` + +and add `"workflow"` to the accepted `type` set in `compat.py`. + +### B2 — What the embed inlines into + +The selector path extracts a small ref payload from the workflow revision's parameters and the +generic resolver substitutes it into a `WorkflowToolConfig` shape at `tools[i]`. Two +sub-options for the selector target (an open question): + +- **(b2-i)** The workflow stores a ready tool surface under `parameters.tool` + (`{name, description, input_schema, workflow_slug}`), mirroring how skills store + `parameters.skill`. The selector path is `parameters.tool`. Cleanest parallel to skills. +- **(b2-ii)** The embed inlines the *whole revision identity* and the resolver/config derives + the tool surface (name/description from the workflow metadata, input schema from the + workflow's declared inputs). Less authoring, more inference. + +Recommendation: **(b2-i)** for symmetry with skills and to keep the tool's model-facing +surface explicit and reviewable. + +### B3 — Resolution to a `callback` spec + +A `WorkflowToolConfig` resolves to the existing `CallbackToolSpec` (resolved `kind: +"callback"`), so the runner needs no new `kind`. Mirror the gateway path: + +- Add a branch in `ToolResolver.resolve` (or, cleaner, a new injected resolver + `WorkflowToolResolver` alongside `GatewayToolResolver`) that turns workflow tool configs + into `CallbackToolSpec`s and one `ToolCallback`. +- The `call_ref` encodes the workflow identity. Propose a distinct grammar from the + Composio 5-segment one, e.g. `workflow.{slug}` or `workflow.{slug}.{version}` (open + question — see status.md). The runner treats `call_ref` as opaque, so only the server-side + parser must agree. +- The `ToolCallback` endpoint points at the execute target (B4). + +`ResolvedToolSet` holds a **single** `tool_callback`. If both gateway and workflow tools are +present in one config, either (a) they share one endpoint that routes by `call_ref` prefix +(`tools.*` vs `workflow.*`), or (b) `ResolvedToolSet`/the wire grows per-spec callbacks. The +single-shared-endpoint route (a) is the smaller change and keeps the wire stable. + +### B4 — Server-side execute endpoint + +A `/tools/call`-style target (extend `api/oss/src/apis/fastapi/tools/router.py` + core) that: + +1. Parses the `workflow.*` `call_ref` into a workflow slug/version. +2. Maps the model's tool-call arguments to the workflow invoke inputs. +3. Invokes the referenced workflow revision (the same invoke path the platform already uses; + reserved `_agenta.*` slugs short-circuit to the catalog). +4. Maps the workflow output back to the tool result envelope `{call:{data:{content}}}` the + runner expects (`callback.ts` reads `parsed.call.data.content`). + +Connections and secrets the workflow needs are resolved **server-side** during that invoke, +exactly like a gateway tool — nothing reaches the sandbox. This is the central safety +property and the reason `callback` is the right executor. + +### B5 — Wire and runner + +No new `/run` field. A workflow tool rides the wire as a `callback` `ResolvedToolSpec` with a +`workflow.*` `callRef` and the shared `toolCallback`. The runner dispatch, the Daytona relay, +the Pi native delivery, and the Claude `agenta-tools` bridge all already handle `callback`. +Golden fixtures gain a workflow-tool example; `protocol.ts` and `wire.py` are unchanged in +shape (only fixture content changes). + +### B6 — Platform tool workflows (optional, later) + +To ship an `_agenta.*` platform tool workflow, generalize `_validate_catalog` in +`api/oss/src/core/workflows/platform_catalog.py` (today it hard-validates every payload as +`SkillConfig`). Store the tool payload under `parameters.tool`. User-authored DB tool +workflows do not need this; they are not in the catalog. + +## Resolution path, end to end (Option B) + +``` +author commits agent config with an @ag.embed in tools[i] + | +SDK ResolverMiddleware: _has_embed_markers(parameters) is true (walks lists) + | POST {api}/workflows/revisions/resolve +API generic resolver: find_object_embeds -> fetch_workflow_revision(slug) + | selector path "parameters.tool" -> WorkflowToolConfig payload + | set_path substitutes it into parameters.tools[i] + v +_agent: AgentConfig.from_params(...) parses the inlined WorkflowToolConfig + | +resolve_tools(agent_config.tools): + | WorkflowToolResolver -> CallbackToolSpec(call_ref="workflow.") + ToolCallback + v +/run wire: customTools[i] = {kind:"callback", callRef:"workflow.", ...}, toolCallback + | +runner dispatch (callback): model calls the tool -> POST /tools/call (or Daytona relay) + v +API execute: parse workflow.* call_ref -> invoke workflow revision with args -> result + | +result -> {call:{data:{content}}} -> back to the model +``` + +## Test plan + +- **SDK unit:** `WorkflowToolConfig` parses (strict + loose coercion); `_ToolEmbedRefSchema` + validates an embed-ref `tools` entry; `WorkflowToolResolver` produces the expected + `CallbackToolSpec` + `ToolCallback`. +- **Schema:** `AgentConfigSchema` JSON Schema emits the embed-ref `oneOf` arm in `tools` + (mirror the skills schema test); `CATALOG_TYPES["agent_config"]` still dereferences. +- **Embed resolution (API):** an `@ag.embed` in `tools[i]` inlines into a `WorkflowToolConfig` + (Option B) or a concrete tool config (Option A); cycle/depth guards still hold. +- **Wire / golden:** a golden `/run` fixture with a workflow tool; `protocol.ts` Zod accepts + it as a `callback` spec. +- **Execute endpoint:** a `/tools/call` with a `workflow.*` `call_ref` invokes the revision + and returns the result; a user workflow's secrets/connections stay server-side. +- **Live matrix (agent-workflows-qa):** force the tool with an unguessable token across + pi_core / claude on local + Daytona + SDK; a pass proves the workflow ran server-side and + the result reached the model. Pin a green cell with agent-replay-test. + +## Rollout + +POC, off no flag needed for the schema arm (it is additive and the resolver already handles +it). The execute endpoint is new surface; gate platform tool workflows (B6) behind the +existing reserved-namespace trust, not a feature flag. Keep docs in sync in the same +implementation PR (tools.md, agent-config-schema.md, the interface inventory). + +## Build order (when implemented) + +1. Option A schema arm (the shared, almost-free win) — `tools` accepts `@ag.embed`, + inlining concrete tool configs. Ships value immediately, validates the embedding half. +2. Option B variant + resolver + execute endpoint — tools-as-workflows proper. +3. Option B6 platform tool workflows (catalog validation generalization) if/when wanted. diff --git a/docs/design/agent-workflows/projects/embedref-tools/research.md b/docs/design/agent-workflows/projects/embedref-tools/research.md new file mode 100644 index 0000000000..c53f1aef8c --- /dev/null +++ b/docs/design/agent-workflows/projects/embedref-tools/research.md @@ -0,0 +1,187 @@ +# Research + +How `skills` embedding works today, the tool taxonomy, and the exact seams to mirror for +`tools`. Everything below is grounded in the current code; file paths are absolute-from-repo. + +## Part 1 — How `@ag.embed` embedding works (the skills case) + +### There is no `EmbedRef` model — an embed is a structural marker + +An embed is a plain dict whose marker key is `@ag.embed`, recognized by a recursive walker. +There is no dedicated Pydantic class for it on the runtime path. + +- SDK marker: `sdks/python/agenta/sdk/middlewares/running/resolver.py` — + `_AG_EMBED_MARKER = "@ag.embed"`. +- API markers: `api/oss/src/core/embeds/utils.py` — + `AG_EMBED_KEY = "@ag.embed"`, `AG_REFERENCES_KEY = "@ag.references"`, + `AG_SELECTOR_KEY = "@ag.selector"`. + +The canonical object-embed shape (the form `skills` uses): + +```jsonc +{ + "@ag.embed": { + "@ag.references": { "workflow": { "slug": "_agenta.agenta-getting-started" } }, + "@ag.selector": { "path": "parameters.skill" } + } +} +``` + +`@ag.references` is `Dict[str, Reference]` keyed by entity type (`workflow`, +`workflow_revision`, ...). The inner `Reference` / `Selector` DTOs are in +`sdks/python/agenta/sdk/models/shared.py` (`Reference(id, slug, version)`, +`Selector(key, path)`). A bare `workflow` key is an **artifact-level** lookup (latest +revision); the comment in the default-config builder is load-bearing: referencing the +artifact (`workflow.slug`) resolves to the latest revision, while a bare *revision* slug with +no version returns 500. + +### The resolver is generic and runs BEFORE the agent handler + +Two layers: + +1. **SDK middleware** — `sdks/python/agenta/sdk/middlewares/running/resolver.py`. + `ResolverMiddleware.__call__` checks `_has_embed_markers(parameters)` (recursive: descends + dicts, **lists**, and strings) and, if any embed is present and the `resolve` flag is on, + POSTs `parameters` to `{api}/workflows/revisions/resolve` and replaces them with the + resolved result. Its own comment says: *"The embed resolver walks arrays, so an + `@ag.embed` inside `parameters.skills[i]` resolves on either path."* The same is true of + `parameters.tools[i]`. + +2. **API generic resolver** — `api/oss/src/core/embeds/utils.py`, `resolve_embeds(...)`. + It deep-copies the config and loops up to `max_depth`, each pass calling + `find_object_embeds(...)` (a recursive walker that records an `ObjectEmbed{location, + references, selector}` for every dict carrying `@ag.embed`, and **recurses into list items + and dict values otherwise**). For each embed it: resolves the references via a callback, + applies the `@ag.selector` `path` to the resolved revision's `data` + (`_extract_with_sdk_resolver`, using the SDK `resolve_any`), and `set_path(...)` + substitutes the extracted value back at the embed's location. Cycle / depth / count + guards exist (`CircularEmbedError`, `MaxDepthExceededError`, `MaxEmbedsExceededError`). + +The resolver callback routes a `workflow` reference to +`workflows_service.fetch_workflow_revision(...)` (`api/oss/src/core/embeds/service.py` wires +`EmbedsService` to the same catalog-aware `WorkflowsService`). + +**Ordering in the agent run path** (`services/oss/src/agent/app.py`, `_agent`): + +1. Embed resolution — already done by the SDK middleware against `parameters`, before + `_agent` is even called. +2. `agent_config = AgentConfig.from_params(params, ...)` — parses the *now-inlined* config. +3. `resolved_tools = await resolve_tools(agent_config.tools)` — sees only concrete, + embed-free tool configs. + +**Implication:** an `@ag.embed` in `tools[i]` is inlined at step 1 with no resolver change. +By step 3 it is a concrete tool config. The work is making steps 2-3 (and the schema) +understand *what* it inlines into. + +### The `_agenta.*` platform catalog short-circuit + +`api/oss/src/core/workflows/platform_catalog.py` defines `PlatformWorkflowCatalog`, a +code-defined, read-only set of platform workflows keyed by a reserved `_agenta.*` slug. +`WorkflowsService.fetch_workflow_revision` calls `_resolve_platform_revision` *first*; a +reserved slug never falls through to Postgres. Each catalog version stores its payload under +`data.parameters.` — for skills that key is `skill` +(`parameters={"skill": skill_config...}`), which is why the skill embed's selector path is +`parameters.skill`. + +One hard constraint to note: `_validate_catalog` currently validates **every** catalog +payload as a `SkillConfig` (`SkillConfig.model_validate(payload)`). To ship a *platform* tool +workflow, that validation must generalize per payload kind. (User-authored tool workflows +live in the DB and do not hit this validation.) + +### Where the union lives (skills, the template to copy) + +- Runtime `AgentConfig.skills`: `sdks/python/agenta/sdk/agents/dtos.py` — + `skills: List[SkillConfig]` (NOT a union; embeds are already resolved by the time it + parses). A `@field_validator("skills", mode="before")` coerces. +- Strict `AgentConfigSchema.skills`: `sdks/python/agenta/sdk/utils/types.py` — + `List[Union["SkillConfigSchema", "_SkillEmbedRefSchema"]]`. The embed arm is + `_SkillEmbedRefSchema` with `embed: Dict[str, Any] = Field(alias="@ag.embed")` and + `extra="forbid"`. This is the exact arm to mirror for tools. +- Default config: `build_agent_v0_default(...)` in + `sdks/python/agenta/sdk/utils/types.py` ships the skill `@ag.embed` block. + +## Part 2 — The tool taxonomy (what an embedded tool must become) + +### Two lives, three axes + +`documentation/tools.md` is the canonical reference. A tool has a **declared config** +(`AgentConfig.tools`, portable, no secrets) and a **resolved spec** (the `/run` wire, secrets +injected, endpoints filled). Three orthogonal axes: **executor** (`type` at config time, +`kind` at runtime), **`needs_approval`**, **`render`**. + +Declared `type` -> resolved `kind`: + +| Declared `type` | Resolved form | Resolved `kind` | Who executes / where | +| --- | --- | --- | --- | +| `builtin` | a bare name | (none) | the harness, natively | +| `gateway` | `CallbackToolSpec` + `call_ref` | `callback` | the Agenta service, via `POST /tools/call` | +| `code` | `CodeToolSpec` + `env` | `code` | the runner, local subprocess | +| `client` | `ClientToolSpec` | `client` | the browser, next turn | + +Models: `sdks/python/agenta/sdk/agents/tools/models.py` +(`ToolConfigBase`, the four `*ToolConfig`, the `ToolConfig = Annotated[Union[...], +Field(discriminator="type")]`, and the resolved `CallbackToolSpec` / `CodeToolSpec` / +`ClientToolSpec` discriminated by `kind`; `ResolvedToolSet{builtin_names, tool_specs, +tool_callback}`). The TS twin is `ResolvedToolSpec` in `services/agent/src/protocol.ts`. + +### How resolution + dispatch work + +- SDK `ToolResolver.resolve` (`sdks/python/agenta/sdk/agents/tools/resolver.py`) partitions + configs by `isinstance`, resolves code secrets via a `ToolSecretProvider`, resolves gateway + configs via a `GatewayToolResolver` (which returns the `CallbackToolSpec` list **and** the + single shared `ToolCallback`), and returns a `ResolvedToolSet`. +- Platform composition `resolve_tools` (`sdks/python/agenta/sdk/agents/platform/resolve.py`) + wires the Agenta adapters (`AgentaNamedSecretProvider`, `AgentaGatewayToolResolver`). +- The gateway adapter (`sdks/python/agenta/sdk/agents/platform/gateway.py`) POSTs to + `POST /tools/resolve`, gets a `call_ref` slug + `tools.{provider}.{integration}.{action}.{connection}`, wraps each in a `CallbackToolSpec`, + and assembles one `ToolCallback(endpoint="{api}/tools/call", authorization=...)`. +- Runner dispatch `runResolvedTool` (`services/agent/src/tools/dispatch.ts`) branches on + `kind`: `code` runs locally; `client` throws (browser-fulfilled); **`callback` (default) + POSTs back to `/tools/call`** (directly, or via the Daytona file relay). Absent `kind` + defaults to `callback`. + +### Why `callback` is the right executor for an embedded-workflow tool + +A workflow tool is **server-executed**: calling it means invoking another Agenta workflow +revision, which lives behind the API and may itself use connections and secrets. That is +exactly the gateway tool's safety shape — the harness decides *which* tool and *with what +arguments*, the service runs it, and no credential reaches the sandbox. So an +embedded-workflow tool should resolve to a `CallbackToolSpec`: + +- The runner needs **no new `kind`** — `callback` already dispatches to `callAgentaTool`, + works under the Daytona file relay, and is delivered to both Pi (native) and Claude (the + `agenta-tools` MCP bridge). +- The only thing that differs from a gateway tool is the `call_ref` grammar and the + server-side execute target: instead of running a Composio action, the service invokes a + workflow revision. + +## Part 3 — The seams to touch (summary) + +| Seam | File | Change | +| --- | --- | --- | +| Strict schema embed arm | `sdks/python/agenta/sdk/utils/types.py` | add `_ToolEmbedRefSchema`, make `AgentConfigSchema.tools` a `Union[ToolConfig-twin, _ToolEmbedRefSchema]` (mirror skills) | +| New tool variant | `sdks/python/agenta/sdk/agents/tools/models.py` | add `WorkflowToolConfig(type="workflow")` to the union, carrying the workflow ref + tool surface (name, description, input schema) | +| Loose coercion allowlist | `sdks/python/agenta/sdk/agents/tools/compat.py` | add `"workflow"` to the accepted `type` set | +| Resolution branch | `sdks/python/agenta/sdk/agents/tools/resolver.py` + a new platform resolver in `.../platform/` | resolve a `WorkflowToolConfig` to a `CallbackToolSpec` + a `ToolCallback` to the new execute endpoint | +| Server-side execute | `api/oss/src/apis/fastapi/tools/router.py` (+ core) | a `/tools/call`-style target that invokes the referenced workflow revision and returns the result | +| Embed resolver | `api/oss/src/core/embeds/utils.py` | **no change** — already walks `tools[]` | +| Platform catalog validation | `api/oss/src/core/workflows/platform_catalog.py` | generalize `_validate_catalog` IF we ship a platform tool workflow (otherwise unchanged for user workflows) | +| Wire | `services/agent/src/protocol.ts`, `sdks/python/agenta/sdk/agents/utils/wire.py`, golden fixtures | **no new field** if it resolves to an existing `callback` spec; only the `call_ref` value differs | +| Docs | `documentation/tools.md`, `interfaces/public-edge/agent-config-schema.md`, the interface inventory | document the new variant + the embed arm | + +## Open research questions (carried into the plan) + +1. **Embed-as-content vs reference-as-tool.** The selector path could inline either a + *concrete tool config* (e.g. a `code`/`gateway` config stored in the workflow's + parameters — pure reuse, no new variant, no new executor) OR a *workflow reference* that + becomes a new `workflow` tool variant (a tool that, when called, runs the workflow). Both + are "embedref like skills." They are very different in cost and meaning. See + [status.md](status.md#key-decision). +2. **What does invoking the workflow mean** — call `/workflows/.../invoke` with the model's + arguments as inputs, and map the workflow output back as the tool result? What is the + input/output contract between a tool call and a workflow invoke? +3. **The `call_ref` grammar** for a workflow tool (today's 5-segment gateway grammar is + Composio-specific and parsed in both `compat.py` and the API router). +4. **Platform tool workflows** — do we want `_agenta.*` platform tools (needs the catalog + validation generalization), or only user-authored DB workflows at first? diff --git a/docs/design/agent-workflows/projects/embedref-tools/status.md b/docs/design/agent-workflows/projects/embedref-tools/status.md new file mode 100644 index 0000000000..6008654ea3 --- /dev/null +++ b/docs/design/agent-workflows/projects/embedref-tools/status.md @@ -0,0 +1,81 @@ +# Status + +This is the source of truth for the project's progress, decisions, and open questions. + +## Current state + +- **Phase:** IMPLEMENTED (the lgtm'd two-syntax design #4837). Spun from PR #4821 review comment + [3469653315](https://github.com/Agenta-AI/agenta/pull/4821#discussion_r3469653315). +- **Docs:** README, context, research, plan, status written. Grounded in current code + (skills embed resolver, the tool taxonomy, the platform catalog) with file paths. +- **Next:** user picks the direction (Option A vs B, and the open questions below), then this + goes through `implement-feature`. + +## Key decision + +**An embed in `tools` can mean two different things; the design recommends supporting both, +A first then B.** + +- **Option A — embed-as-content (almost free):** the embed inlines a *concrete, already- + supported tool config* (`gateway`/`code`/`client`) that an author stored inside a workflow. + Pure reuse. The only code change is the strict-schema embed-ref arm; the generic resolver, + `resolve_tools`, the wire, and the runner are all untouched. +- **Option B — reference-as-tool (the real "tools as workflows"):** the embed references a + workflow that *becomes* a callable tool. Calling it invokes the workflow revision + server-side and returns its output. Fits the existing `callback` executor (like a gateway + tool), so the runner needs no new `kind`. Adds a `workflow` tool variant, a resolver + branch, and a server-side execute endpoint. + +**Recommendation:** ship Option A immediately (it is the literal "embedref like skills" and +costs one schema arm), and build Option B as the substantive feature. Option A is a clean +stepping stone, not a dead end — both share the same embed-ref arm. + +**Why `callback`, not a new executor (Option B):** a workflow tool is server-executed and may +use connections/secrets, which is exactly the gateway tool's safety shape. Resolving to a +`CallbackToolSpec` keeps every credential server-side and reuses the runner's existing +callback delivery (direct, Daytona relay, Pi native, Claude `agenta-tools` bridge). + +## Settled by research + +- The `@ag.embed` resolver is **generic and already walks `tools[]`** — no resolver change is + needed for embedding. (`ResolverMiddleware` + `api/oss/src/core/embeds/utils.py`.) +- Embed resolution runs **before** `AgentConfig.from_params` and `resolve_tools`, so by the + time tools resolve, the embed is already a concrete config. +- The skills schema arm (`_SkillEmbedRefSchema`) and the `_agenta.*` platform catalog are the + exact templates to mirror. +- The platform catalog `_validate_catalog` hard-codes `SkillConfig`; shipping a *platform* + tool workflow (not user workflows) would need that generalized. + +## Open questions for the user + +1. **Option A, Option B, or both?** A is nearly free and matches the comment literally; B is + the deeper feature the comment hints at ("creating tools as workflows"). Recommendation: + both, A first. +2. **Selector target for Option B** — does the tool workflow store a ready tool surface under + `parameters.tool` (explicit, mirrors skills, recommended) or do we infer the tool surface + (name/description/input schema) from the workflow's metadata and declared inputs? +3. **Tool-call to workflow-invoke contract** — how do the model's tool arguments map to the + workflow's invoke inputs, and how does the workflow output map back to the tool result? + Free-form passthrough, or a declared input/output schema on the tool workflow? +4. **`call_ref` grammar for workflow tools** — `workflow.{slug}` / `workflow.{slug}.{version}`? + Today's gateway grammar (`tools.{provider}.{integration}.{action}.{connection}`) is + Composio-specific and parsed in two places; a workflow tool needs its own opaque slug. +5. **Single shared callback endpoint vs per-spec callbacks** — `ResolvedToolSet` holds one + `tool_callback`. With both gateway and workflow tools present, do we route one endpoint by + `call_ref` prefix (smaller change, recommended) or grow the wire to per-spec callbacks? +6. **Platform tool workflows now or later?** Generalizing `_validate_catalog` to ship + `_agenta.*` tools is optional; user-authored DB tool workflows do not need it. +7. **Approval / render axes** — a workflow tool can carry `needs_approval` and `render` like + any tool; confirm there is no special handling wanted (default: they compose as usual). + +## Risks / watch-fors + +- **One callback channel.** The single `tool_callback` is a real constraint if mixing tool + types; the prefix-routing answer (Q5) avoids a wire change. +- **Embed must reference the artifact** (`workflow.slug`), not a bare revision slug with no + version (returns 500) — same gotcha skills have. +- **Two models, one contract.** The strict `AgentConfigSchema` and the permissive runtime + `AgentConfig` must move together (and a golden fixture), per agent-config-schema.md's + "watch for when changing." +- **Keep docs in sync** in the implementation PR: `documentation/tools.md`, + `interfaces/public-edge/agent-config-schema.md`, and the interface inventory. From ad6abc5cd62147bb507a7e4444eebfaff57cfb2d Mon Sep 17 00:00:00 2001 From: Mahmoud Mabrouk Date: Wed, 24 Jun 2026 23:23:05 +0200 Subject: [PATCH 2/3] docs(agent): simplify EmbedRef tools to runnable-vs-not per PR #4837 review Reference a workflow as a tool; branch only on runnable vs not. Runnable -> server-side callback execute (like gateway); non-runnable (client) -> resolve the reference into a concrete client tool config, fulfilled client-side. Any workflow type qualifies (no WorkflowToolConfig variant, no tool-workflow type; is_tool is a later FE-only display hint). Platform tools stay in the existing tools endpoints, not the workflow catalog. Dropped the Option A/B split and the B1-B6 machinery. Kept the load-bearing finding that the @ag.embed resolver is already generic and already walks tools[]. Claude-Session: https://claude.ai/code/session_01GYo3UEfvsZpncagqb28Mbc --- .../projects/embedref-tools/README.md | 60 ++-- .../projects/embedref-tools/context.md | 87 +++--- .../projects/embedref-tools/plan.md | 263 ++++++++---------- .../projects/embedref-tools/research.md | 70 +++-- .../projects/embedref-tools/status.md | 136 +++++---- 5 files changed, 316 insertions(+), 300 deletions(-) diff --git a/docs/design/agent-workflows/projects/embedref-tools/README.md b/docs/design/agent-workflows/projects/embedref-tools/README.md index 0ab617b587..c61ae7c907 100644 --- a/docs/design/agent-workflows/projects/embedref-tools/README.md +++ b/docs/design/agent-workflows/projects/embedref-tools/README.md @@ -1,9 +1,11 @@ # EmbedRef tools (tools-as-workflows) -Index for the design workspace that lets the agent config `tools` field accept an -`@ag.embed` reference, the same way `skills` already does. An embed in `tools` lets an -author write a tool **as a workflow** and have the backend inline it into a runnable tool -spec before the runner ever sees it. +Index for the design workspace that lets the agent config `tools` field **reference a +workflow**, the same way `skills` already does. A tool is just a workflow — any workflow +(agent, completion, channel, chain) can be referenced as a tool. What happens when the model +calls it depends only on whether that workflow is **runnable** (server-side callback execute, +like gateway) or **non-runnable** (resolved into a concrete `client` tool, fulfilled in the +browser). Spun out of PR #4821 review comment [3469653315](https://github.com/Agenta-AI/agenta/pull/4821#discussion_r3469653315) on @@ -15,31 +17,33 @@ pre-production: no back-compat is required. ## Files -- [context.md](context.md) — why this exists, goals, non-goals, the reviewer's ask, and - the one hard difference from skills (a tool must be *invoked*, a skill is just content). -- [research.md](research.md) — how `skills` embedding actually works today (the generic - `@ag.embed` resolver, the `ResolverMiddleware`, the `_agenta.*` platform catalog), the - tool taxonomy (type/executor model), and the exact seams to mirror, with file paths. -- [plan.md](plan.md) — the proposed change: the embed-ref schema arm, the inlined tool - shape, the new `workflow` tool variant and its `callback` executor, the server-side - execute endpoint, the wire, tests, and rollout. -- [status.md](status.md) — current state, the key decision, and the open questions for the - user. +- [context.md](context.md) — why this exists, goals, non-goals, the reviewer's ask, and the + one branch that matters: runnable vs non-runnable. +- [research.md](research.md) — how `skills` referencing works today (the generic `@ag.embed` + resolver, the `ResolverMiddleware`), the tool taxonomy (type/executor model), and the exact + seams to mirror, with file paths. The load-bearing finding: the resolver is already generic + and already walks `tools[]`. +- [plan.md](plan.md) — the simplified design: the embed-ref schema arm, the single + runnable-vs-not branch (callback execute vs resolve-to-`client`), the server-side execute + endpoint, the wire, tests, and rollout. Explicitly drops the old Option A/B split, the + `workflow` tool variant, and platform-tools-as-workflows. +- [status.md](status.md) — current state, the settled design, and the remaining open + questions. ## One-paragraph answer to the reviewer -Yes, and the embedding half is almost free. The `@ag.embed` resolver is **already generic**: -the `ResolverMiddleware` walks the whole `parameters` tree (lists included) and inlines every -embed server-side *before* `AgentConfig.from_params` parses the config or `resolve_tools` -runs, so an `@ag.embed` placed inside `tools[i]` already resolves with **zero resolver -changes**. The real design work is three smaller things: (1) make the strict +Yes, and the referencing half is almost free. The `@ag.embed` resolver is **already generic**: +the `ResolverMiddleware` walks the whole `parameters` tree (lists included) and resolves every +reference server-side *before* `AgentConfig.from_params` parses the config or `resolve_tools` +runs, so a reference placed inside `tools[i]` already resolves with **zero resolver changes**. +The real design is two things, and one branch. The two things: (1) make the strict `AgentConfigSchema.tools` accept the embed-ref arm (mirroring `_SkillEmbedRefSchema`), so a -referenced tool validates in the playground; (2) decide **what shape** the embedded workflow -inlines into — the cleanest answer is a new `type: "workflow"` tool variant that the inline -substitutes into; and (3) decide **how that becomes callable** — a workflow tool is -server-executed, so it fits the existing `callback` executor exactly like a `gateway` tool: -it resolves to a `CallbackToolSpec` whose `call_ref` encodes the workflow identity, and a -server-side execute endpoint runs the referenced workflow revision and returns the result. -The runner needs **no new `kind`**. See [the key decision](status.md#key-decision) for the -two embedding flavors (embed-as-content vs reference-as-tool) and why the design recommends -the second. +referenced tool validates in the playground; (2) a server-side execute endpoint that invokes a +referenced workflow revision. The one branch is **runnable vs non-runnable**, decided in the +service / resolve step: a **runnable** workflow resolves to the existing `callback` executor +(like a gateway tool — a `CallbackToolSpec` whose `call_ref` encodes the workflow identity, run +server-side, no new runner `kind`); a **non-runnable** (client) workflow is **resolved into its +value** — a concrete `client` tool config — and fulfilled the existing client way. There is no +`workflow` tool variant (a tool is just a workflow; any type qualifies), and platform tools +stay in the existing tools endpoints, not the workflow catalog. See +[the design](status.md#design) for the details. diff --git a/docs/design/agent-workflows/projects/embedref-tools/context.md b/docs/design/agent-workflows/projects/embedref-tools/context.md index 0d9eb4a76a..5683c39778 100644 --- a/docs/design/agent-workflows/projects/embedref-tools/context.md +++ b/docs/design/agent-workflows/projects/embedref-tools/context.md @@ -20,54 +20,67 @@ close that gap: > we should also allow here embedref like skills. these would allow creating tools as > workflows and embedding them. -This unlocks **tools-as-workflows**: a tool is authored once as a workflow revision (with its -own versioning, history, and editing surface), then referenced from any agent config by an -embed. The agent author does not re-declare the tool's body; they point at it. +This unlocks **tools-as-workflows**: a tool is just a workflow (with its own versioning, +history, and editing surface), referenced from any agent config. The agent author does not +re-declare the tool's body; they point at it. **Any** workflow qualifies — agent, completion, +channel, chain — there is no special "tool workflow" type. ## Goals -- Make `tools` accept an `@ag.embed` reference, mirroring the `skills` shape. -- Define what the embedded workflow inlines into (which tool variant / shape). -- Define how an embedded-workflow tool becomes a **callable** tool at run time, within the - existing three-axis tool taxonomy (executor / approval / render) and the resolved-spec - model — without a new runner `kind` if avoidable. -- Define the resolution path end to end: schema arm -> embed resolution (already generic) -> - parse -> tool resolution -> `/run` wire -> runner dispatch -> server-side execute. -- Keep secrets and connection auth server-side, the same safety property gateway tools have. +- Make `tools` accept a workflow **reference** (the `@ag.embed` arm), mirroring the `skills` + shape. +- Define the one branch that matters: **runnable vs non-runnable**, and how each is handled + (runnable → server-side callback execute, like gateway; non-runnable → resolve-to-value plus + the existing client-tool handling). +- Keep the runner free of a new `kind`: runnable rides as a `callback` spec, non-runnable as a + `client` spec. +- Keep secrets and connection auth server-side for the runnable case, the same safety property + gateway tools have. ## Non-goals - Back-compat. This is POC / pre-production; we may change the union and the wire freely. -- Building the workflow-authoring UI for tools (the editor that produces the referenced - workflow revision). This design assumes a workflow revision exists; producing it is a - separate surface. +- A `workflow` tool variant. A referenced workflow is just a workflow; no new tool type in the + discriminated union. +- **Platform tools as workflows.** Platform tools belong in the **existing tools endpoints** + (the same place gateway tools are added), not in the workflow catalog. The `_agenta.*` + tool-workflow / catalog-validation direction is dropped from this design. +- The `is_tool` flag. It is a later, FE-only display hint so referenced workflows surface in + the tool picker; it is noted, not designed here. +- Building the workflow-authoring UI for tools. This design assumes a workflow revision exists; + producing it is a separate surface. - Changing the generic embed resolver. It already walks `tools[]`; this design relies on that, it does not modify it. -- A new vault or connection concept. Embedded-workflow tools reuse the existing named-secret +- A new vault or connection concept. Runnable workflow tools reuse the existing named-secret and connection resolution. - MCP. `mcp_servers` is a sibling field with its own deferral; out of scope here. ## The reviewer's ask, restated -Add an embed-ref arm to `tools` so a tool can be created as a workflow and embedded. Two -mechanisms must meet: the **embedding** mechanism (already exists, generic) and the -**tool-ness** mechanism (the embedded thing has to end up as a tool the agent can call). - -## The one hard difference from skills - -A skill and a tool are inlined the same way, but they *do* different things, and that is the -whole design problem: - -- A **skill** is passive content. Once inlined into a `SkillConfig`, it is just markdown + - files laid into the workspace. Nothing executes it; the model reads it. So skill embedding - needed no executor design at all — inline the content, done. -- A **tool** is active. Once inlined, the agent must be able to **call** it: select it by - name, pass arguments, and get a result. So "inline the workflow into a tool" raises a - question skills never had to answer: *what runs when the model calls this tool, and where?* - -The answer this design lands on: an embedded-workflow tool resolves to the existing -`callback` executor (the same one gateway tools use). When the model calls it, the runner -posts the call back to an Agenta service endpoint, which **invokes the referenced workflow -revision** and returns the result. Execution stays server-side; the runner only relays. See -[research.md](research.md) for why `callback` is the right fit and -[plan.md](plan.md) for the concrete shape. +Add an embed-ref arm to `tools` so a tool can be created as a workflow and referenced. Two +mechanisms must meet: the **referencing** mechanism (already exists, generic) and the +**tool-ness** mechanism (the referenced workflow has to end up as a tool the agent can call). + +## The one branch that matters: runnable vs non-runnable + +A skill is always passive content (markdown + files; the model reads it, nothing executes). +A referenced workflow is not uniform — it can be runnable or not — and that is the whole +design: + +- **Runnable** (a completion, an agent, a channel, a chain — anything the platform can + invoke). You **reference** it because you want to **call** it. When the model calls the + tool, the call routes server-side and Agenta **invokes the workflow revision**, exactly like + a gateway tool: the sidecar/runner relays the call back, the service runs it, the result + returns to the model. Execution and any connections/secrets stay server-side. This resolves + to the existing `callback` executor — **no new runner `kind`**. + +- **Non-runnable** (a client tool — fulfilled in the browser, nothing to execute + server-side). Referencing-to-call does not apply. It is handled the way client tools are + handled today: the resolve step in the service **resolves the reference into its value** (a + concrete `client` tool config), and at run time the model's call is fulfilled client-side + next turn, the existing `client` path. + +So **what you reference decides the behavior**. The runnable/not decision is made in the +service / the resolve step, where the referenced workflow is known. A unifying way to say it: +reference everything as a tool, and **in the sidecar, if it is runnable, run it; if it is not, +return its schema**. See [plan.md](plan.md) for the concrete shape. diff --git a/docs/design/agent-workflows/projects/embedref-tools/plan.md b/docs/design/agent-workflows/projects/embedref-tools/plan.md index b15179bb79..13f40b4c19 100644 --- a/docs/design/agent-workflows/projects/embedref-tools/plan.md +++ b/docs/design/agent-workflows/projects/embedref-tools/plan.md @@ -1,183 +1,158 @@ # Plan -The proposed change to let `tools` accept an `@ag.embed` reference and turn an embedded -workflow into a callable tool. POC / pre-production: no back-compat. - -The plan has two layers, because "embedref like skills" can mean two genuinely different -things. The recommended design is **Option B** (reference-as-tool); **Option A** -(embed-as-content) is the cheaper first step and is fully compatible as a stepping stone. -Both are detailed; the open questions in [status.md](status.md) decide which we build first. - -## Option A — Embed-as-content (cheapest, reuses everything) - -An embed in `tools` inlines a **concrete, already-supported tool config** that an author -committed inside a workflow's parameters. The workflow is just a reusable container for a -tool declaration. - -How it works: - -1. Schema: add the embed-ref arm to `tools` (the only required code change shared by both - options). In `sdks/python/agenta/sdk/utils/types.py`, add `_ToolEmbedRefSchema` (copy of - `_SkillEmbedRefSchema`) and make `AgentConfigSchema.tools` a - `List[Union[ToolConfig, _ToolEmbedRefSchema]]`. -2. Authoring: a tool workflow stores a tool config under `data.parameters.tool`, e.g. - `{"tool": {"type": "gateway", "provider": "composio", ...}}`. -3. Embed: the agent config references it: - ```jsonc - { "@ag.embed": { "@ag.references": { "workflow": { "slug": "my-github-issue-tool" } }, - "@ag.selector": { "path": "parameters.tool" } } } - ``` -4. Resolution: the **generic resolver inlines it** into a concrete `gateway`/`code`/`client` - config at `tools[i]`. `resolve_tools` then resolves it on the existing path. **No new tool - variant, no new executor, no wire change, no runner change.** - -What it buys: reuse and versioning of a tool declaration across agents. What it does not buy: -a tool whose *body is itself a workflow* — it can only inline tool types we already run. - -This is the smallest possible "embedref like skills" and is almost free (one schema arm). - -## Option B — Reference-as-tool (the real "tools as workflows") - -The embedded workflow is **not** a tool config; it is a workflow that *becomes* a tool. When -the model calls the tool, Agenta **invokes that workflow revision** with the model's arguments -and returns its output as the tool result. This is what "creating tools as workflows" most -naturally means: any workflow (a prompt, a chain, an evaluator-style step) is exposed to the -agent as a single callable tool. - -### B1 — New tool variant `workflow` - -In `sdks/python/agenta/sdk/agents/tools/models.py`, add to the discriminated union: - -```python -class WorkflowToolConfig(ToolConfigBase): - type: Literal["workflow"] = "workflow" - name: str = Field(min_length=1) # the tool name the model calls - description: Optional[str] = None # what the tool does (for the model) - input_schema: Dict[str, Any] = Field(default_factory=_empty_object_schema) - # the referenced workflow (filled by the embed inline, or authored directly): - workflow_slug: str = Field(min_length=1) - workflow_version: Optional[str] = None # None = latest revision -``` - -and add `"workflow"` to the accepted `type` set in `compat.py`. +Let the agent config `tools` field accept a **reference to a workflow**, so any workflow can +be used as a tool. POC / pre-production: no back-compat. -### B2 — What the embed inlines into +## The model (one path, split by runnable vs not) -The selector path extracts a small ref payload from the workflow revision's parameters and the -generic resolver substitutes it into a `WorkflowToolConfig` shape at `tools[i]`. Two -sub-options for the selector target (an open question): +The old plan had two competing options (embed-as-content vs reference-as-tool) and a special +`workflow` tool variant. That was over-built. The simplified model the author landed on: -- **(b2-i)** The workflow stores a ready tool surface under `parameters.tool` - (`{name, description, input_schema, workflow_slug}`), mirroring how skills store - `parameters.skill`. The selector path is `parameters.tool`. Cleanest parallel to skills. -- **(b2-ii)** The embed inlines the *whole revision identity* and the resolver/config derives - the tool surface (name/description from the workflow metadata, input schema from the - workflow's declared inputs). Less authoring, more inference. +**A tool is just a referenced workflow.** You point `tools[i]` at a workflow (by reference, +not by inlining its config). What happens when the model calls it depends only on whether that +workflow is **runnable** (executable) or **not**. -Recommendation: **(b2-i)** for symmetry with skills and to keep the tool's model-facing -surface explicit and reviewable. +- **Runnable** (a completion, an agent, a channel, a chain — anything the platform can + invoke): you *reference* it because you want to *call* it. The model's call routes + server-side and Agenta **runs the workflow revision**, exactly like a gateway tool. The + sidecar relays the call back; the service invokes; the result returns to the model. Secrets + and connections the workflow needs stay server-side. +- **Non-runnable** (a client tool — fulfilled in the browser, nothing to execute + server-side): referencing-to-call does not apply. It is handled the way client tools are + handled today. Its value is **resolved/embedded into the config** server-side (the resolve + step in the service), and at run time the model's call is fulfilled client-side next turn, + the existing `client` path. -### B3 — Resolution to a `callback` spec +So **what you reference decides the behavior**: runnable → server-side callback execute; +non-runnable → resolve-to-value + the existing client handling. -A `WorkflowToolConfig` resolves to the existing `CallbackToolSpec` (resolved `kind: -"callback"`), so the runner needs no new `kind`. Mirror the gateway path: +### Any workflow qualifies — there is no "tool workflow" type -- Add a branch in `ToolResolver.resolve` (or, cleaner, a new injected resolver - `WorkflowToolResolver` alongside `GatewayToolResolver`) that turns workflow tool configs - into `CallbackToolSpec`s and one `ToolCallback`. -- The `call_ref` encodes the workflow identity. Propose a distinct grammar from the - Composio 5-segment one, e.g. `workflow.{slug}` or `workflow.{slug}.{version}` (open - question — see status.md). The runner treats `call_ref` as opaque, so only the server-side - parser must agree. -- The `ToolCallback` endpoint points at the execute target (B4). +An invocable tool *is* a workflow. There is no need for a workflow specially marked as a tool. +Any workflow type — agent, completion, channel, chain — can be referenced as a tool. We do +**not** add a `WorkflowToolConfig` variant. -`ResolvedToolSet` holds a **single** `tool_callback`. If both gateway and workflow tools are -present in one config, either (a) they share one endpoint that routes by `call_ref` prefix -(`tools.*` vs `workflow.*`), or (b) `ResolvedToolSet`/the wire grows per-spec callbacks. The -single-shared-endpoint route (a) is the smaller change and keeps the wire stable. +Later (note only, out of scope here): add an `is_tool` flag on a workflow purely so the +frontend can list it in the tool picker. It is a display hint; it changes no runtime behavior. -### B4 — Server-side execute endpoint +### One unifying rule for the sidecar -A `/tools/call`-style target (extend `api/oss/src/apis/fastapi/tools/router.py` + core) that: +Reference everything as a tool. **In the sidecar, if the referenced thing is runnable, run it; +if it is not runnable, return its schema** (instead of executing). That single rule covers +both cases without branching the wire by tool kind: -1. Parses the `workflow.*` `call_ref` into a workflow slug/version. -2. Maps the model's tool-call arguments to the workflow invoke inputs. -3. Invokes the referenced workflow revision (the same invoke path the platform already uses; - reserved `_agenta.*` slugs short-circuit to the catalog). -4. Maps the workflow output back to the tool result envelope `{call:{data:{content}}}` the - runner expects (`callback.ts` reads `parsed.call.data.content`). +- runnable → the callback executes the workflow and returns the result; +- non-runnable → the callback (or the resolve step) returns the schema/value, and the model is + fulfilled the client way. -Connections and secrets the workflow needs are resolved **server-side** during that invoke, -exactly like a gateway tool — nothing reaches the sandbox. This is the central safety -property and the reason `callback` is the right executor. +## What the embed inlines into -### B5 — Wire and runner +The `@ag.embed` resolver is **already generic** and already walks `tools[]` (see +[research.md](research.md)) — this is the one genuinely-useful research finding and it still +holds. Embed resolution runs in the SDK `ResolverMiddleware` *before* +`AgentConfig.from_params` parses the config and *before* `resolve_tools` runs. So a reference +placed in `tools[i]` is resolved with **zero resolver changes**. -No new `/run` field. A workflow tool rides the wire as a `callback` `ResolvedToolSpec` with a -`workflow.*` `callRef` and the shared `toolCallback`. The runner dispatch, the Daytona relay, -the Pi native delivery, and the Claude `agenta-tools` bridge all already handle `callback`. -Golden fixtures gain a workflow-tool example; `protocol.ts` and `wire.py` are unchanged in -shape (only fixture content changes). +The split decides what the resolve step produces: -### B6 — Platform tool workflows (optional, later) +- **Runnable** → keep the reference. The config carries a workflow reference (slug, optional + version) plus the model-facing surface (name, description, input schema). It resolves to the + existing `callback` executor: a `CallbackToolSpec` whose `call_ref` encodes the workflow + identity, plus the shared `ToolCallback` pointing at a server-side execute target. The runner + needs **no new `kind`** — `callback` already dispatches everywhere (direct, Daytona relay, Pi + native, the Claude `agenta-tools` bridge). +- **Non-runnable** → resolve to a value. The resolve step in the service turns the reference + into a concrete `client` tool config (name, description, input schema). At run time it is the + existing `client` path: the runner returns a `client` spec, the browser fulfills it next + turn. No callback, no server-side execute. -To ship an `_agenta.*` platform tool workflow, generalize `_validate_catalog` in -`api/oss/src/core/workflows/platform_catalog.py` (today it hard-validates every payload as -`SkillConfig`). Store the tool payload under `parameters.tool`. User-authored DB tool -workflows do not need this; they are not in the catalog. +Where the runnable/not decision is made: in the **service / the embed (resolve) step**, when +the reference is resolved. That is where we know what the referenced workflow is. -## Resolution path, end to end (Option B) +## Resolution path, end to end ``` -author commits agent config with an @ag.embed in tools[i] +author commits agent config with a workflow reference in tools[i] | -SDK ResolverMiddleware: _has_embed_markers(parameters) is true (walks lists) +SDK ResolverMiddleware: _has_embed_markers(parameters) true (walks lists) | POST {api}/workflows/revisions/resolve -API generic resolver: find_object_embeds -> fetch_workflow_revision(slug) - | selector path "parameters.tool" -> WorkflowToolConfig payload - | set_path substitutes it into parameters.tools[i] - v -_agent: AgentConfig.from_params(...) parses the inlined WorkflowToolConfig +API generic resolver + service resolve step: fetch the referenced workflow revision | -resolve_tools(agent_config.tools): - | WorkflowToolResolver -> CallbackToolSpec(call_ref="workflow.") + ToolCallback + |-- runnable? -> keep the reference -> CallbackToolSpec(call_ref="workflow.") + | + the shared ToolCallback to the execute target + | + '-- not runnable -> resolve to a concrete `client` tool config (name/desc/input_schema) v -/run wire: customTools[i] = {kind:"callback", callRef:"workflow.", ...}, toolCallback +_agent: AgentConfig.from_params(...) parses the now-resolved tools | -runner dispatch (callback): model calls the tool -> POST /tools/call (or Daytona relay) +resolve_tools(agent_config.tools): callback spec for runnable; client spec for non-runnable v -API execute: parse workflow.* call_ref -> invoke workflow revision with args -> result +/run wire: customTools[i] = {kind:"callback", callRef:"workflow.", ...} OR {kind:"client", ...} | -result -> {call:{data:{content}}} -> back to the model +runner dispatch: + | callback -> model calls -> POST /tools/call -> API invokes the workflow revision + | client -> returned to the browser, fulfilled next turn + v +result -> back to the model ``` +## The seams + +| Seam | File | Change | +| --- | --- | --- | +| Strict schema arm | `sdks/python/agenta/sdk/utils/types.py` | add the embed-ref arm to `AgentConfigSchema.tools` (mirror `_SkillEmbedRefSchema`) so a referenced tool validates in the playground | +| Resolve step (runnable vs not) | service resolve step (where `@ag.embed`/references resolve) | decide runnable vs not for the referenced workflow; produce a callback-bound reference (runnable) or a concrete `client` config (non-runnable) | +| Runnable resolution branch | `sdks/python/agenta/sdk/agents/tools/resolver.py` + a platform resolver in `.../platform/` | a referenced runnable workflow resolves to a `CallbackToolSpec` + the shared `ToolCallback`, mirroring the gateway path | +| Server-side execute | `api/oss/src/apis/fastapi/tools/router.py` (+ core) | a `/tools/call`-style target that parses the `workflow.*` `call_ref`, invokes the referenced workflow revision with the model's arguments, and returns the result envelope | +| Embed resolver | `api/oss/src/core/embeds/utils.py` | **no change** — already walks `tools[]` | +| Wire | `services/agent/src/protocol.ts`, `sdks/python/agenta/sdk/agents/utils/wire.py`, golden fixtures | **no new field** — runnable rides as a `callback` spec, non-runnable as a `client` spec; only `call_ref` content is new | + +`call_ref` grammar for a runnable workflow: an opaque slug, e.g. `workflow.{slug}` or +`workflow.{slug}.{version}`. Distinct from the Composio 5-segment grammar +(`tools.{provider}.{integration}.{action}.{connection}`). The runner treats `call_ref` as +opaque; only the server-side parser must agree. `ResolvedToolSet` keeps its single shared +`tool_callback`; if both gateway and workflow tools are present, one endpoint routes by +`call_ref` prefix (`tools.*` vs `workflow.*`) — the smaller change, keeps the wire stable. + +## Out of scope (explicitly dropped from the old plan) + +- **No `workflow` tool variant.** A referenced workflow is just a workflow; no + `WorkflowToolConfig` in the discriminated union, no `"workflow"` `type` allowlist entry. +- **No platform tools as workflows.** Platform tools belong in the **existing tools + endpoints** (the same place gateway tools are added), not in the workflow catalog. Drop the + `_agenta.*` tool-workflow / `_validate_catalog` generalization direction entirely. (PR #4837 + review [3470356903](https://github.com/Agenta-AI/agenta/pull/4837#discussion_r3470356903).) +- **No Option A / Option B split.** There is one path; the only branch is runnable vs not. +- **`is_tool` flag** is a later, FE-only display hint — not built here. + ## Test plan -- **SDK unit:** `WorkflowToolConfig` parses (strict + loose coercion); `_ToolEmbedRefSchema` - validates an embed-ref `tools` entry; `WorkflowToolResolver` produces the expected - `CallbackToolSpec` + `ToolCallback`. -- **Schema:** `AgentConfigSchema` JSON Schema emits the embed-ref `oneOf` arm in `tools` - (mirror the skills schema test); `CATALOG_TYPES["agent_config"]` still dereferences. -- **Embed resolution (API):** an `@ag.embed` in `tools[i]` inlines into a `WorkflowToolConfig` - (Option B) or a concrete tool config (Option A); cycle/depth guards still hold. -- **Wire / golden:** a golden `/run` fixture with a workflow tool; `protocol.ts` Zod accepts - it as a `callback` spec. -- **Execute endpoint:** a `/tools/call` with a `workflow.*` `call_ref` invokes the revision - and returns the result; a user workflow's secrets/connections stay server-side. -- **Live matrix (agent-workflows-qa):** force the tool with an unguessable token across - pi_core / claude on local + Daytona + SDK; a pass proves the workflow ran server-side and +- **SDK unit:** the embed-ref `tools` arm validates (mirror the skills schema test); a + resolved runnable reference produces the expected `CallbackToolSpec` + `ToolCallback`; a + resolved non-runnable reference produces a `client` spec. +- **Schema:** `AgentConfigSchema` JSON Schema emits the embed-ref `oneOf` arm in `tools`; + `CATALOG_TYPES["agent_config"]` still dereferences. +- **Embed resolution (API/service):** a reference in `tools[i]` resolves to a callback-bound + reference (runnable) or a concrete `client` config (non-runnable); cycle/depth guards hold. +- **Wire / golden:** a golden `/run` fixture with a runnable workflow tool (a `callback` spec) + and one with a non-runnable (a `client` spec); `protocol.ts` Zod accepts both. +- **Execute endpoint:** a `/tools/call` with a `workflow.*` `call_ref` invokes the revision and + returns the result; the workflow's secrets/connections stay server-side. +- **Live matrix (agent-workflows-qa):** force a runnable workflow tool with an unguessable + token across pi_core / claude on local + Daytona + SDK; a pass proves it ran server-side and the result reached the model. Pin a green cell with agent-replay-test. ## Rollout -POC, off no flag needed for the schema arm (it is additive and the resolver already handles -it). The execute endpoint is new surface; gate platform tool workflows (B6) behind the -existing reserved-namespace trust, not a feature flag. Keep docs in sync in the same -implementation PR (tools.md, agent-config-schema.md, the interface inventory). +POC, no flag needed for the schema arm (additive; the resolver already handles it). The +execute endpoint is new server surface. Keep docs in sync in the same implementation PR +(`documentation/tools.md`, `interfaces/public-edge/agent-config-schema.md`, the interface +inventory). ## Build order (when implemented) -1. Option A schema arm (the shared, almost-free win) — `tools` accepts `@ag.embed`, - inlining concrete tool configs. Ships value immediately, validates the embedding half. -2. Option B variant + resolver + execute endpoint — tools-as-workflows proper. -3. Option B6 platform tool workflows (catalog validation generalization) if/when wanted. +1. Schema arm — `tools` accepts a workflow reference (the embed-ref arm), validates in the + playground. +2. Resolve step — decide runnable vs not; runnable → `CallbackToolSpec` + execute endpoint; + non-runnable → concrete `client` config. +3. (Later, FE) `is_tool` flag so referenced workflows surface in the tool picker. diff --git a/docs/design/agent-workflows/projects/embedref-tools/research.md b/docs/design/agent-workflows/projects/embedref-tools/research.md index c53f1aef8c..8e6195e817 100644 --- a/docs/design/agent-workflows/projects/embedref-tools/research.md +++ b/docs/design/agent-workflows/projects/embedref-tools/research.md @@ -73,20 +73,19 @@ The resolver callback routes a `workflow` reference to By step 3 it is a concrete tool config. The work is making steps 2-3 (and the schema) understand *what* it inlines into. -### The `_agenta.*` platform catalog short-circuit +### The `_agenta.*` platform catalog short-circuit (background only) `api/oss/src/core/workflows/platform_catalog.py` defines `PlatformWorkflowCatalog`, a code-defined, read-only set of platform workflows keyed by a reserved `_agenta.*` slug. `WorkflowsService.fetch_workflow_revision` calls `_resolve_platform_revision` *first*; a -reserved slug never falls through to Postgres. Each catalog version stores its payload under -`data.parameters.` — for skills that key is `skill` -(`parameters={"skill": skill_config...}`), which is why the skill embed's selector path is -`parameters.skill`. +reserved slug never falls through to Postgres. This is how the default skill embed resolves +(`_agenta.agenta-getting-started`). -One hard constraint to note: `_validate_catalog` currently validates **every** catalog -payload as a `SkillConfig` (`SkillConfig.model_validate(payload)`). To ship a *platform* tool -workflow, that validation must generalize per payload kind. (User-authored tool workflows -live in the DB and do not hit this validation.) +**Not relevant to this design's scope.** Per the PR #4837 review, platform *tools* do **not** +go in this catalog — they belong in the existing tools endpoints (like gateway). So this design +does **not** touch `_validate_catalog` (which today validates catalog payloads as `SkillConfig`) +and does not ship `_agenta.*` tool workflows. User-authored workflows referenced as tools live +in the DB and never hit this validation. ### Where the union lives (skills, the template to copy) @@ -141,47 +140,46 @@ tool_callback}`). The TS twin is `ResolvedToolSpec` in `services/agent/src/proto POSTs back to `/tools/call`** (directly, or via the Daytona file relay). Absent `kind` defaults to `callback`. -### Why `callback` is the right executor for an embedded-workflow tool +### Why `callback` is the right executor for a *runnable* workflow tool -A workflow tool is **server-executed**: calling it means invoking another Agenta workflow -revision, which lives behind the API and may itself use connections and secrets. That is -exactly the gateway tool's safety shape — the harness decides *which* tool and *with what -arguments*, the service runs it, and no credential reaches the sandbox. So an -embedded-workflow tool should resolve to a `CallbackToolSpec`: +The branch that matters is **runnable vs non-runnable** (see [plan.md](plan.md)). The taxonomy +already has a home for each: -- The runner needs **no new `kind`** — `callback` already dispatches to `callAgentaTool`, - works under the Daytona file relay, and is delivered to both Pi (native) and Claude (the - `agenta-tools` MCP bridge). -- The only thing that differs from a gateway tool is the `call_ref` grammar and the - server-side execute target: instead of running a Composio action, the service invokes a +- A **runnable** workflow tool is **server-executed**: calling it means invoking another Agenta + workflow revision, which lives behind the API and may itself use connections and secrets. That + is exactly the gateway tool's safety shape — the harness decides *which* tool and *with what + arguments*, the service runs it, and no credential reaches the sandbox. So it resolves to a + `CallbackToolSpec`. The runner needs **no new `kind`** — `callback` already dispatches to + `callAgentaTool`, works under the Daytona file relay, and is delivered to both Pi (native) and + Claude (the `agenta-tools` MCP bridge). The only difference from a gateway tool is the + `call_ref` grammar and the execute target: instead of a Composio action, the service invokes a workflow revision. +- A **non-runnable** (client) workflow tool fits the existing **`client`** executor: the resolve + step turns the reference into a concrete `client` tool config, and at run time the runner + returns a `client` spec for the browser to fulfill next turn (`models.py:206` — + `kind: "client"`). No callback, no server-side execute. ## Part 3 — The seams to touch (summary) | Seam | File | Change | | --- | --- | --- | | Strict schema embed arm | `sdks/python/agenta/sdk/utils/types.py` | add `_ToolEmbedRefSchema`, make `AgentConfigSchema.tools` a `Union[ToolConfig-twin, _ToolEmbedRefSchema]` (mirror skills) | -| New tool variant | `sdks/python/agenta/sdk/agents/tools/models.py` | add `WorkflowToolConfig(type="workflow")` to the union, carrying the workflow ref + tool surface (name, description, input schema) | -| Loose coercion allowlist | `sdks/python/agenta/sdk/agents/tools/compat.py` | add `"workflow"` to the accepted `type` set | -| Resolution branch | `sdks/python/agenta/sdk/agents/tools/resolver.py` + a new platform resolver in `.../platform/` | resolve a `WorkflowToolConfig` to a `CallbackToolSpec` + a `ToolCallback` to the new execute endpoint | +| Resolve step (runnable vs not) | service resolve step (where references resolve) | decide runnable vs not; runnable → keep the reference for callback resolution; non-runnable → resolve to a concrete `client` tool config | +| Runnable resolution branch | `sdks/python/agenta/sdk/agents/tools/resolver.py` + a platform resolver in `.../platform/` | resolve a referenced runnable workflow to a `CallbackToolSpec` + a `ToolCallback` to the new execute endpoint (mirror gateway) | | Server-side execute | `api/oss/src/apis/fastapi/tools/router.py` (+ core) | a `/tools/call`-style target that invokes the referenced workflow revision and returns the result | | Embed resolver | `api/oss/src/core/embeds/utils.py` | **no change** — already walks `tools[]` | -| Platform catalog validation | `api/oss/src/core/workflows/platform_catalog.py` | generalize `_validate_catalog` IF we ship a platform tool workflow (otherwise unchanged for user workflows) | -| Wire | `services/agent/src/protocol.ts`, `sdks/python/agenta/sdk/agents/utils/wire.py`, golden fixtures | **no new field** if it resolves to an existing `callback` spec; only the `call_ref` value differs | -| Docs | `documentation/tools.md`, `interfaces/public-edge/agent-config-schema.md`, the interface inventory | document the new variant + the embed arm | +| Wire | `services/agent/src/protocol.ts`, `sdks/python/agenta/sdk/agents/utils/wire.py`, golden fixtures | **no new field** — runnable rides as a `callback` spec, non-runnable as a `client` spec; only the `call_ref` content is new | +| Docs | `documentation/tools.md`, `interfaces/public-edge/agent-config-schema.md`, the interface inventory | document the embed arm + the runnable-vs-not behavior | + +No `WorkflowToolConfig` variant, no `compat.py` `"workflow"` allowlist entry, no +platform-catalog change — all dropped per the PR #4837 review. ## Open research questions (carried into the plan) -1. **Embed-as-content vs reference-as-tool.** The selector path could inline either a - *concrete tool config* (e.g. a `code`/`gateway` config stored in the workflow's - parameters — pure reuse, no new variant, no new executor) OR a *workflow reference* that - becomes a new `workflow` tool variant (a tool that, when called, runs the workflow). Both - are "embedref like skills." They are very different in cost and meaning. See - [status.md](status.md#key-decision). +1. **Where the runnable/not decision is made** — confirm it is the service resolve step (where + the referenced workflow is fetched), so the SDK/runner stay schema-driven. 2. **What does invoking the workflow mean** — call `/workflows/.../invoke` with the model's arguments as inputs, and map the workflow output back as the tool result? What is the input/output contract between a tool call and a workflow invoke? -3. **The `call_ref` grammar** for a workflow tool (today's 5-segment gateway grammar is - Composio-specific and parsed in both `compat.py` and the API router). -4. **Platform tool workflows** — do we want `_agenta.*` platform tools (needs the catalog - validation generalization), or only user-authored DB workflows at first? +3. **The `call_ref` grammar** for a runnable workflow tool (today's 5-segment gateway grammar + is Composio-specific and parsed in both `compat.py` and the API router). diff --git a/docs/design/agent-workflows/projects/embedref-tools/status.md b/docs/design/agent-workflows/projects/embedref-tools/status.md index 6008654ea3..e071d70b6c 100644 --- a/docs/design/agent-workflows/projects/embedref-tools/status.md +++ b/docs/design/agent-workflows/projects/embedref-tools/status.md @@ -6,74 +6,100 @@ This is the source of truth for the project's progress, decisions, and open ques - **Phase:** IMPLEMENTED (the lgtm'd two-syntax design #4837). Spun from PR #4821 review comment [3469653315](https://github.com/Agenta-AI/agenta/pull/4821#discussion_r3469653315). -- **Docs:** README, context, research, plan, status written. Grounded in current code - (skills embed resolver, the tool taxonomy, the platform catalog) with file paths. -- **Next:** user picks the direction (Option A vs B, and the open questions below), then this - goes through `implement-feature`. - -## Key decision - -**An embed in `tools` can mean two different things; the design recommends supporting both, -A first then B.** - -- **Option A — embed-as-content (almost free):** the embed inlines a *concrete, already- - supported tool config* (`gateway`/`code`/`client`) that an author stored inside a workflow. - Pure reuse. The only code change is the strict-schema embed-ref arm; the generic resolver, - `resolve_tools`, the wire, and the runner are all untouched. -- **Option B — reference-as-tool (the real "tools as workflows"):** the embed references a - workflow that *becomes* a callable tool. Calling it invokes the workflow revision - server-side and returns its output. Fits the existing `callback` executor (like a gateway - tool), so the runner needs no new `kind`. Adds a `workflow` tool variant, a resolver - branch, and a server-side execute endpoint. - -**Recommendation:** ship Option A immediately (it is the literal "embedref like skills" and -costs one schema arm), and build Option B as the substantive feature. Option A is a clean -stepping stone, not a dead end — both share the same embed-ref arm. - -**Why `callback`, not a new executor (Option B):** a workflow tool is server-executed and may -use connections/secrets, which is exactly the gateway tool's safety shape. Resolving to a -`CallbackToolSpec` keeps every credential server-side and reuses the runner's existing -callback delivery (direct, Daytona relay, Pi native, Claude `agenta-tools` bridge). +- **Docs:** README, context, research, plan, status written, then **revised per the author's + review on PR #4837** to the simplified runnable-vs-not model (Option A/B split, the + `workflow` tool variant, and platform-tools-as-workflows all removed). +- **Built (this slice):** + - SDK marker + config: `AG_REFERENCE_MARKER` and `ReferenceToolConfig` + (`type: "reference"`, `slug`/`version`/`name`/`description`/`input_schema`, `.call_ref` + `workflow.{slug}[.{version}]`) in `tools/models.py`; `compat.py` coerces the kept + `@ag.reference` marker into it. + - `resolve_tools` mapping: a new `WorkflowToolResolver` port + `AgentaWorkflowToolResolver` + platform adapter (`platform/workflow.py`) build a `CallbackToolSpec` + the shared + `ToolCallback`; `ToolResolver` partitions reference configs and reconciles the single + callback with gateway. The generic resolver stays tool-agnostic. + - Generic resolver "leave it" guard: `AG_REFERENCE_KEY` in `api/oss/src/core/embeds/utils.py` + — all three finders treat a kept `@ag.reference` node as opaque (the SDK `_has_embed_markers` + already ignores it, so an embed-free reference simply passes through). + - Strict schema arms: `_ToolEmbedRefSchema` + `_ToolReferenceSchema` on + `AgentConfigSchema.tools` (a union) in `utils/types.py`. + - Server-side execute: `/tools/call` routes a `workflow.*` call_ref to `_call_workflow_tool` + (`api/oss/src/apis/fastapi/tools/router.py`), which invokes the workflow revision via + `WorkflowsService.invoke_workflow` (wired into `ToolsRouter` in `entrypoints/routers.py`). + - Wire: UNCHANGED. A reference rides as a `callback` spec, an embed as a `client` spec; only the + `call_ref` content (`workflow.*`) is new. Golden fixtures untouched. + - Tests: SDK (parsing/models/resolver/platform/catalog) + API (embeds leave-it + router + execute branch). Live end-to-end DEFERRED to the dedicated embedref live QA (after the gate). +- **Next:** CTO (JP) review of the PR; live end-to-end QA. + +## Design + +**A tool is just a referenced workflow.** `tools[i]` points at a workflow (by reference, not by +inlining its config). Any workflow type qualifies — agent, completion, channel, chain. There is +**no `workflow` tool variant** and **no "tool workflow" type**. The only branch is **runnable +vs non-runnable**, decided server-side in the resolve step where the referenced workflow is +known: + +- **Runnable** (executable: completion / agent / channel / chain). You reference it *because + you want to call it*. It resolves to the existing **`callback`** executor — a + `CallbackToolSpec` whose `call_ref` encodes the workflow identity, plus the shared + `ToolCallback` to a server-side execute endpoint. The model's call routes back, the service + invokes the workflow revision, the result returns. Connections/secrets stay server-side, + exactly like a gateway tool. **No new runner `kind`.** +- **Non-runnable** (a client tool). Referencing-to-call does not apply. The resolve step + **resolves the reference into its value** — a concrete `client` tool config — and at run time + it is the existing `client` path (fulfilled in the browser next turn). + +Unifying rule: reference everything as a tool; **in the sidecar, if it is runnable, run it; if +it is not, return its schema.** + +**Why `callback` for the runnable case:** a runnable workflow tool is server-executed and may +use connections/secrets — exactly the gateway tool's safety shape. Resolving to a +`CallbackToolSpec` keeps every credential server-side and reuses the runner's existing callback +delivery (direct, Daytona relay, Pi native, Claude `agenta-tools` bridge). + +**Explicitly dropped from the first design** (per the author's PR #4837 review): + +- the Option A / Option B framing (one path, branch on runnable); +- the `WorkflowToolConfig` variant / the `"workflow"` `type` allowlist entry (a tool is just a + workflow); +- **platform tools as workflows** — they go in the **existing tools endpoints** (like gateway), + not the workflow catalog, so the `_validate_catalog` generalization is gone. ## Settled by research - The `@ag.embed` resolver is **generic and already walks `tools[]`** — no resolver change is - needed for embedding. (`ResolverMiddleware` + `api/oss/src/core/embeds/utils.py`.) -- Embed resolution runs **before** `AgentConfig.from_params` and `resolve_tools`, so by the - time tools resolve, the embed is already a concrete config. -- The skills schema arm (`_SkillEmbedRefSchema`) and the `_agenta.*` platform catalog are the - exact templates to mirror. -- The platform catalog `_validate_catalog` hard-codes `SkillConfig`; shipping a *platform* - tool workflow (not user workflows) would need that generalized. + needed for referencing. (`ResolverMiddleware` + `api/oss/src/core/embeds/utils.py`.) This is + the load-bearing finding and it survives the simplification. +- Reference resolution runs **before** `AgentConfig.from_params` and `resolve_tools`, so by the + time tools resolve, the reference is already concrete. +- The skills schema arm (`_SkillEmbedRefSchema`) is the exact template to mirror. ## Open questions for the user -1. **Option A, Option B, or both?** A is nearly free and matches the comment literally; B is - the deeper feature the comment hints at ("creating tools as workflows"). Recommendation: - both, A first. -2. **Selector target for Option B** — does the tool workflow store a ready tool surface under - `parameters.tool` (explicit, mirrors skills, recommended) or do we infer the tool surface - (name/description/input schema) from the workflow's metadata and declared inputs? -3. **Tool-call to workflow-invoke contract** — how do the model's tool arguments map to the +1. **Where the runnable/not decision lives, precisely** — confirm it is the service resolve + step (where the referenced workflow is fetched), so the SDK/runner stay schema-driven. +2. **Tool-call to workflow-invoke contract** — how do the model's tool arguments map to the workflow's invoke inputs, and how does the workflow output map back to the tool result? - Free-form passthrough, or a declared input/output schema on the tool workflow? -4. **`call_ref` grammar for workflow tools** — `workflow.{slug}` / `workflow.{slug}.{version}`? - Today's gateway grammar (`tools.{provider}.{integration}.{action}.{connection}`) is - Composio-specific and parsed in two places; a workflow tool needs its own opaque slug. -5. **Single shared callback endpoint vs per-spec callbacks** — `ResolvedToolSet` holds one - `tool_callback`. With both gateway and workflow tools present, do we route one endpoint by + Free-form passthrough, or a declared input/output schema? +3. **`call_ref` grammar for runnable workflow tools** — `workflow.{slug}` / + `workflow.{slug}.{version}`? Today's gateway grammar + (`tools.{provider}.{integration}.{action}.{connection}`) is Composio-specific and parsed in + two places; a workflow tool needs its own opaque slug. +4. **Single shared callback endpoint vs per-spec callbacks** — `ResolvedToolSet` holds one + `tool_callback`. With both gateway and workflow tools present, route one endpoint by `call_ref` prefix (smaller change, recommended) or grow the wire to per-spec callbacks? -6. **Platform tool workflows now or later?** Generalizing `_validate_catalog` to ship - `_agenta.*` tools is optional; user-authored DB tool workflows do not need it. -7. **Approval / render axes** — a workflow tool can carry `needs_approval` and `render` like - any tool; confirm there is no special handling wanted (default: they compose as usual). +5. **`is_tool` FE flag** — confirm it is deferred (later, display-only so referenced workflows + surface in the tool picker) and not part of this slice. +6. **Approval / render axes** — a referenced tool can carry `needs_approval` and `render` like + any tool; confirm no special handling is wanted (default: they compose as usual). ## Risks / watch-fors - **One callback channel.** The single `tool_callback` is a real constraint if mixing tool - types; the prefix-routing answer (Q5) avoids a wire change. -- **Embed must reference the artifact** (`workflow.slug`), not a bare revision slug with no - version (returns 500) — same gotcha skills have. + types; the prefix-routing answer (Q4) avoids a wire change. +- **Reference the artifact** (`workflow.slug`), not a bare revision slug with no version + (returns 500) — same gotcha skills have. - **Two models, one contract.** The strict `AgentConfigSchema` and the permissive runtime `AgentConfig` must move together (and a golden fixture), per agent-config-schema.md's "watch for when changing." From a03582414bd42dedf3cf17c41561ea672ce80cc5 Mon Sep 17 00:00:00 2001 From: Mahmoud Mabrouk Date: Thu, 25 Jun 2026 12:40:44 +0200 Subject: [PATCH 3/3] docs(agent): revise EmbedRef tools to two-syntax (embed vs reference) per PR #4837 review Adopt the author's iteration-3 direction (#4837 r3473648119): two syntaxes, the syntax decides the behavior. @ag.embed inlines a value (-> client spec); a new top-level @ag.reference is kept by the generic resolver (-> resolve_tools builds a CallbackToolSpec). The generic resolver stays tool-agnostic; all tool-specific mapping lives in resolve_tools. Also addresses CodeRabbit: concrete client tool config wording (context), closes the resolve-step open question (README/status), resolves the keep-the-reference-vs-inlined tension via the two syntaxes (research), and labels the resolution-path diagram fence (plan, MD040). Claude-Session: https://claude.ai/code/session_01GYo3UEfvsZpncagqb28Mbc --- .../projects/embedref-tools/README.md | 67 +++--- .../projects/embedref-tools/context.md | 93 ++++---- .../projects/embedref-tools/plan.md | 203 ++++++++++-------- .../projects/embedref-tools/research.md | 105 +++++---- .../projects/embedref-tools/status.md | 108 ++++++---- 5 files changed, 343 insertions(+), 233 deletions(-) diff --git a/docs/design/agent-workflows/projects/embedref-tools/README.md b/docs/design/agent-workflows/projects/embedref-tools/README.md index c61ae7c907..e53e6d33ec 100644 --- a/docs/design/agent-workflows/projects/embedref-tools/README.md +++ b/docs/design/agent-workflows/projects/embedref-tools/README.md @@ -1,11 +1,21 @@ # EmbedRef tools (tools-as-workflows) -Index for the design workspace that lets the agent config `tools` field **reference a -workflow**, the same way `skills` already does. A tool is just a workflow — any workflow -(agent, completion, channel, chain) can be referenced as a tool. What happens when the model -calls it depends only on whether that workflow is **runnable** (server-side callback execute, -like gateway) or **non-runnable** (resolved into a concrete `client` tool, fulfilled in the -browser). +Index for the design workspace that lets the agent config `tools` field point at a **workflow**, +the same way `skills` already does. A tool is just a workflow — any workflow (agent, completion, +channel, chain) can be used as a tool. + +The author picks one of **two syntaxes**, and the syntax decides the behavior: + +- **`@ag.reference`** (new) — keep the reference in the config; the workflow stays a *reference* + because you want to **call** it. At tool-resolution time it becomes a server-side `callback` + call spec (the service runs the workflow revision, like a gateway tool). +- **`@ag.embed`** (existing) — resolve the reference **to its value** and inline it. For a tool + this inlines a concrete `client` tool config; at tool-resolution time it becomes a `client` + spec (fulfilled in the browser). + +The generic resolver does not know about tools. It only knows the two syntaxes (inline-the-value +vs leave-the-reference). The tool-specific logic — turn a kept reference into a callback spec, +turn an embedded value into a client spec — lives in `resolve_tools`. Spun out of PR #4821 review comment [3469653315](https://github.com/Agenta-AI/agenta/pull/4821#discussion_r3469653315) on @@ -18,32 +28,35 @@ pre-production: no back-compat is required. ## Files - [context.md](context.md) — why this exists, goals, non-goals, the reviewer's ask, and the - one branch that matters: runnable vs non-runnable. -- [research.md](research.md) — how `skills` referencing works today (the generic `@ag.embed` + two syntaxes (embed vs reference) and what each does. +- [research.md](research.md) — how `skills` embedding works today (the generic `@ag.embed` resolver, the `ResolverMiddleware`), the tool taxonomy (type/executor model), and the exact - seams to mirror, with file paths. The load-bearing finding: the resolver is already generic - and already walks `tools[]`. -- [plan.md](plan.md) — the simplified design: the embed-ref schema arm, the single - runnable-vs-not branch (callback execute vs resolve-to-`client`), the server-side execute - endpoint, the wire, tests, and rollout. Explicitly drops the old Option A/B split, the + seams to mirror, with file paths. The load-bearing finding: the resolver is already generic; + it walks `tools[]` and handles embeds, and a second `@ag.reference` syntax stays just as + generic (leave-the-reference). +- [plan.md](plan.md) — the design: the two-syntax model, the schema arms, the `resolve_tools` + branch (kept reference → callback spec / embedded value → client spec), the server-side + execute endpoint, the wire, tests, and rollout. Explicitly drops the old Option A/B split, the `workflow` tool variant, and platform-tools-as-workflows. - [status.md](status.md) — current state, the settled design, and the remaining open questions. ## One-paragraph answer to the reviewer -Yes, and the referencing half is almost free. The `@ag.embed` resolver is **already generic**: -the `ResolverMiddleware` walks the whole `parameters` tree (lists included) and resolves every -reference server-side *before* `AgentConfig.from_params` parses the config or `resolve_tools` -runs, so a reference placed inside `tools[i]` already resolves with **zero resolver changes**. -The real design is two things, and one branch. The two things: (1) make the strict -`AgentConfigSchema.tools` accept the embed-ref arm (mirroring `_SkillEmbedRefSchema`), so a -referenced tool validates in the playground; (2) a server-side execute endpoint that invokes a -referenced workflow revision. The one branch is **runnable vs non-runnable**, decided in the -service / resolve step: a **runnable** workflow resolves to the existing `callback` executor -(like a gateway tool — a `CallbackToolSpec` whose `call_ref` encodes the workflow identity, run -server-side, no new runner `kind`); a **non-runnable** (client) workflow is **resolved into its -value** — a concrete `client` tool config — and fulfilled the existing client way. There is no -`workflow` tool variant (a tool is just a workflow; any type qualifies), and platform tools -stay in the existing tools endpoints, not the workflow catalog. See +Yes, and the referencing half reuses the generic resolver. There are **two syntaxes** an author +can put inside `tools[i]`: `@ag.embed` (existing — the resolver inlines the referenced value) +and `@ag.reference` (new — the resolver leaves the reference in place). The `ResolverMiddleware` +and the API resolver stay generic: they only know "inline this value" vs "leave this reference," +nothing about tools. The tool-specific logic lives in **`resolve_tools`**, which runs after the +config is parsed: a **kept `@ag.reference`** becomes a `callback` call spec (a `CallbackToolSpec` +whose `call_ref` encodes the workflow identity; the service runs the referenced workflow revision +server-side, like a gateway tool — no new runner `kind`); an **`@ag.embed`** value that resolved +to a concrete `client` tool config becomes a `client` spec (fulfilled in the browser). The +author's syntax choice (reference vs embed) maps to runnable-vs-not: you *reference* a runnable +workflow because you want to call it; you *embed* a non-runnable (client) tool because it is a +value. The real new code is: (1) two embed/reference arms on the strict `AgentConfigSchema.tools` +(mirroring `_SkillEmbedRefSchema`); (2) a `resolve_tools` branch that builds a callback spec from +a kept reference; (3) a server-side execute endpoint that invokes a referenced workflow revision. +There is no `workflow` tool variant (a tool is just a workflow; any type qualifies), and platform +tools stay in the existing tools endpoints, not the workflow catalog. See [the design](status.md#design) for the details. diff --git a/docs/design/agent-workflows/projects/embedref-tools/context.md b/docs/design/agent-workflows/projects/embedref-tools/context.md index 5683c39778..a646190e96 100644 --- a/docs/design/agent-workflows/projects/embedref-tools/context.md +++ b/docs/design/agent-workflows/projects/embedref-tools/context.md @@ -9,9 +9,10 @@ not symmetric today. `SkillConfig`, OR drop an `@ag.embed` reference to a workflow and the backend inlines that workflow's content into a concrete `SkillConfig` before the runner sees it. The default config ships exactly such an embed (the `_agenta.agenta-getting-started` platform skill). + A skill is always passive content, so embedding (inline the value) is the only mode it needs. - **`tools`** accepts only the four concrete variants `ToolConfig = builtin | gateway | code - | client`. There is no embed-ref arm, so a tool cannot be authored as a workflow and reused - by reference. + | client`. There is no embed/reference arm, so a tool cannot be authored as a workflow and + reused by pointing at it. PR #4821 review comment [3469653315](https://github.com/Agenta-AI/agenta/pull/4821#discussion_r3469653315) asks to @@ -27,15 +28,20 @@ channel, chain — there is no special "tool workflow" type. ## Goals -- Make `tools` accept a workflow **reference** (the `@ag.embed` arm), mirroring the `skills` - shape. -- Define the one branch that matters: **runnable vs non-runnable**, and how each is handled - (runnable → server-side callback execute, like gateway; non-runnable → resolve-to-value plus - the existing client-tool handling). -- Keep the runner free of a new `kind`: runnable rides as a `callback` spec, non-runnable as a +- Make `tools` accept a workflow via the **same two syntaxes** skills can use plus one more: + `@ag.embed` (inline the value) and a new `@ag.reference` (keep the reference). Mirror the + `skills` schema shape for the embed arm and add a reference arm. +- Define the model: **the author's syntax decides the behavior.** `@ag.reference` → a kept + reference → a server-side `callback` call spec (the service runs the referenced workflow + revision, like gateway). `@ag.embed` → an inlined value → a `client` spec. +- Keep the **generic resolver tool-agnostic.** It only does "inline the value" (embed) vs + "leave the reference" (reference). It learns nothing about tools. +- Put the **tool-specific logic in `resolve_tools`**: a kept reference becomes a callback spec, + an embedded value becomes a client spec. +- Keep the runner free of a new `kind`: a reference rides as a `callback` spec, an embed as a `client` spec. -- Keep secrets and connection auth server-side for the runnable case, the same safety property - gateway tools have. +- Keep secrets and connection auth server-side for the reference (callback) case, the same + safety property gateway tools have. ## Non-goals @@ -49,38 +55,43 @@ channel, chain — there is no special "tool workflow" type. the tool picker; it is noted, not designed here. - Building the workflow-authoring UI for tools. This design assumes a workflow revision exists; producing it is a separate surface. -- Changing the generic embed resolver. It already walks `tools[]`; this design relies on - that, it does not modify it. -- A new vault or connection concept. Runnable workflow tools reuse the existing named-secret - and connection resolution. +- Changing the generic resolver's contract. It already inlines `@ag.embed` and walks `tools[]`; + the only addition is teaching it to **leave** an `@ag.reference` in place (a "leave it" + branch, not tool-aware logic). It does not gain any tool knowledge. +- A new vault or connection concept. Referenced (callback) workflow tools reuse the existing + named-secret and connection resolution. - MCP. `mcp_servers` is a sibling field with its own deferral; out of scope here. ## The reviewer's ask, restated -Add an embed-ref arm to `tools` so a tool can be created as a workflow and referenced. Two -mechanisms must meet: the **referencing** mechanism (already exists, generic) and the -**tool-ness** mechanism (the referenced workflow has to end up as a tool the agent can call). - -## The one branch that matters: runnable vs non-runnable - -A skill is always passive content (markdown + files; the model reads it, nothing executes). -A referenced workflow is not uniform — it can be runnable or not — and that is the whole -design: - -- **Runnable** (a completion, an agent, a channel, a chain — anything the platform can - invoke). You **reference** it because you want to **call** it. When the model calls the - tool, the call routes server-side and Agenta **invokes the workflow revision**, exactly like - a gateway tool: the sidecar/runner relays the call back, the service runs it, the result - returns to the model. Execution and any connections/secrets stay server-side. This resolves - to the existing `callback` executor — **no new runner `kind`**. - -- **Non-runnable** (a client tool — fulfilled in the browser, nothing to execute - server-side). Referencing-to-call does not apply. It is handled the way client tools are - handled today: the resolve step in the service **resolves the reference into its value** (a - concrete `client` tool config), and at run time the model's call is fulfilled client-side - next turn, the existing `client` path. - -So **what you reference decides the behavior**. The runnable/not decision is made in the -service / the resolve step, where the referenced workflow is known. A unifying way to say it: -reference everything as a tool, and **in the sidecar, if it is runnable, run it; if it is not, -return its schema**. See [plan.md](plan.md) for the concrete shape. +Add an embed/reference arm to `tools` so a tool can be created as a workflow and pointed at. Two +mechanisms must meet: the **pointing** mechanism (the resolver, generic, now with two syntaxes) +and the **tool-ness** mechanism (in `resolve_tools`: the pointed-at workflow has to end up as a +tool the agent can call or a client tool the browser fulfills). + +## The two syntaxes: embed vs reference + +A skill is always passive content (markdown + files; the model reads it, nothing executes), so +it only ever needs **embedding** — inline the value. A tool is not uniform: it can be a runnable +workflow you want to **call**, or a non-runnable client tool that is just a **value**. So tools +need both syntaxes, and **the syntax the author writes decides the behavior**: + +- **`@ag.reference`** (new) — the resolver **leaves the reference in the config**. You reference + a workflow *because you want to call it* (a completion, an agent, a channel, a chain — + anything the platform can run). `resolve_tools` turns the kept reference into a + `CallbackToolSpec`: when the model calls the tool, the call routes server-side and Agenta + **invokes the workflow revision**, exactly like a gateway tool — the sidecar/runner relays the + call back, the service runs it, the result returns to the model. Execution and any + connections/secrets stay server-side. This rides on the existing `callback` executor — **no + new runner `kind`**. + +- **`@ag.embed`** (existing) — the resolver **inlines the value**. You embed when the referenced + thing is a non-runnable client tool: there is nothing to call server-side, so the resolver + resolves the reference into its value (**a concrete `client` tool config**). `resolve_tools` + sees that concrete config and produces a `client` spec, and at run time the model's call is + fulfilled client-side next turn — the existing `client` path. + +So **the syntax decides the behavior**, and the choice is made by the author at config time, not +inferred server-side. The decision boundary is clean: the **generic resolver** does inline-vs-leave; +**`resolve_tools`** does the tool-specific mapping (kept reference → callback spec; embedded value +→ client spec). See [plan.md](plan.md) for the concrete shape. diff --git a/docs/design/agent-workflows/projects/embedref-tools/plan.md b/docs/design/agent-workflows/projects/embedref-tools/plan.md index 13f40b4c19..e68475fcb2 100644 --- a/docs/design/agent-workflows/projects/embedref-tools/plan.md +++ b/docs/design/agent-workflows/projects/embedref-tools/plan.md @@ -1,30 +1,52 @@ # Plan -Let the agent config `tools` field accept a **reference to a workflow**, so any workflow can -be used as a tool. POC / pre-production: no back-compat. +Let the agent config `tools` field point at a **workflow** via one of two syntaxes — embed or +reference — so any workflow can be used as a tool. POC / pre-production: no back-compat. -## The model (one path, split by runnable vs not) +## The model: two syntaxes, the syntax decides the behavior The old plan had two competing options (embed-as-content vs reference-as-tool) and a special -`workflow` tool variant. That was over-built. The simplified model the author landed on: - -**A tool is just a referenced workflow.** You point `tools[i]` at a workflow (by reference, -not by inlining its config). What happens when the model calls it depends only on whether that -workflow is **runnable** (executable) or **not**. - -- **Runnable** (a completion, an agent, a channel, a chain — anything the platform can - invoke): you *reference* it because you want to *call* it. The model's call routes - server-side and Agenta **runs the workflow revision**, exactly like a gateway tool. The - sidecar relays the call back; the service invokes; the result returns to the model. Secrets - and connections the workflow needs stay server-side. -- **Non-runnable** (a client tool — fulfilled in the browser, nothing to execute - server-side): referencing-to-call does not apply. It is handled the way client tools are - handled today. Its value is **resolved/embedded into the config** server-side (the resolve - step in the service), and at run time the model's call is fulfilled client-side next turn, - the existing `client` path. - -So **what you reference decides the behavior**: runnable → server-side callback execute; -non-runnable → resolve-to-value + the existing client handling. +`workflow` tool variant. That was over-built. An earlier revision then said "the resolver inlines +everything and the runnable/not decision is made server-side in a resolve step." Per the author's +review ([3473648119](https://github.com/Agenta-AI/agenta/pull/4837#discussion_r3473648119)) that +is also not the right shape: don't infer the behavior server-side by inspecting the target. Use +**two syntaxes** and let the author's choice decide. + +**A tool is just a workflow.** You point `tools[i]` at a workflow with one of two markers: + +- **`@ag.reference`** (new) — keep the reference. You reference a workflow *because you want to + call it* (a completion, an agent, a channel, a chain — anything the platform can run). The + generic resolver **leaves the reference in the config** (it does not inline it). `resolve_tools` + later turns the kept reference into a `CallbackToolSpec`: the model's call routes server-side + and Agenta **runs the workflow revision**, exactly like a gateway tool. The sidecar relays the + call back; the service invokes; the result returns to the model. Secrets and connections the + workflow needs stay server-side. +- **`@ag.embed`** (existing) — inline the value. You embed when the referenced thing is a + non-runnable client tool: there is nothing to call server-side. The generic resolver + **resolves the reference into its value** — a concrete `client` tool config (name, description, + input schema). `resolve_tools` sees that concrete config and produces a `client` spec; at run + time the model's call is fulfilled client-side next turn, the existing `client` path. + +So **the syntax decides the behavior**: `@ag.reference` → server-side callback execute; +`@ag.embed` → inline-to-value + the existing client handling. The author makes this choice at +config-authoring time. It maps to runnable-vs-not (reference a runnable workflow you want to call; +embed a non-runnable client tool that is a value), but the design does **not** inspect the target +to decide — the marker is authoritative. + +### The decision boundary: generic resolver vs `resolve_tools` + +The clean separation the author asked for: + +- The **generic resolver** (SDK `ResolverMiddleware` + the API embed resolver) knows only two + operations and **nothing about tools**: inline the value (`@ag.embed`) or leave the reference + (`@ag.reference`). It is the same recursive walker that already handles skills; it gains one + "leave it" branch for the reference syntax. +- **`resolve_tools`** (where tool configs are already partitioned by type) owns all + tool-specific logic: a kept `@ag.reference` becomes a `CallbackToolSpec` + the shared + `ToolCallback`; an `@ag.embed`-resolved concrete `client` config becomes a `client` spec. + +This keeps embedding/referencing reusable for any field (skills, tools, future fields) while the +"these are tools" knowledge stays in one place. ### Any workflow qualifies — there is no "tool workflow" type @@ -37,55 +59,58 @@ frontend can list it in the tool picker. It is a display hint; it changes no run ### One unifying rule for the sidecar -Reference everything as a tool. **In the sidecar, if the referenced thing is runnable, run it; -if it is not runnable, return its schema** (instead of executing). That single rule covers -both cases without branching the wire by tool kind: - -- runnable → the callback executes the workflow and returns the result; -- non-runnable → the callback (or the resolve step) returns the schema/value, and the model is - fulfilled the client way. - -## What the embed inlines into - -The `@ag.embed` resolver is **already generic** and already walks `tools[]` (see -[research.md](research.md)) — this is the one genuinely-useful research finding and it still -holds. Embed resolution runs in the SDK `ResolverMiddleware` *before* -`AgentConfig.from_params` parses the config and *before* `resolve_tools` runs. So a reference -placed in `tools[i]` is resolved with **zero resolver changes**. - -The split decides what the resolve step produces: - -- **Runnable** → keep the reference. The config carries a workflow reference (slug, optional - version) plus the model-facing surface (name, description, input schema). It resolves to the - existing `callback` executor: a `CallbackToolSpec` whose `call_ref` encodes the workflow - identity, plus the shared `ToolCallback` pointing at a server-side execute target. The runner - needs **no new `kind`** — `callback` already dispatches everywhere (direct, Daytona relay, Pi - native, the Claude `agenta-tools` bridge). -- **Non-runnable** → resolve to a value. The resolve step in the service turns the reference - into a concrete `client` tool config (name, description, input schema). At run time it is the - existing `client` path: the runner returns a `client` spec, the browser fulfills it next +Point at everything as a tool. **In the sidecar, if the entry is a kept reference, run it (the +callback executes the referenced workflow); if it is an embedded value, it is a concrete `client` +tool config and the model is fulfilled the client way.** That single rule covers both cases +without branching the wire by tool kind: + +- `@ag.reference` → the callback executes the workflow and returns the result; +- `@ag.embed` → the inlined `client` config rides as a `client` spec and is fulfilled in the + browser. + +## What each syntax produces + +The resolver is **already generic** and already walks `tools[]` (see [research.md](research.md)) +— this is the one genuinely-useful research finding and it still holds. Embed resolution runs in +the SDK `ResolverMiddleware`, which today inlines every `@ag.embed`. The one resolver addition is +a "leave it" branch so an `@ag.reference` passes through untouched. `resolve_tools` then maps each +form: + +- **`@ag.reference`** → a kept reference. The config carries a workflow reference (slug, optional + version) plus the model-facing surface (name, description, input schema). `resolve_tools` turns + it into a `CallbackToolSpec` whose `call_ref` encodes the workflow identity, plus the shared + `ToolCallback` pointing at a server-side execute target. The runner needs **no new `kind`** — + `callback` already dispatches everywhere (direct, Daytona relay, Pi native, the Claude + `agenta-tools` bridge). +- **`@ag.embed`** → an inlined value. The resolver resolves the reference into a concrete + `client` tool config (name, description, input schema) *before* `resolve_tools` runs, so + `resolve_tools` sees a plain `ClientToolConfig` and produces a `client` spec. At run time it is + the existing `client` path: the runner returns a `client` spec, the browser fulfills it next turn. No callback, no server-side execute. -Where the runnable/not decision is made: in the **service / the embed (resolve) step**, when -the reference is resolved. That is where we know what the referenced workflow is. +Where the tool-specific decision is made: in **`resolve_tools`**, which already partitions tool +configs by type. The kept `@ag.reference` is the only new arm it has to recognize; the embed case +arrives as an already-concrete `client` config. ## Resolution path, end to end -``` -author commits agent config with a workflow reference in tools[i] +```text +author commits agent config; tools[i] is @ag.embed OR @ag.reference | SDK ResolverMiddleware: _has_embed_markers(parameters) true (walks lists) | POST {api}/workflows/revisions/resolve -API generic resolver + service resolve step: fetch the referenced workflow revision - | - |-- runnable? -> keep the reference -> CallbackToolSpec(call_ref="workflow.") - | + the shared ToolCallback to the execute target - | - '-- not runnable -> resolve to a concrete `client` tool config (name/desc/input_schema) +API generic resolver: + |-- @ag.embed -> inline the referenced value into tools[i] + | (a concrete `client` tool config) + '-- @ag.reference -> LEAVE the reference in tools[i] (do not inline) v -_agent: AgentConfig.from_params(...) parses the now-resolved tools +_agent: AgentConfig.from_params(...) parses tools[i] + | (a kept @ag.reference is a reference arm; an embedded value is a ClientToolConfig) | -resolve_tools(agent_config.tools): callback spec for runnable; client spec for non-runnable +resolve_tools(agent_config.tools): tool-specific mapping + |-- kept @ag.reference -> CallbackToolSpec(call_ref="workflow.") + | + the shared ToolCallback to the execute target + '-- embedded client cfg -> ClientToolSpec v /run wire: customTools[i] = {kind:"callback", callRef:"workflow.", ...} OR {kind:"client", ...} | @@ -100,14 +125,13 @@ result -> back to the model | Seam | File | Change | | --- | --- | --- | -| Strict schema arm | `sdks/python/agenta/sdk/utils/types.py` | add the embed-ref arm to `AgentConfigSchema.tools` (mirror `_SkillEmbedRefSchema`) so a referenced tool validates in the playground | -| Resolve step (runnable vs not) | service resolve step (where `@ag.embed`/references resolve) | decide runnable vs not for the referenced workflow; produce a callback-bound reference (runnable) or a concrete `client` config (non-runnable) | -| Runnable resolution branch | `sdks/python/agenta/sdk/agents/tools/resolver.py` + a platform resolver in `.../platform/` | a referenced runnable workflow resolves to a `CallbackToolSpec` + the shared `ToolCallback`, mirroring the gateway path | +| Strict schema arms | `sdks/python/agenta/sdk/utils/types.py` | add an embed arm (mirror `_SkillEmbedRefSchema`) **and** a reference arm to `AgentConfigSchema.tools` so both a `@ag.embed` and a `@ag.reference` tool validate in the playground | +| Generic resolver "leave it" branch | SDK `ResolverMiddleware` + `api/oss/src/core/embeds/utils.py` | teach the generic resolver to **leave** an `@ag.reference` in place (inline only `@ag.embed`). Still tool-agnostic — no tool knowledge added | +| `resolve_tools` reference arm | `sdks/python/agenta/sdk/agents/tools/resolver.py` + a platform resolver in `.../platform/` | partition out a kept `@ag.reference`; resolve it to a `CallbackToolSpec` + the shared `ToolCallback`, mirroring the gateway path. The embed case arrives as a plain `ClientToolConfig` and needs no new arm | | Server-side execute | `api/oss/src/apis/fastapi/tools/router.py` (+ core) | a `/tools/call`-style target that parses the `workflow.*` `call_ref`, invokes the referenced workflow revision with the model's arguments, and returns the result envelope | -| Embed resolver | `api/oss/src/core/embeds/utils.py` | **no change** — already walks `tools[]` | -| Wire | `services/agent/src/protocol.ts`, `sdks/python/agenta/sdk/agents/utils/wire.py`, golden fixtures | **no new field** — runnable rides as a `callback` spec, non-runnable as a `client` spec; only `call_ref` content is new | +| Wire | `services/agent/src/protocol.ts`, `sdks/python/agenta/sdk/agents/utils/wire.py`, golden fixtures | **no new field** — a reference rides as a `callback` spec, an embed as a `client` spec; only `call_ref` content is new | -`call_ref` grammar for a runnable workflow: an opaque slug, e.g. `workflow.{slug}` or +`call_ref` grammar for a referenced workflow: an opaque slug, e.g. `workflow.{slug}` or `workflow.{slug}.{version}`. Distinct from the Composio 5-segment grammar (`tools.{provider}.{integration}.{action}.{connection}`). The runner treats `call_ref` as opaque; only the server-side parser must agree. `ResolvedToolSet` keeps its single shared @@ -122,37 +146,42 @@ opaque; only the server-side parser must agree. `ResolvedToolSet` keeps its sing endpoints** (the same place gateway tools are added), not in the workflow catalog. Drop the `_agenta.*` tool-workflow / `_validate_catalog` generalization direction entirely. (PR #4837 review [3470356903](https://github.com/Agenta-AI/agenta/pull/4837#discussion_r3470356903).) -- **No Option A / Option B split.** There is one path; the only branch is runnable vs not. +- **No Option A / Option B split.** There is one path; the only branch is the author's syntax + (`@ag.embed` vs `@ag.reference`). - **`is_tool` flag** is a later, FE-only display hint — not built here. ## Test plan -- **SDK unit:** the embed-ref `tools` arm validates (mirror the skills schema test); a - resolved runnable reference produces the expected `CallbackToolSpec` + `ToolCallback`; a - resolved non-runnable reference produces a `client` spec. -- **Schema:** `AgentConfigSchema` JSON Schema emits the embed-ref `oneOf` arm in `tools`; - `CATALOG_TYPES["agent_config"]` still dereferences. -- **Embed resolution (API/service):** a reference in `tools[i]` resolves to a callback-bound - reference (runnable) or a concrete `client` config (non-runnable); cycle/depth guards hold. -- **Wire / golden:** a golden `/run` fixture with a runnable workflow tool (a `callback` spec) - and one with a non-runnable (a `client` spec); `protocol.ts` Zod accepts both. +- **SDK unit:** both `tools` arms validate (mirror the skills schema test) — a `@ag.embed` tool + and a `@ag.reference` tool; a kept `@ag.reference` resolves to the expected `CallbackToolSpec` + + `ToolCallback`; an `@ag.embed`-resolved concrete `client` config produces a `client` spec. +- **Schema:** `AgentConfigSchema` JSON Schema emits both the embed and reference `oneOf` arms in + `tools`; `CATALOG_TYPES["agent_config"]` still dereferences. +- **Generic resolver:** an `@ag.embed` in `tools[i]` is inlined to its value; an `@ag.reference` + in `tools[i]` is **left in place** (not inlined); cycle/depth guards hold for both. +- **`resolve_tools`:** a kept reference becomes a callback-bound `CallbackToolSpec`; an embedded + `client` config becomes a `client` spec. +- **Wire / golden:** a golden `/run` fixture with a referenced workflow tool (a `callback` spec) + and one with an embedded client tool (a `client` spec); `protocol.ts` Zod accepts both. - **Execute endpoint:** a `/tools/call` with a `workflow.*` `call_ref` invokes the revision and returns the result; the workflow's secrets/connections stay server-side. -- **Live matrix (agent-workflows-qa):** force a runnable workflow tool with an unguessable +- **Live matrix (agent-workflows-qa):** force a referenced workflow tool with an unguessable token across pi_core / claude on local + Daytona + SDK; a pass proves it ran server-side and the result reached the model. Pin a green cell with agent-replay-test. ## Rollout -POC, no flag needed for the schema arm (additive; the resolver already handles it). The -execute endpoint is new server surface. Keep docs in sync in the same implementation PR -(`documentation/tools.md`, `interfaces/public-edge/agent-config-schema.md`, the interface -inventory). +POC, no flag needed for the schema arms (additive; the resolver already handles the embed arm +and gains a small "leave it" branch for the reference arm). The execute endpoint is new server +surface. Keep docs in sync in the same implementation PR (`documentation/tools.md`, +`interfaces/public-edge/agent-config-schema.md`, the interface inventory). ## Build order (when implemented) -1. Schema arm — `tools` accepts a workflow reference (the embed-ref arm), validates in the - playground. -2. Resolve step — decide runnable vs not; runnable → `CallbackToolSpec` + execute endpoint; - non-runnable → concrete `client` config. -3. (Later, FE) `is_tool` flag so referenced workflows surface in the tool picker. +1. Schema arms — `tools` accepts a `@ag.embed` tool and a `@ag.reference` tool; both validate in + the playground. +2. Generic resolver — add the "leave it" branch so `@ag.reference` passes through uninlined while + `@ag.embed` keeps inlining to its value. +3. `resolve_tools` — a kept `@ag.reference` → `CallbackToolSpec` + the execute endpoint; the + embedded `client` config already lands as a `client` spec. +4. (Later, FE) `is_tool` flag so referenced workflows surface in the tool picker. diff --git a/docs/design/agent-workflows/projects/embedref-tools/research.md b/docs/design/agent-workflows/projects/embedref-tools/research.md index 8e6195e817..d89acd1e6d 100644 --- a/docs/design/agent-workflows/projects/embedref-tools/research.md +++ b/docs/design/agent-workflows/projects/embedref-tools/research.md @@ -1,9 +1,10 @@ # Research How `skills` embedding works today, the tool taxonomy, and the exact seams to mirror for -`tools`. Everything below is grounded in the current code; file paths are absolute-from-repo. +`tools` — under the **two-syntax** model (embed vs reference). Everything below is grounded in +the current code; file paths are absolute-from-repo. -## Part 1 — How `@ag.embed` embedding works (the skills case) +## Part 1 — How `@ag.embed` embedding works (the skills case), and what `@ag.reference` adds ### There is no `EmbedRef` model — an embed is a structural marker @@ -16,6 +17,13 @@ There is no dedicated Pydantic class for it on the runtime path. `AG_EMBED_KEY = "@ag.embed"`, `AG_REFERENCES_KEY = "@ag.references"`, `AG_SELECTOR_KEY = "@ag.selector"`. +**Confirmed: there is no reference-only marker today.** `@ag.references` and `@ag.selector` are +strictly **sub-keys inside an `@ag.embed` block** — they are not standalone top-level markers, +and the embed resolver always *inlines* the resolved value. The two-syntax model needs a **new +top-level marker** (e.g. `@ag.reference`, singular) that the same recursive walker recognizes but +treats as "leave in place" instead of "inline." It reuses the same inner `@ag.references` / +`@ag.selector` shape to name the target; only the inline-vs-leave behavior differs. + The canonical object-embed shape (the form `skills` uses): ```jsonc @@ -45,7 +53,9 @@ Two layers: POSTs `parameters` to `{api}/workflows/revisions/resolve` and replaces them with the resolved result. Its own comment says: *"The embed resolver walks arrays, so an `@ag.embed` inside `parameters.skills[i]` resolves on either path."* The same is true of - `parameters.tools[i]`. + `parameters.tools[i]`. Under the two-syntax model the marker check also recognizes + `@ag.reference`, but the resolve pass **leaves that node untouched** (inline-vs-leave is the + only difference); the walk and the list-descent are unchanged. 2. **API generic resolver** — `api/oss/src/core/embeds/utils.py`, `resolve_embeds(...)`. It deep-copies the config and loops up to `max_depth`, each pass calling @@ -63,15 +73,19 @@ The resolver callback routes a `workflow` reference to **Ordering in the agent run path** (`services/oss/src/agent/app.py`, `_agent`): -1. Embed resolution — already done by the SDK middleware against `parameters`, before - `_agent` is even called. -2. `agent_config = AgentConfig.from_params(params, ...)` — parses the *now-inlined* config. -3. `resolved_tools = await resolve_tools(agent_config.tools)` — sees only concrete, - embed-free tool configs. +1. Resolution — done by the SDK middleware against `parameters`, before `_agent` is even + called. `@ag.embed` nodes are inlined to their value; `@ag.reference` nodes are **left in + place**. +2. `agent_config = AgentConfig.from_params(params, ...)` — parses the config. An inlined + `@ag.embed` is now a concrete tool config; a kept `@ag.reference` parses as the reference arm. +3. `resolved_tools = await resolve_tools(agent_config.tools)` — sees concrete tool configs **and** + any kept `@ag.reference` arms. -**Implication:** an `@ag.embed` in `tools[i]` is inlined at step 1 with no resolver change. -By step 3 it is a concrete tool config. The work is making steps 2-3 (and the schema) -understand *what* it inlines into. +**Implication:** an `@ag.embed` in `tools[i]` is inlined at step 1 with only a tiny resolver +addition (the "leave it" branch for the sibling `@ag.reference` marker — embed inlining itself is +unchanged). A kept `@ag.reference` survives to step 3, where `resolve_tools` does the +tool-specific mapping. The work is two schema arms (step 2) plus the `resolve_tools` reference arm +(step 3); the generic resolver gains only the "leave it" branch. ### The `_agenta.*` platform catalog short-circuit (background only) @@ -95,11 +109,13 @@ in the DB and never hit this validation. - Strict `AgentConfigSchema.skills`: `sdks/python/agenta/sdk/utils/types.py` — `List[Union["SkillConfigSchema", "_SkillEmbedRefSchema"]]`. The embed arm is `_SkillEmbedRefSchema` with `embed: Dict[str, Any] = Field(alias="@ag.embed")` and - `extra="forbid"`. This is the exact arm to mirror for tools. + `extra="forbid"`. This is the exact arm to mirror for the tools **embed** arm. Tools add one + more arm — a `_ToolReferenceSchema` with `reference: Dict[str, Any] = Field(alias="@ag.reference")` + — for the kept-reference syntax. Skills do not need it (a skill is always a value). - Default config: `build_agent_v0_default(...)` in `sdks/python/agenta/sdk/utils/types.py` ships the skill `@ag.embed` block. -## Part 2 — The tool taxonomy (what an embedded tool must become) +## Part 2 — The tool taxonomy (what each syntax must become) ### Two lives, three axes @@ -140,44 +156,55 @@ tool_callback}`). The TS twin is `ResolvedToolSpec` in `services/agent/src/proto POSTs back to `/tools/call`** (directly, or via the Daytona file relay). Absent `kind` defaults to `callback`. -### Why `callback` is the right executor for a *runnable* workflow tool - -The branch that matters is **runnable vs non-runnable** (see [plan.md](plan.md)). The taxonomy -already has a home for each: - -- A **runnable** workflow tool is **server-executed**: calling it means invoking another Agenta - workflow revision, which lives behind the API and may itself use connections and secrets. That - is exactly the gateway tool's safety shape — the harness decides *which* tool and *with what - arguments*, the service runs it, and no credential reaches the sandbox. So it resolves to a - `CallbackToolSpec`. The runner needs **no new `kind`** — `callback` already dispatches to - `callAgentaTool`, works under the Daytona file relay, and is delivered to both Pi (native) and - Claude (the `agenta-tools` MCP bridge). The only difference from a gateway tool is the - `call_ref` grammar and the execute target: instead of a Composio action, the service invokes a - workflow revision. -- A **non-runnable** (client) workflow tool fits the existing **`client`** executor: the resolve - step turns the reference into a concrete `client` tool config, and at run time the runner - returns a `client` spec for the browser to fulfill next turn (`models.py:206` — - `kind: "client"`). No callback, no server-side execute. +### Why `callback` for `@ag.reference` and `client` for `@ag.embed` + +The branch is the **author's syntax** (see [plan.md](plan.md)), and the taxonomy already has a +home for each: + +- An **`@ag.reference`** workflow tool is **server-executed**: calling it means invoking another + Agenta workflow revision, which lives behind the API and may itself use connections and + secrets. That is exactly the gateway tool's safety shape — the harness decides *which* tool and + *with what arguments*, the service runs it, and no credential reaches the sandbox. So + `resolve_tools` maps it to a `CallbackToolSpec`. The runner needs **no new `kind`** — `callback` + already dispatches to `callAgentaTool`, works under the Daytona file relay, and is delivered to + both Pi (native) and Claude (the `agenta-tools` MCP bridge). The only difference from a gateway + tool is the `call_ref` grammar and the execute target: instead of a Composio action, the + service invokes a workflow revision. **Crucially, the reference is *not* inlined before + `resolve_tools` runs** — that is the whole point of the second syntax: the generic resolver + leaves it, so `resolve_tools` sees the kept reference (slug + version + the model-facing + surface) and builds the callback spec from it. The callback path never needs the *resolved + workflow artifact* at config time; it carries only the identity (`call_ref`) and resolves the + revision lazily, server-side, when the model actually calls the tool. +- An **`@ag.embed`** (client) workflow tool fits the existing **`client`** executor: the generic + resolver inlines the reference into a concrete `client` tool config *before* `resolve_tools` + runs, so `resolve_tools` sees a plain `ClientToolConfig` and the runner returns a `client` spec + for the browser to fulfill next turn (`models.py:206` — `kind: "client"`). No callback, no + server-side execute. + +This is what the earlier "keep the reference but it's already inlined" tension was about: with a +single `@ag.embed` syntax, a tool could not both stay a reference for callback resolution *and* be +inlined before `resolve_tools`. The two-syntax model removes the contradiction — embed inlines, +reference is kept — so each path sees exactly the form it needs. ## Part 3 — The seams to touch (summary) | Seam | File | Change | | --- | --- | --- | -| Strict schema embed arm | `sdks/python/agenta/sdk/utils/types.py` | add `_ToolEmbedRefSchema`, make `AgentConfigSchema.tools` a `Union[ToolConfig-twin, _ToolEmbedRefSchema]` (mirror skills) | -| Resolve step (runnable vs not) | service resolve step (where references resolve) | decide runnable vs not; runnable → keep the reference for callback resolution; non-runnable → resolve to a concrete `client` tool config | -| Runnable resolution branch | `sdks/python/agenta/sdk/agents/tools/resolver.py` + a platform resolver in `.../platform/` | resolve a referenced runnable workflow to a `CallbackToolSpec` + a `ToolCallback` to the new execute endpoint (mirror gateway) | +| Strict schema arms | `sdks/python/agenta/sdk/utils/types.py` | add `_ToolEmbedRefSchema` (alias `@ag.embed`) **and** `_ToolReferenceSchema` (alias `@ag.reference`); make `AgentConfigSchema.tools` a `Union[ToolConfig-twin, _ToolEmbedRefSchema, _ToolReferenceSchema]` (the embed arm mirrors skills; the reference arm is new) | +| Generic resolver "leave it" branch | SDK `ResolverMiddleware` + `api/oss/src/core/embeds/utils.py` | recognize the new `@ag.reference` marker and **leave it in place** (inline only `@ag.embed`). Tool-agnostic — no tool knowledge added | +| `resolve_tools` reference arm | `sdks/python/agenta/sdk/agents/tools/resolver.py` + a platform resolver in `.../platform/` | partition out a kept `@ag.reference`; resolve it to a `CallbackToolSpec` + a `ToolCallback` to the new execute endpoint (mirror gateway). The embed case arrives as a plain `ClientToolConfig` — no new arm | | Server-side execute | `api/oss/src/apis/fastapi/tools/router.py` (+ core) | a `/tools/call`-style target that invokes the referenced workflow revision and returns the result | -| Embed resolver | `api/oss/src/core/embeds/utils.py` | **no change** — already walks `tools[]` | -| Wire | `services/agent/src/protocol.ts`, `sdks/python/agenta/sdk/agents/utils/wire.py`, golden fixtures | **no new field** — runnable rides as a `callback` spec, non-runnable as a `client` spec; only the `call_ref` content is new | -| Docs | `documentation/tools.md`, `interfaces/public-edge/agent-config-schema.md`, the interface inventory | document the embed arm + the runnable-vs-not behavior | +| Wire | `services/agent/src/protocol.ts`, `sdks/python/agenta/sdk/agents/utils/wire.py`, golden fixtures | **no new field** — a reference rides as a `callback` spec, an embed as a `client` spec; only the `call_ref` content is new | +| Docs | `documentation/tools.md`, `interfaces/public-edge/agent-config-schema.md`, the interface inventory | document both syntaxes + the syntax-decides-behavior model | No `WorkflowToolConfig` variant, no `compat.py` `"workflow"` allowlist entry, no platform-catalog change — all dropped per the PR #4837 review. ## Open research questions (carried into the plan) -1. **Where the runnable/not decision is made** — confirm it is the service resolve step (where - the referenced workflow is fetched), so the SDK/runner stay schema-driven. +1. **The `@ag.reference` marker shape** — confirm it reuses the inner `@ag.references` / + `@ag.selector` block (same target-naming as `@ag.embed`) and differs only in the "leave it" + behavior; confirm the singular `@ag.reference` name. 2. **What does invoking the workflow mean** — call `/workflows/.../invoke` with the model's arguments as inputs, and map the workflow output back as the tool result? What is the input/output contract between a tool call and a workflow invoke? diff --git a/docs/design/agent-workflows/projects/embedref-tools/status.md b/docs/design/agent-workflows/projects/embedref-tools/status.md index e071d70b6c..024e56f8f6 100644 --- a/docs/design/agent-workflows/projects/embedref-tools/status.md +++ b/docs/design/agent-workflows/projects/embedref-tools/status.md @@ -6,9 +6,14 @@ This is the source of truth for the project's progress, decisions, and open ques - **Phase:** IMPLEMENTED (the lgtm'd two-syntax design #4837). Spun from PR #4821 review comment [3469653315](https://github.com/Agenta-AI/agenta/pull/4821#discussion_r3469653315). -- **Docs:** README, context, research, plan, status written, then **revised per the author's - review on PR #4837** to the simplified runnable-vs-not model (Option A/B split, the - `workflow` tool variant, and platform-tools-as-workflows all removed). +- **Docs:** README, context, research, plan, status written, then revised twice on PR #4837. + Iteration 2 simplified to one path branching on runnable-vs-not (dropped Option A/B, the + `workflow` tool variant, platform-tools-as-workflows). **Iteration 3** (per the author's + comment [3473648119](https://github.com/Agenta-AI/agenta/pull/4837#discussion_r3473648119)) + replaces the "infer runnable/not server-side in a resolve step" mechanism with **two syntaxes**: + `@ag.embed` (inline the value) and a new `@ag.reference` (keep the reference). The author's + syntax choice decides the behavior; the generic resolver stays tool-agnostic; the tool-specific + logic lives in `resolve_tools`. - **Built (this slice):** - SDK marker + config: `AG_REFERENCE_MARKER` and `ReferenceToolConfig` (`type: "reference"`, `slug`/`version`/`name`/`description`/`input_schema`, `.call_ref` @@ -34,65 +39,87 @@ This is the source of truth for the project's progress, decisions, and open ques ## Design -**A tool is just a referenced workflow.** `tools[i]` points at a workflow (by reference, not by -inlining its config). Any workflow type qualifies — agent, completion, channel, chain. There is -**no `workflow` tool variant** and **no "tool workflow" type**. The only branch is **runnable -vs non-runnable**, decided server-side in the resolve step where the referenced workflow is -known: +**A tool is just a workflow, pointed at via one of two syntaxes.** `tools[i]` carries either an +`@ag.embed` (inline the value) or an `@ag.reference` (keep the reference). Any workflow type +qualifies — agent, completion, channel, chain. There is **no `workflow` tool variant** and **no +"tool workflow" type**. **The author's syntax decides the behavior** (the decision is *not* +inferred server-side by inspecting the target): -- **Runnable** (executable: completion / agent / channel / chain). You reference it *because - you want to call it*. It resolves to the existing **`callback`** executor — a - `CallbackToolSpec` whose `call_ref` encodes the workflow identity, plus the shared +- **`@ag.reference`** (new — for a runnable workflow you want to *call*). The generic resolver + **leaves the reference in the config**. `resolve_tools` turns it into the existing **`callback`** + executor — a `CallbackToolSpec` whose `call_ref` encodes the workflow identity, plus the shared `ToolCallback` to a server-side execute endpoint. The model's call routes back, the service - invokes the workflow revision, the result returns. Connections/secrets stay server-side, - exactly like a gateway tool. **No new runner `kind`.** -- **Non-runnable** (a client tool). Referencing-to-call does not apply. The resolve step - **resolves the reference into its value** — a concrete `client` tool config — and at run time - it is the existing `client` path (fulfilled in the browser next turn). - -Unifying rule: reference everything as a tool; **in the sidecar, if it is runnable, run it; if -it is not, return its schema.** - -**Why `callback` for the runnable case:** a runnable workflow tool is server-executed and may + invokes the workflow revision, the result returns. Connections/secrets stay server-side, exactly + like a gateway tool. **No new runner `kind`.** +- **`@ag.embed`** (existing — for a non-runnable client tool that is a *value*). The generic + resolver **resolves the reference into its value** — a concrete `client` tool config. By the + time `resolve_tools` runs it is a plain `ClientToolConfig` and rides the existing `client` path + (fulfilled in the browser next turn). + +**The decision boundary:** the generic resolver (`ResolverMiddleware` + the API embed resolver) +knows only inline-the-value (`@ag.embed`) vs leave-the-reference (`@ag.reference`) and **nothing +about tools**; **`resolve_tools`** owns all tool-specific mapping (kept reference → callback spec; +embedded value → client spec). + +Unifying rule for the sidecar: point at everything as a tool; a kept reference is run (the +callback executes the workflow), an embedded value is a concrete `client` tool config fulfilled in +the browser. + +**Why `callback` for the reference case:** a referenced workflow tool is server-executed and may use connections/secrets — exactly the gateway tool's safety shape. Resolving to a `CallbackToolSpec` keeps every credential server-side and reuses the runner's existing callback delivery (direct, Daytona relay, Pi native, Claude `agenta-tools` bridge). -**Explicitly dropped from the first design** (per the author's PR #4837 review): +**Explicitly dropped across iterations** (per the author's PR #4837 reviews): -- the Option A / Option B framing (one path, branch on runnable); -- the `WorkflowToolConfig` variant / the `"workflow"` `type` allowlist entry (a tool is just a - workflow); -- **platform tools as workflows** — they go in the **existing tools endpoints** (like gateway), - not the workflow catalog, so the `_validate_catalog` generalization is gone. +- iteration 2: the Option A / Option B framing; the `WorkflowToolConfig` variant / the + `"workflow"` `type` allowlist entry (a tool is just a workflow); **platform tools as workflows** + (they go in the existing tools endpoints, like gateway, not the workflow catalog, so the + `_validate_catalog` generalization is gone); +- iteration 3: inferring runnable/not server-side in a resolve step — replaced by the + author-chosen syntax (`@ag.embed` vs `@ag.reference`). ## Settled by research -- The `@ag.embed` resolver is **generic and already walks `tools[]`** — no resolver change is - needed for referencing. (`ResolverMiddleware` + `api/oss/src/core/embeds/utils.py`.) This is - the load-bearing finding and it survives the simplification. -- Reference resolution runs **before** `AgentConfig.from_params` and `resolve_tools`, so by the - time tools resolve, the reference is already concrete. -- The skills schema arm (`_SkillEmbedRefSchema`) is the exact template to mirror. +- The resolver is **generic and already walks `tools[]`** — embedding needs no resolver change, + and the new `@ag.reference` syntax adds only a small "leave it" branch (no tool knowledge). + (`ResolverMiddleware` + `api/oss/src/core/embeds/utils.py`.) This is the load-bearing finding + and it survives the reframe. +- There is **no reference-only marker today** — `@ag.references` / `@ag.selector` are sub-keys + inside an `@ag.embed`. The two-syntax model adds a new top-level `@ag.reference` marker. +- An `@ag.embed` resolves **before** `AgentConfig.from_params` and `resolve_tools` (so by tool + resolution it is concrete); an `@ag.reference` is **deliberately kept** so `resolve_tools` sees + it and builds the callback spec. +- The skills schema arm (`_SkillEmbedRefSchema`) is the template for the tools embed arm; the + reference arm (`_ToolReferenceSchema`) is new and tools-only. + +## Settled by the author (was open, now closed) + +- **Where the tool-specific decision lives** — in **`resolve_tools`**, not a server-side resolve + step that inspects the target. The generic resolver only does inline-vs-leave; the + reference-vs-embed choice is the author's, encoded in the syntax. (Closes the old "where the + runnable/not decision lives" question; resolves CodeRabbit's "intro reads settled but status + treats it as open" flag.) ## Open questions for the user -1. **Where the runnable/not decision lives, precisely** — confirm it is the service resolve - step (where the referenced workflow is fetched), so the SDK/runner stay schema-driven. -2. **Tool-call to workflow-invoke contract** — how do the model's tool arguments map to the +1. **Tool-call to workflow-invoke contract** — how do the model's tool arguments map to the workflow's invoke inputs, and how does the workflow output map back to the tool result? Free-form passthrough, or a declared input/output schema? -3. **`call_ref` grammar for runnable workflow tools** — `workflow.{slug}` / +2. **`call_ref` grammar for referenced workflow tools** — `workflow.{slug}` / `workflow.{slug}.{version}`? Today's gateway grammar (`tools.{provider}.{integration}.{action}.{connection}`) is Composio-specific and parsed in two places; a workflow tool needs its own opaque slug. +3. **The `@ag.reference` marker name/shape** — confirm the singular `@ag.reference` top-level + marker reusing the inner `@ag.references` / `@ag.selector` block (same target-naming as + `@ag.embed`, differing only in leave-vs-inline). 4. **Single shared callback endpoint vs per-spec callbacks** — `ResolvedToolSet` holds one `tool_callback`. With both gateway and workflow tools present, route one endpoint by `call_ref` prefix (smaller change, recommended) or grow the wire to per-spec callbacks? 5. **`is_tool` FE flag** — confirm it is deferred (later, display-only so referenced workflows surface in the tool picker) and not part of this slice. -6. **Approval / render axes** — a referenced tool can carry `needs_approval` and `render` like - any tool; confirm no special handling is wanted (default: they compose as usual). +6. **Approval / render axes** — a referenced or embedded tool can carry `needs_approval` and + `render` like any tool; confirm no special handling is wanted (default: they compose as usual). ## Risks / watch-fors @@ -103,5 +130,8 @@ delivery (direct, Daytona relay, Pi native, Claude `agenta-tools` bridge). - **Two models, one contract.** The strict `AgentConfigSchema` and the permissive runtime `AgentConfig` must move together (and a golden fixture), per agent-config-schema.md's "watch for when changing." +- **New marker, two resolvers.** The `@ag.reference` marker must be recognized in **both** the + SDK `ResolverMiddleware` and the API embed resolver, and both must agree to *leave it* (not + inline). A miss in either inlines a reference and breaks the callback path. - **Keep docs in sync** in the implementation PR: `documentation/tools.md`, `interfaces/public-edge/agent-config-schema.md`, and the interface inventory.