Skip to content

docs(agents): clarify default vs. custom agent distinction #3398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ralphbean merged 7 commits into from
Jul 9, 2026
Merged

docs(agents): clarify default vs. custom agent distinction #3398

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ecf4e25
docs(agents): clarify default vs. custom agent distinction
ralphbean Jul 7, 2026
8c0c1e2
docs(glossary): add default agent, configured default agent, custom a…
ralphbean Jul 7, 2026
1e16ff4
docs(glossary): differentiate custom-extending-default from custom-fr…
ralphbean Jul 7, 2026
cb0f38e
docs(custom-agents): rename to "from scratch", suggest extending defa…
ralphbean Jul 7, 2026
a0b711c
docs: introduce 'derived agent' tier between configured default and c…
ralphbean Jul 8, 2026
589d766
docs: address review feedback — terminology sweep and cleanups
ralphbean Jul 8, 2026
32f8fe5
docs(glossary): move Derived Agent to section D, alphabetize
ralphbean Jul 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions docs/agents/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,17 @@ config (see [ADR 0058](../ADRs/0058-agent-registration.md)).
| [Fix](fix.md) | Addresses review feedback on open PRs |
| [Retro](retro.md) | Analyzes completed workflows and proposes system improvements |

## Customization
## Configuration

All agents can be customized by adding instructions and skills to your
All agents can be configured by adding instructions and skills to your
repository. Changes to `AGENTS.md` affect every agent; skills let you tune how
a specific agent performs a specific task. See
[Customizing with AGENTS.md](../guides/user/customizing-with-agents-md.md) and
[Customizing with Skills](../guides/user/customizing-with-skills.md).
[Configuring with AGENTS.md](../guides/user/customizing-with-agents-md.md) and
[Configuring with Skills](../guides/user/customizing-with-skills.md).

At some point, enough configuration turns a configured default agent into a
derived agent. See [Default, derived, and custom agents](topics/default-vs-custom.md)
for where that line is and why it matters.

## Custom Agents

Expand Down
4 changes: 2 additions & 2 deletions docs/agents/code.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ on issues (not PRs). The code agent is also triggered automatically when the

## Configuration and extension

See [Customizing with AGENTS.md](../guides/user/customizing-with-agents-md.md) and
[Customizing with Skills](../guides/user/customizing-with-skills.md).
See [Configuring with AGENTS.md](../guides/user/customizing-with-agents-md.md) and
[Configuring with Skills](../guides/user/customizing-with-skills.md).

### Variables

Expand Down
4 changes: 2 additions & 2 deletions docs/agents/fix.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,8 @@ Remove the label or use `/fs-fix` to re-engage.

## Configuration and extension

See [Customizing with AGENTS.md](../guides/user/customizing-with-agents-md.md) and
[Customizing with Skills](../guides/user/customizing-with-skills.md).
See [Configuring with AGENTS.md](../guides/user/customizing-with-agents-md.md) and
[Configuring with Skills](../guides/user/customizing-with-skills.md).

### Variables

Expand Down
4 changes: 2 additions & 2 deletions docs/agents/retro.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ The retro agent also runs automatically when a PR is closed (merged or not).

## Configuration and extension

See [Customizing with AGENTS.md](../guides/user/customizing-with-agents-md.md) and
[Customizing with Skills](../guides/user/customizing-with-skills.md).
See [Configuring with AGENTS.md](../guides/user/customizing-with-agents-md.md) and
[Configuring with Skills](../guides/user/customizing-with-skills.md).

### Variables

Expand Down
4 changes: 2 additions & 2 deletions docs/agents/review.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ the upstream default -- no other configuration needed.
> config-driven agent registration instead. Run `fullsend agent migrate-customizations`
> to migrate existing overrides.

See [Customizing with AGENTS.md](../guides/user/customizing-with-agents-md.md) and
[Customizing with Skills](../guides/user/customizing-with-skills.md).
See [Configuring with AGENTS.md](../guides/user/customizing-with-agents-md.md) and
[Configuring with Skills](../guides/user/customizing-with-skills.md).

### Variables

Expand Down
100 changes: 100 additions & 0 deletions docs/agents/topics/default-vs-custom.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Default, derived, and custom agents

Fullsend ships a set of default agents in
[fullsend-ai/agents](https://github.com/fullsend-ai/agents). Each can be
configured and extended.
At some point, enough modification turns a configured default into something
different. This document defines three tiers:

1. **Configured default agent** — uses only documented extension points

@fullsend-ai-review fullsend-ai-review Bot Jul 8, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] internal-consistency

Tier-1 intro definition ('uses only documented extension points') is narrower than The Rule ('documented extension point for that agent, or a general-purpose harness field that does not alter the agent's identity'). The glossary matches The Rule; the intro does not. This leaves the 'policy:' classification in the table unjustified by the intro.

Suggested fix: Update the intro to: 'uses only documented extension points or general-purpose harness fields that do not alter the agent's identity'.

(env vars, skills, `AGENTS.md`, plugins, host files, sandbox image layers).
Still recognizably the same default agent.
2. **Derived agent** — starts from a default via `base` inheritance but
replaces identity-defining components (system prompt, scripts, slug, or
validation loop). It re-uses parts of a default but is no longer
recognizably that agent.
3. **Custom agent** — its `base` chain does not trace back to a default agent
harness, or it has no `base` at all. Built from scratch.

## Why the distinction matters

We want two things simultaneously:

1. **Encourage derived and custom agents.** The harness, sandbox, and `base`
composition system exist so teams can build agents tailored to their
workflows.
2. **Encourage contribution to default agents.** When a change improves a
default agent for everyone, it should flow upstream rather than live in a
private fork.

Knowing whether you are running a derived agent or a custom agent helps you
decide: should this change be contributed back, or does it only make sense for
my team? Clear language helps us all communicate about this.

**Default to contributing.** If your modification would benefit other users,
contribute it to the default agent's definition. Build a custom agent when
your needs genuinely diverge from the default agent's charter.

Comment thread
ralphbean marked this conversation as resolved.
## The rule

@fullsend-ai-review fullsend-ai-review Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] technical-accuracy

"The rule" states a modification is still-default only if it "uses a method documented as a recommended extension point for that agent." The classification table and glossary apply a broader identity-preserving criterion for fields like env:, plugins:, and host_files: that no default agent documents as extension points. The rule and table apply different classification criteria.

Suggested fix: Broaden the rule to acknowledge both criteria (documented extension points AND identity-preserving harness infrastructure fields), or note in the table that these fields are implicitly considered extension points.

@fullsend-ai-review fullsend-ai-review Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] technical-accuracy

Internal inconsistency between 'The rule' (requires documented extension points) and the classification table (marks env:, plugins:, host_files:, sandbox image as still-default based on broader identity-preserving rationale). The glossary's Configured Default Agent entry also uses the broader interpretation. Consider tightening the rule or noting that general-purpose fields are implicitly considered extension points.

@fullsend-ai-review fullsend-ai-review Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] technical-accuracy

Internal inconsistency between 'The rule' and the classification table. The rule requires 'a method documented as a recommended extension point for that agent' but the table marks env:, plugins:, host_files:, and sandbox image replacement as still-default using an identity-preservation rationale not tied to per-agent extension point documentation. The glossary bridges the gap by listing these as universal extension points, but per-agent docs (e.g., review.md) only document agent-specific extension points.

Suggested fix: Consider softening the rule to 'uses a documented extension point or a harness field that does not alter the agent's identity' to match the actual rationale in the table and glossary.


> If a modification uses a documented extension point for that agent, or a
> general-purpose harness field that does not alter the agent's identity, the
> result is still a **configured default agent**. If it replaces
> identity-defining components, the result is a **derived agent**.

Each default agent documents its extension points in
[`docs/agents/<agent>.md`](../). The review agent, for example, documents
`REVIEW_FINDING_SEVERITY_THRESHOLD` as a configuration variable and
Comment on lines +45 to +47

@qodo-code-review qodo-code-review Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Informational

2. Mislinked extension-point docs 🐞 Bug ⚙ Maintainability

In the new “default vs custom” topic, the link text claims extension points live in per-agent files
(docs/agents/<agent>.md), but the hyperlink target points to the agents index (../), making the
reference internally inconsistent and less usable for readers trying to jump to the relevant agent
doc.
Agent Prompt 
### Issue description
`docs/agents/topics/default-vs-custom.md` says extension points are documented in per-agent docs (`docs/agents/<agent>.md`), but the embedded link points to `../` (agents index). This is misleading and makes the cross-reference less actionable.

### Issue Context
Per-agent docs exist (e.g. `docs/agents/review.md`) and are linked from the agents index.

### Fix Focus Areas
- docs/agents/topics/default-vs-custom.md[35-37]

### Suggested change
Update the sentence/link to match intent, e.g.:
- Keep the pattern as code (no link) and link “Agents reference” to `../README.md` (or `../`), **or**
- Link to a concrete example like `../review.md` and optionally mention the general pattern in text.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

`issue-labels` as an overloadable skill. Using those mechanisms produces a
configured review agent, not a derived one.

## The `base` lineage test

The `base` field in a harness YAML
([ADR 0045](../../ADRs/0045-forge-portable-harness-schema.md)) is the first
thing to check. If a harness's `base` chain — through one or more levels of
inheritance — traces back to a default agent harness in `fullsend-ai/fullsend`,
the harness *started from* a default agent. What you override on top of that
base determines whether the result is still a configured default or has crossed
into derived territory.

If the `base` chain does **not** trace back to a default agent harness, the
agent is custom by definition — regardless of how similar it looks.

## Classification by harness field

| Modification | Classification | Rationale |
|---|---|---|
| Set a documented configuration variable (e.g., `REVIEW_FINDING_SEVERITY_THRESHOLD`) | Configured default | Documented extension point. The agent was designed for this. |
| Add environment variables via `env:` | Configured default | Env vars augment behavior without changing identity. |
| Add skills via `skills:` | Configured default | Skills extend knowledge. The agent's core behavior is unchanged. |
| Add repo-level skills in `.agents/skills/` | Configured default | Repo skills are discovered automatically; no harness change needed. |
| Add project instructions via `AGENTS.md` | Configured default | All agents read `AGENTS.md`. This is the standard customization path. |
| Override a built-in skill via `customized/skills/` | Configured default | Documented extension point ([Configuring with Skills](../../guides/user/customizing-with-skills.md#overriding-built-in-skills)). |
| Replace the sandbox image with one based on the default image | Configured default | The agent's behavior is unchanged; the environment is augmented. |
| Add plugins via `plugins:` | Configured default | Plugins extend tooling without changing the agent's identity. |
| Add host files via `host_files:` | Configured default | Additional data for the sandbox. The agent itself is unchanged. |
| Change the sandbox policy (`policy:`) | Configured default | Policy composition lets you augment an agent's policy without changing its identity. |
| **Replace the agent system prompt** (`agent:`) | **Derived** | The system prompt (sometimes called the subagent definition file) provides the primary instructions for the agent. Replacing it creates a different agent. |
| **Replace pre or post scripts** (`pre_script:`, `post_script:`) | **Derived** | Scripts control the agent's integration with external systems. Different scripts mean different behavior at the pipeline boundary. |
| **Replace the app role slug** (`slug:`) | **Derived**\* | The slug determines who the agent authenticates as. A different identity is a different agent. |
| **Replace the validation loop** (`validation_loop:`) | **Derived** | The validation loop defines the contract between the agent and the harness. Changing it changes what the agent is expected to produce. |

\* Replacing the slug is acceptable in limited cases we may document in the
future — for example, granting the review agent merge rights via a different
GitHub App. When a specific agent's documentation recommends a slug override
for a stated purpose, that override does not make the agent derived.

## See also

- [Agents reference](../) — default agent documentation and extension points
- [Configuring agents](../../guides/user/customizing-agents.md) — harness
configuration and layered content resolution
- [Configuring with AGENTS.md](../../guides/user/customizing-with-agents-md.md)
— project-wide instructions for all agents
- [Configuring with skills](../../guides/user/customizing-with-skills.md) —
extending or replacing built-in skills
- [Building custom agents](../../guides/user/building-custom-agents.md) —
creating a new agent from scratch
- [ADR 0045](../../ADRs/0045-forge-portable-harness-schema.md) — `base`
composition and harness inheritance
22 changes: 22 additions & 0 deletions docs/glossary.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,35 @@ See [autonomy-spectrum.md](problems/autonomy-spectrum.md).
The scope of damage a compromised or misbehaving agent can cause. A core design constraint: every architectural decision about sandboxing, identity scoping, and network policy is evaluated by asking "what is the blast radius if this agent is compromised?" Minimizing blast radius is the primary goal of the sandbox layer.
See [security-threat-model.md](problems/security-threat-model.md) and [architecture.md](architecture.md).

## C

### Configured Default Agent

A [default agent](#default-agent) whose behavior has been adjusted using documented extension points or general-purpose harness fields that do not alter the agent's identity — environment variables, skills, `AGENTS.md` instructions, sandbox image layers, plugins, or host files. The agent's identity (system prompt, scripts, slug, validation loop) is unchanged; it is still recognizably the same default agent.
See [Default, derived, and custom agents](agents/topics/default-vs-custom.md).

### Custom Agent

An agent whose `base` chain does not trace back to a default agent harness in `fullsend-ai/fullsend`, or that has no `base` at all. A custom agent is built from scratch, even if it happens to resemble a default agent. Contrast with [derived agent](#derived-agent), which starts from a default.
See [Default, derived, and custom agents](agents/topics/default-vs-custom.md) and [Building custom agents](guides/user/building-custom-agents.md).

## D

### Debouncing

Collapsing rapid-fire events on the same issue or PR into a single agent invocation. Without debouncing, a burst of edits to an issue body could trigger multiple redundant triage runs. The [webhook + dispatch service](ADRs/0002-initial-fullsend-design.md#1-webhook--dispatch-service) is responsible for deduplicating flapping events before dispatching work to agents. On GitHub this uses real-time webhooks; on GitLab the cron poller provides watermark-based deduplication at 5–60 minute intervals, which is functionally analogous but operates on a coarser time scale (see [ADR 0067](ADRs/0067-gitlab-cron-polling-event-dispatch.md)).
See [architecture.md](architecture.md) (building block 1).

### Default Agent

An agent shipped by fullsend, defined by harness files in `fullsend-ai/fullsend` and agent definitions in `fullsend-ai/agents`. A default agent remains a default agent when configured through its documented extension points (becoming a [configured default agent](#configured-default-agent)); it becomes a [derived agent](#derived-agent) when modifications go beyond those points.
See [Agents reference](agents/) and [Default, derived, and custom agents](agents/topics/default-vs-custom.md).

### Derived Agent

An agent that uses `base` inheritance from a default agent harness but replaces identity-defining components (system prompt, pre/post scripts, slug, or validation loop) beyond the documented extension points. It re-uses parts of a default agent but is no longer recognizably that agent. Contrast with [configured default agent](#configured-default-agent) (stays within extension points) and [custom agent](#custom-agent) (built from scratch).
See [Default, derived, and custom agents](agents/topics/default-vs-custom.md).

## E

### Entry Point
Expand Down
9 changes: 5 additions & 4 deletions docs/guides/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,11 @@ Guides for developers working in repositories where fullsend is active.

- [Bugfix workflow](user/bugfix-workflow.md) — End-to-end guide to how fullsend handles a bug report from issue to merge
- [Running agents locally](user/running-agents-locally.md) — Run fullsend agents on your machine using released binaries (macOS + Linux)
- [Customizing agents](user/customizing-agents.md) — Harness configurations and layered content resolution for your org and repos
- [Customizing with AGENTS.md](user/customizing-with-agents-md.md) — Guide agents using your repo's AGENTS.md file
- [Customizing with skills](user/customizing-with-skills.md) — Extend or replace built-in agent skills with custom skill documents
- [Building custom agents](user/building-custom-agents.md) — Create a new agent from scratch on a per-repo fullsend installation
- [Configuring agents](user/customizing-agents.md) — Harness configurations and layered content resolution for your org and repos
- [Configuring with AGENTS.md](user/customizing-with-agents-md.md) — Guide agents using your repo's AGENTS.md file
- [Configuring with skills](user/customizing-with-skills.md) — Extend or replace built-in agent skills with custom skill documents
- [Building custom agents from scratch](user/building-custom-agents.md) — Create a new agent from scratch on a per-repo fullsend installation
- [Default, derived, and custom agents](../agents/topics/default-vs-custom.md) — When configuration crosses into derived or custom agent territory

## Development

Expand Down
2 changes: 1 addition & 1 deletion docs/guides/dev/cli-internals.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -603,4 +603,4 @@ var executableFiles = map[string]struct{}{
- [Advanced setup](../infrastructure/advanced-setup.md) — Alternative installation paths and setup flags
- [Mint service administration](../infrastructure/mint-administration.md) — Deploying and managing the token mint
- [Infrastructure Reference](../infrastructure/infrastructure-reference.md) — Infrastructure details
- [Customizing Agents](../user/customizing-agents.md) — User customization guide
- [Configuring Agents](../user/customizing-agents.md) — User configuration guide
2 changes: 1 addition & 1 deletion docs/guides/infrastructure/private-repositories.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,6 @@ Not all private repos are equal. A repo containing open-source code that happens
## See also

- [Getting Started](../getting-started/) — Initial fullsend setup
- [Customizing agents](../user/customizing-agents.md) — Harness configuration and layered overrides
- [Configuring agents](../user/customizing-agents.md) — Harness configuration and layered overrides
- [Security threat model](../../problems/security-threat-model.md) — Threat priority and defense considerations
- [#1189](https://github.com/fullsend-ai/fullsend/issues/1189) — Retro agent private content leak risk
21 changes: 14 additions & 7 deletions docs/guides/user/building-custom-agents.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,21 @@
# Building custom agents
# Building custom agents from scratch

> **Deprecated:** This guide uses the `customized/` directory overlay, which is
> deprecated per [ADR-0064](../../ADRs/0064-deprecate-customized-directory-overlay.md).
> For new custom agents, register them in `config.yaml` with a local `source:`
> path instead. Run `fullsend agent migrate-customizations --dry-run` to
> preview migrating existing customizations.

This guide walks through creating a new custom agent from scratch on a per-repo fullsend installation.
This guide walks through creating a custom from-scratch agent on a per-repo
fullsend installation.

For customizing existing agents (overriding harnesses, skills, or policies), see [Customizing agents](customizing-agents.md).
Before building from scratch, consider whether extending a default agent would
meet your needs. You can use `base` inheritance to start from a default agent's
harness and override only what differs — see
[Default, derived, and custom agents](../../agents/topics/default-vs-custom.md)
for the distinction and when each approach makes sense.

For configuring existing agents (overriding harnesses, skills, or policies), see [Configuring agents](customizing-agents.md).

## Prerequisites

Expand All @@ -28,7 +35,7 @@ A custom agent is composed of six parts:
skills/ # Knowledge documents mounted into the sandbox
```

At build time, the workflow layers these customized files on top of the upstream fullsend defaults. Your files override the defaults — anything you don't customize uses the standard fullsend configuration. See [Customizing agents — Layered Configuration Resolution](customizing-agents.md#layered-configuration-resolution) for details on how the layering works.
At build time, the workflow layers these customized files on top of the upstream fullsend defaults. Your files override the defaults — anything you don't customize uses the standard fullsend configuration. See [Configuring agents — Layered Configuration Resolution](customizing-agents.md#layered-configuration-resolution) for details on how the layering works.

The key security invariant: agents run inside an untrusted [sandbox](../../glossary.md#sandbox) with no credentials. Pre-scripts fetch data *before* the sandbox starts; post-scripts act on agent output *after* the sandbox exits. Agents never have direct write access to external systems. See the [security threat model](../../problems/security-threat-model.md) for the full trust model.

Expand Down Expand Up @@ -176,7 +183,7 @@ timeout_minutes: 20
# max_runtime_fetches: 10
```

See [Customizing agents — Harness YAML Structure](customizing-agents.md#harness-yaml-structure) for the full field reference (including optional `security`, `providers`, `plugins`, and runtime fetch blocks).
See [Configuring agents — Harness YAML Structure](customizing-agents.md#harness-yaml-structure) for the full field reference (including optional `security`, `providers`, `plugins`, and runtime fetch blocks).

The key pattern to understand is how data flows into the sandbox through `host_files`:

Expand Down Expand Up @@ -368,7 +375,7 @@ The post-script runs on the trusted runner with full credentials, but reads outp

## Step 6: Create skills (optional)

[Skills](../../glossary.md#skill) are Markdown documents mounted into the sandbox that provide domain knowledge the agent can reference. See [Customizing agents — Adding a Custom Skill](customizing-agents.md#adding-a-custom-skill) for how to create one.
[Skills](../../glossary.md#skill) are Markdown documents mounted into the sandbox that provide domain knowledge the agent can reference. See [Configuring agents — Adding a Custom Skill](customizing-agents.md#adding-a-custom-skill) for how to create one.

Place your skill at `.fullsend/customized/skills/my-skill/SKILL.md`, then reference it in both the agent frontmatter (`skills: [my-skill]`) and the harness (`skills: [customized/skills/my-skill]`).

Expand Down Expand Up @@ -619,7 +626,7 @@ When creating a new agent, you need these files:

## Reference

- [Customizing agents](customizing-agents.md) — override existing agent harnesses, skills, and policies
- [Configuring agents](customizing-agents.md) — override existing agent harnesses, skills, and policies
- [Bugfix workflow](bugfix-workflow.md) — how the built-in agents work together end to end
- [Getting Started](../getting-started/README.md) — prerequisite: admin setup guide
- [Architecture overview](../../architecture.md) — component vocabulary and execution stack
Expand Down
1 change: 1 addition & 0 deletions docs/guides/user/customizing-agents.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,7 @@ my-repo/

## See Also

- [Default, derived, and custom agents](../../agents/topics/default-vs-custom.md) - When does configuration cross into derived or custom agent territory?
- [Getting Started](../getting-started/) - Initial setup
- [Bugfix Workflow](bugfix-workflow.md) - How agents work together
- [Standalone Mint](../infrastructure/standalone-mint.md) - Running your own mint with custom agent roles
Expand Down
Loading
Loading