Skip to content

Agent version rollback silently drops GuardrailIds, Scope and AgentType #404

Description

@mattbrailsford

Which component is this issue related to?

Umbraco.AI.Agent

Which Umbraco AI version are you using?

18.2.0-rc.3 (v18/dev). Also present on 17.1.7 (v17/dev) — both active lines are affected.

Bug summary

AIAgentVersionableEntityAdapter.CreateSnapshot omits GuardrailIds and Scope, and RestoreFromSnapshot cannot read the AgentType it writes. Because RollbackAsync restores from that snapshot and saves it as the current version, rolling an agent back to a previous version silently strips its guardrails and its scope rules, and converts an Orchestrated agent into a Standard one.

Three distinct faults, all in Umbraco.AI.Agent.Core/Agents/AIAgentVersionableEntityAdapter.cs:

  1. GuardrailIds and Scope are never written to the snapshot. CreateSnapshot builds an anonymous object that simply does not include them, so RestoreFromSnapshot falls back to the AIAgent defaults — [] and null.

  2. The loss is invisible in the diff. CompareVersions does not compare either field, so the version-history UI reports no change between an agent and a restore of it that has had both fields emptied.

  3. AgentType round-trips incorrectly. CreateSnapshot serializes it as a string ("agentType":"Orchestrated"), but RestoreFromSnapshot only reads it when atEl.ValueKind == JsonValueKind.Number, so it always falls through to AIAgentType.Standard. Config is then deserialized as the wrong concrete type, discarding WorkflowId and Settings.

Fault 3 also breaks the invariant documented in Umbraco.AI.Agent.Deploy/CLAUDE.md: "AgentType is immutable: Once an agent is created with a type, it cannot be changed."

Specifics

Probed directly against the adapter (no mocking of the methods under test — only the two constructor dependencies, which are unused on these paths).

Guardrails and scope, with an agent carrying 2 guardrails and 1 allow rule:

SNAPSHOT: {"id":"27e91dbd-…","alias":"probe","name":"Probe","description":null,
           "agentType":"Standard","profileId":"9dfa0fa6-…","surfaceIds":null,
           "config":"{…}","starterPrompts":[],"isActive":true,"version":1,
           "dateCreated":"…","dateModified":"…","createdByUserId":null,"modifiedByUserId":null}

ORIGINAL guardrails=2 scopeAllowRules=1
RESTORED guardrails=0 scope=NULL
CHANGES REPORTED: (none)

Note the snapshot JSON has no guardrailIds key and no scope key at all.

Agent type, with an Orchestrated agent whose config carries a WorkflowId:

ORIGINAL type=Orchestrated configType=AIOrchestratedAgentConfig
RESTORED type=Standard      configType=AIStandardAgentConfig

The WorkflowId is gone — the config was rebuilt as the wrong type.

Steps to reproduce

On a clean install with Umbraco.AI.Agent installed:

  1. Create a Standard agent. Assign it one or more guardrails and add a scope rule (for example, allow section content / entity type document). Save.
  2. Edit any other field on the agent (for example its description) and save again, so there are two versions.
  3. Open the agent's version history and roll back to version 1.
  4. Reopen the agent.

Expected: the agent's guardrails and scope rules are exactly as they were in version 1.
Actual: the guardrail list is empty and the scope is cleared. The version-history diff for the rollback lists no change to either field.

For fault 3:

  1. Create an Orchestrated agent with a workflow selected. Save, then save again to create a second version.
  2. Roll back to version 1.

Expected: it is still an Orchestrated agent with its workflow.
Actual: it is now a Standard agent, and the workflow is gone.

Expected result / actual result

A version snapshot should be a complete representation of the entity, so that rollback is lossless for every persisted field. At present three fields are dropped, two of them (GuardrailIds, Scope) being the controls that constrain what an agent may do and where it may do it — so a rollback silently widens an agent's reach, with nothing in the UI indicating it happened.

Suggested fix, for whoever picks this up:

  • Add GuardrailIds and Scope to CreateSnapshot and read them back in RestoreFromSnapshot, tolerating snapshots written before the fix (treat missing as "unchanged from current" rather than "empty", or at minimum document that older snapshots cannot restore them).
  • Accept both string and number for agentType in RestoreFromSnapshot, so existing snapshots written by the current code still restore correctly.
  • Extend CompareVersions to cover all three fields, so the omission cannot recur silently.

StarterPrompts was added to all three methods correctly and is not affected.

Dependencies

None beyond Umbraco.AI.Agent itself. Affects both v17/dev (17.1.7) and v18/dev (18.2.0-rc.3), so it needs fixing on both active support lines.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions