Skip to content

feat(logfire): add ManagedPrompt capability#257

Open
dmontagu wants to merge 6 commits into
mainfrom
feature/logfire-managed-prompt
Open

feat(logfire): add ManagedPrompt capability#257
dmontagu wants to merge 6 commits into
mainfrom
feature/logfire-managed-prompt

Conversation

@dmontagu
Copy link
Copy Markdown
Contributor

@dmontagu dmontagu commented May 26, 2026

Adds ManagedPrompt, a capability that backs an agent's instructions with a Logfire-managed prompt so you can version, label, roll out, and A/B test prompts from the Logfire UI without redeploying.

Usage

from pydantic_ai import Agent
from pydantic_ai_harness.logfire import ManagedPrompt

agent = Agent(
    'openai:gpt-5',
    capabilities=[
        ManagedPrompt(
            'support_agent',
            default='You are a helpful customer support agent. Be friendly and concise.',
            label='production',
        )
    ],
)

The slug support_agent is declared as the managed variable prompt__support_agent (Logfire's Prompt-management naming; hyphens → underscores). The default is the offline safety net used when no remote value is available.

How it works

  • Resolves the prompt once per run inside wrap_run, keeping the ResolvedVariable open as a context manager for the whole run — so the selected label/version are attached as baggage to every child span (model requests, tools, and the run span). Runs outermost (wraps=[Instrumentation]) so the run span is tagged too.
  • The per-run resolution is held in a ContextVar, so a single capability instance is safe across concurrent runs.

API

  • ManagedPrompt(slug, *, default, label=, targeting_key=, attributes=, render_template=, logfire_instance=).
  • targeting_key/attributes accept a static value or a callable of RunContext (e.g. lambda ctx: ctx.deps.user_id).
  • Pass an existing logfire.Variable instead of a slug to use your own declaration. Declaring the same slug twice is fine (each builds its own backing variable).
  • render_template=True renders the prompt as Handlebars against deps (like TemplateStr); off by default. Requires pydantic-ai-slim[spec].
  • ManagedPrompt.resolved exposes the active run's ResolvedVariable (value/label/version/reason).

Packaging

  • New optional extra: pydantic-ai-harness[logfire] (logfire>=4.31.0, the CI-tested floor).
  • Built directly on the public logfire.Variable API (no registry side effects, so duplicate slugs don't raise on any supported logfire version).
  • Lives in pydantic_ai_harness.logfire; the test directory is named logfire_variables to avoid shadowing the third-party logfire package under pyright's tests execution-environment root.

Testing

  • 100% branch coverage; pyright-strict and ruff clean.
  • Covers code-default fallback, provider-backed resolution via LocalVariablesOptions (remote value + label baggage), span/baggage assertions via capfire inline snapshots, targeting, dedup, the prompt__ prefix warning, and Handlebars rendering.

Future: first-party Managed capability

A broader pydantic_ai.managed.logfire.Managed capability is in flight upstream in pydantic-ai#5107 — it'll cover instructions, model settings, and whole-spec variables, with a planned phase-2 contribute_run_spec hook. Once it ships, users will be able to import it directly from pydantic-ai and ManagedPrompt here can either delegate to it or stay as the prompt-only convenience.

The README's ManagedPrompt section now carries a callout pointing at #5107 so users picking it up today know where things are headed.

Related

  • #90 — Prompt Templates / Parameterized User Prompts (complementary: user-prompt side)
  • #99 — Cache-Aware Prompt Construction (this capability is the first concrete example of the trade-off; see the new README section)
  • #147ModelSettings.cache_control (the cache-control surface that managed-prompt rollouts will interact with)

Back an agent's instructions with a Logfire-managed prompt, resolved once per
run inside wrap_run so the selected label/version are attached as baggage to
every child span. Lets prompts be iterated, versioned, and rolled out from the
Logfire UI without redeploying, with a code default as the offline safety net.

Lives in pydantic_ai_harness.logfire; adds a 'logfire' optional dependency extra.
The test directory is named logfire_variables to avoid shadowing the third-party
logfire package under pyright's tests execution-environment root.
@dmontagu dmontagu force-pushed the feature/logfire-managed-prompt branch from 7707a3c to f20d60d Compare May 26, 2026 09:12
@dmontagu dmontagu changed the title feat(logfire_variables): add ManagedPrompt capability feat(logfire): add ManagedPrompt capability May 26, 2026
Copy link
Copy Markdown

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 5 additional findings.

Open in Devin Review

dmontagu added 2 commits May 27, 2026 00:15
The examples fall back to their code default with no Logfire backend and run
clean under a mocked model, matching the main README's unmarked quick-start.
Comment thread pydantic_ai_harness/logfire/_managed_prompt.py Outdated
Comment thread pydantic_ai_harness/logfire/_managed_prompt.py Outdated
Comment thread pydantic_ai_harness/logfire/_managed_prompt.py Outdated
Comment thread pydantic_ai_harness/logfire/_managed_prompt.py Outdated
Comment thread pydantic_ai_harness/logfire/_managed_prompt.py Outdated
Comment thread pydantic_ai_harness/logfire/_managed_prompt.py
Comment thread pydantic_ai_harness/logfire/README.md Outdated
dmontagu and others added 3 commits May 27, 2026 00:57
Logfire calls a managed prompt by its name, so `ManagedPrompt(name=...)`
reads better than `ManagedPrompt(prompt=...)`. Also clarifies the `label`
and `targeting_key` docstrings per review.
- README: prompt-cache trade-off section, notes on once-per-run resolution,
  label+version baggage semantics, and the targeting_context outer setter.
  Callout pointing at pydantic-ai#5107 so users know a first-party `Managed`
  capability is in flight.
- Switch to the public `logfire.variables` import paths (was reaching into
  `logfire.variables.variable` / `.abstract`).
- Pull `pydantic-ai-slim[spec]` from the `logfire` extra so `render_template=True`
  works without a separate install.
- Warn when `logfire_instance` is passed alongside a prebuilt `Variable` (silently
  ignored before).
- Scrub `code.lineno` from span snapshots and expand the volatile-attributes
  comment so the snapshot doesn't rot on line shifts.
- Wrap the provider-backed test in a context manager that restores the module's
  baseline Logfire config in `finally`, so future tests don't inherit a provider.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants