Skip to content
Draft
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 2 additions & 2 deletions docs/blog/entries/annotate-your-llm-response-preview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ This is useful to:
- Run custom evaluation workflows
- Measure application performance in real-time

Check out the how to [annotate traces from API](/observability/trace-with-python-sdk/annotate-traces) for more details. Or try our new tutorial (available as [jupyter notebook](https://github.com/Agenta-AI/agenta/blob/main/examples/jupyter/capture_user_feedback.ipynb)) [here](/tutorials/cookbooks/capture-user-feedback).
Check out the how to [annotate traces from API](/1.0/observability/trace-with-python-sdk/annotate-traces) for more details. Or try our new tutorial (available as [jupyter notebook](https://github.com/Agenta-AI/agenta/blob/main/examples/jupyter/capture_user_feedback.ipynb)) [here](/1.0/tutorials/cookbooks/capture-user-feedback).

<Image
style={{
Expand Down Expand Up @@ -50,7 +50,7 @@ This is useful to:
- Run custom evaluation workflows
- Measure application performance in real-time

Check out the how to [annotate traces from API](/observability/trace-with-python-sdk/annotate-traces) for more details. Or try our new tutorial (available as [jupyter notebook](https://github.com/Agenta-AI/agenta/blob/main/examples/jupyter/observability/capture_user_feedback.ipynb)) [here](/tutorials/cookbooks/capture-user-feedback).
Check out the how to [annotate traces from API](/1.0/observability/trace-with-python-sdk/annotate-traces) for more details. Or try our new tutorial (available as [jupyter notebook](https://github.com/Agenta-AI/agenta/blob/main/examples/jupyter/observability/capture_user_feedback.ipynb)) [here](/1.0/tutorials/cookbooks/capture-user-feedback).

<Image
style={{
Expand Down
6 changes: 3 additions & 3 deletions docs/blog/entries/chat-sessions-observability.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -148,9 +148,9 @@ Identify latency issues across entire conversations, not just single requests. S

Learn more in our documentation:

- [Track Chat Sessions (Python SDK)](/observability/trace-with-python-sdk/track-chat-sessions)
- [Session Tracking (OpenTelemetry)](/observability/trace-with-opentelemetry/session-tracking)
- [Observability Overview](/observability/overview)
- [Track Chat Sessions (Python SDK)](/1.0/observability/trace-with-python-sdk/track-chat-sessions)
- [Session Tracking (OpenTelemetry)](/1.0/observability/trace-with-opentelemetry/session-tracking)
- [Observability Overview](/1.0/observability/overview)

## What's Next

Expand Down
4 changes: 2 additions & 2 deletions docs/blog/entries/customize-llm-as-a-judge-output-schemas.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ The LLM-as-a-Judge evaluator now supports custom output schemas. Create multiple

You can configure output types (binary, multiclass), include reasoning to improve prediction quality, or provide a raw JSON schema with any structure you define. Use these custom schemas in your evaluations to capture exactly the feedback you need.

Learn more in the [LLM-as-a-Judge documentation](/evaluation/configure-evaluators/llm-as-a-judge).
Learn more in the [LLM-as-a-Judge documentation](/1.0/evaluation/configure-evaluators/llm-as-a-judge).

</Summary>

Expand Down Expand Up @@ -93,4 +93,4 @@ To use custom output schemas with LLM-as-a-Judge:
4. For advanced use, provide your JSON schema
5. Run your evaluation

Learn more in the [LLM-as-a-Judge documentation](/evaluation/configure-evaluators/llm-as-a-judge).
Learn more in the [LLM-as-a-Judge documentation](/1.0/evaluation/configure-evaluators/llm-as-a-judge).
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ You can now trigger automations when a prompt deployment happens in Agenta. Send

This makes it easier to connect prompt deployments to CI, repository sync jobs, and pull request workflows. If your GitHub workflow needs the latest prompt content, fetch it from Agenta during the run and commit the result back to your repo.

Learn more: [Webhooks](/prompt-engineering/integrating-prompts/webhooks) | [GitHub Automations](/prompt-engineering/integrating-prompts/github)
Learn more: [Webhooks](/1.0/prompt-engineering/integrating-prompts/webhooks) | [GitHub Automations](/1.0/prompt-engineering/integrating-prompts/github)

</Summary>

Expand Down Expand Up @@ -56,6 +56,6 @@ For GitHub automations, Agenta calls the GitHub API directly. `repository_dispat

Start with these guides:

- [Webhooks](/prompt-engineering/integrating-prompts/webhooks)
- [GitHub](/prompt-engineering/integrating-prompts/github)
- [Fetch Prompts via SDK/API](/prompt-engineering/integrating-prompts/fetch-prompt-programatically)
- [Webhooks](/1.0/prompt-engineering/integrating-prompts/webhooks)
- [GitHub](/1.0/prompt-engineering/integrating-prompts/github)
- [Fetch Prompts via SDK/API](/1.0/prompt-engineering/integrating-prompts/fetch-prompt-programatically)
12 changes: 6 additions & 6 deletions docs/blog/entries/documentation-architecture-overhaul.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ We've completely rewritten and restructured our documentation with a new archite

Key improvements include:
- **[Diataxis Framework](https://diataxis.fr/)**: Organized content into Tutorials, How-to Guides, Reference, and Explanation sections for better discoverability
- **[Expanded Observability Docs](/observability/overview)**: Added missing documentation for tracing, annotations, and observability features
- **[JavaScript/TypeScript Support](/observability/quick-start-opentelemetry)**: Added code examples and documentation for JavaScript developers alongside Python
- **[Expanded Observability Docs](/1.0/observability/overview)**: Added missing documentation for tracing, annotations, and observability features
- **[JavaScript/TypeScript Support](/1.0/observability/quick-start-opentelemetry)**: Added code examples and documentation for JavaScript developers alongside Python
- **Ask AI Feature**: Ask questions directly to the documentation for instant answers

</Summary>
Expand All @@ -33,10 +33,10 @@ We've reorganized all documentation using the [Diataxis framework](https://diata

One of the biggest gaps in our previous documentation was observability. We've added comprehensive documentation covering:

- [Tracing with OpenTelemetry](/observability/trace-with-opentelemetry/getting-started)
- [Tracing LLM applications with JS/TS](/observability/quick-start-opentelemetry)
- [Using the Metrics API to fetch metrics](/observability/query-data/analytics-data)
- [Using the Query API to fetch traces](/observability/query-data/query-api)
- [Tracing with OpenTelemetry](/1.0/observability/trace-with-opentelemetry/getting-started)
- [Tracing LLM applications with JS/TS](/1.0/observability/quick-start-opentelemetry)
- [Using the Metrics API to fetch metrics](/1.0/observability/query-data/analytics-data)
- [Using the Query API to fetch traces](/1.0/observability/query-data/query-api)

### JavaScript/TypeScript Support

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ We've made significant improvements across Agenta with a major documentation ove

We've completely rewritten our prompt management and prompt engineering documentation.

Start exploring the new documentation in our updated [Quick Start Guide](/prompt-engineering/quick-start).
Start exploring the new documentation in our updated [Quick Start Guide](/1.0/prompt-engineering/quick-start).

**New Model Support**:

Expand Down Expand Up @@ -55,7 +55,7 @@ We've made significant improvements across Agenta with a major documentation ove

We've completely rewritten our prompt management and prompt engineering documentation.

Start exploring the new documentation in our updated [Quick Start Guide](/prompt-engineering/quick-start).
Start exploring the new documentation in our updated [Quick Start Guide](/1.0/prompt-engineering/quick-start).

**New Model Support**:

Expand Down
2 changes: 1 addition & 1 deletion docs/blog/entries/dspy-integration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,4 @@ We've added DSPy integration to Agenta. You can now trace and debug your DSPy ap
loading="lazy"
/>

[**View the full DSPy integration →**](/integrations/frameworks/dspy/observability)
[**View the full DSPy integration →**](/1.0/integrations/frameworks/dspy/observability)
4 changes: 2 additions & 2 deletions docs/blog/entries/evaluation-sdk.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ You can now run programmatic evaluations of complex AI agents and workflows dire

The SDK lets you create test sets in code or fetch them from Agenta. You can use built-in evaluators like LLM-as-a-Judge, semantic similarity, or regex matching. You can also write custom Python evaluators. The SDK evaluates end-to-end workflows or specific spans in execution traces. Evaluations run on your own infrastructure; results display in the Agenta dashboard.

Check out the [Evaluation SDK documentation](/evaluation/evaluation-from-sdk/quick-start) to get started.
Check out the [Evaluation SDK documentation](/1.0/evaluation/evaluation-from-sdk/quick-start) to get started.

</Summary>

Expand Down Expand Up @@ -122,4 +122,4 @@ Every evaluation run gets a shareable dashboard link. The dashboard shows full e

## Next Steps

Check out the [Quick Start Guide](/evaluation/evaluation-from-sdk/quick-start) to build your first evaluation.
Check out the [Quick Start Guide](/1.0/evaluation/evaluation-from-sdk/quick-start) to build your first evaluation.
4 changes: 2 additions & 2 deletions docs/blog/entries/filtering-traces-by-annotation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The new annotation filtering lets you find:
- Spans evaluated by a specific evaluator
- Spans with user feedback like `success=True`

This enables powerful workflows: [capture user feedback](/tutorials/cookbooks/capture-user-feedback) from your app, filter to find traces with bad feedback, add them to test sets, and improve your prompts based on real user data.
This enables powerful workflows: [capture user feedback](/1.0/tutorials/cookbooks/capture-user-feedback) from your app, filter to find traces with bad feedback, add them to test sets, and improve your prompts based on real user data.

</Summary>

Expand Down Expand Up @@ -47,7 +47,7 @@ Filter traces based on evaluations and feedback:

This feature enables powerful workflows:

1. **Capture user feedback** from your application using our API ([see tutorial](/tutorials/cookbooks/capture-user-feedback))
1. **Capture user feedback** from your application using our API ([see tutorial](/1.0/tutorials/cookbooks/capture-user-feedback))
2. **Filter traces** to find those with bad feedback or low scores
3. **Add them to test sets** to track problematic cases
4. **Improve your prompts** based on real user feedback
Expand Down
4 changes: 2 additions & 2 deletions docs/blog/entries/jinja2-template-support.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import Image from "@theme/IdealImage";

You can now use Jinja2 templates in your prompts. Jinja2 is available in both the Playground and in prompt management.

Learn more in our [blog post](https://agenta.ai/blog/launch-week-2-day-5-jinja2-prompt-templates) or check the [documentation](/prompt-engineering/playground/using-playground#switching-template-formats).
Learn more in our [blog post](https://agenta.ai/blog/launch-week-2-day-5-jinja2-prompt-templates) or check the [documentation](/1.0/prompt-engineering/playground/using-playground#switching-template-formats).

</Summary>

Expand All @@ -23,7 +23,7 @@ We're excited to announce a powerful update to the Agenta playground. You can no

This means you can add sophisticated logic directly into your prompt templates. Use conditional statements, apply filters to variables, and transform data on the fly.

Learn more in our [blog post](https://agenta.ai/blog/launch-week-2-day-5-jinja2-prompt-templates) or check the [documentation](/prompt-engineering/playground/using-playground#switching-template-formats).
Learn more in our [blog post](https://agenta.ai/blog/launch-week-2-day-5-jinja2-prompt-templates) or check the [documentation](/1.0/prompt-engineering/playground/using-playground#switching-template-formats).

## Example

Expand Down
2 changes: 1 addition & 1 deletion docs/blog/entries/json-multi-field-match.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -129,4 +129,4 @@ Existing JSON Field Match configurations continue to work. We recommend migratin

## Next Steps

Learn more about configuring and using the JSON Multi-Field Match evaluator in the [Classification and Entity Extraction Evaluators](/evaluation/configure-evaluators/classification-entity-extraction#json-multi-field-match) documentation.
Learn more about configuring and using the JSON Multi-Field Match evaluator in the [Classification and Entity Extraction Evaluators](/1.0/evaluation/configure-evaluators/classification-entity-extraction#json-multi-field-match) documentation.
2 changes: 1 addition & 1 deletion docs/blog/entries/llamaindex-integration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ The integration is auto-instrumentation - just add one line of code and you'll s

This helps when you need to understand what's happening inside your RAG pipeline, track performance bottlenecks, or debug issues in production.

Check out the [tutorial](/integrations/frameworks/llamaindex/observability) and the [Jupyter notebook](https://github.com/Agenta-AI/agenta/blob/main/examples/jupyter/llamaindex_integration.ipynb) for more details.
Check out the [tutorial](/1.0/integrations/frameworks/llamaindex/observability) and the [Jupyter notebook](https://github.com/Agenta-AI/agenta/blob/main/examples/jupyter/llamaindex_integration.ipynb) for more details.
4 changes: 2 additions & 2 deletions docs/blog/entries/multiple-metrics-in-human-evaluation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ We rebuilt the human evaluation workflow from scratch. Now you can set multiple

This lets you evaluate the same output on different metrics like **relevance** or **completeness**. You can also create binary, numerical scores, or even use strings for **comments** or **expected answer**.

Watch the video below and read the [post](/changelog/multiple-metrics-in-human-evaluation) for more details. Or check out the [docs](/evaluation/human-evaluation/quick-start) to learn how to use the new human evaluation workflow.
Watch the video below and read the [post](/changelog/multiple-metrics-in-human-evaluation) for more details. Or check out the [docs](/1.0/evaluation/human-evaluation/quick-start) to learn how to use the new human evaluation workflow.

<div style={{display: 'flex', justifyContent: 'center', marginTop: "20px", marginBottom: "20px", flexDirection: 'column', alignItems: 'center'}}>
<iframe
Expand Down Expand Up @@ -46,6 +46,6 @@ This unlocks a whole new set of use cases:
- Use human evaluation to bootstrap automatic evaluation. You can annotate your outputs with the expected answer or a rubic, then use it to set up an automatic evaluation.


Watch the video below and read the [post](/changelog/multiple-metrics-in-human-evaluation) for more details. Or check out the [docs](/evaluation/human-evaluation/quick-start) to learn how to use the new human evaluation workflow.
Watch the video below and read the [post](/changelog/multiple-metrics-in-human-evaluation) for more details. Or check out the [docs](/1.0/evaluation/human-evaluation/quick-start) to learn how to use the new human evaluation workflow.

---
Loading
Loading