Skip to content

Add example: rag-knowledge-assistant with source-citation policies #54

Description

@kmadan

Goal

Create examples/rag-knowledge-assistant/ showing a retrieval-augmented assistant evaluated for citation accuracy, hallucination, source disclosure, and data-leakage.

Why this matters

RAG is the most common production LLM pattern. Compliance for RAG is its own discipline — citation accuracy, source-confidentiality, hallucination detection — and a documented example shows AICertify users how to encode those concerns.

Files likely involved

  • New: examples/rag-knowledge-assistant/ (six-file shape) over a small fixture corpus
  • examples/README.md table entry
  • Optionally: a small new gopal policy proposal for RAG-specific concerns (open in gopal repo)

Acceptance criteria

  • Contract with 6-10 interactions including: correctly-cited answer, no-source-available answer, hallucination test, sensitive-doc query
  • policy_config.yaml that exercises content-safety + transparency + accountability
  • Retrieved sources in interaction metadata
  • expected_report.md describes the citation-accuracy pass case and the common failure modes (uncited claim, fabricated source)
  • Honest scope: RAG eval is corpus-dependent — a green report on one corpus is not transferable

Help

Comment here if you want to work on this and I will help you get started. First-time contributors are welcome — open a draft PR early and I'll review.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    🏗️ help wantedExtra attention is needed📦 examplesApplication examples and templates🦜 llm-appsLangChain / LlamaIndex / agent-framework integrations

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions