Skip to content

AGENTS.md

Latest commit

 

History

History
92 lines (67 loc) · 3.9 KB

AGENTS.md

File metadata and controls

92 lines (67 loc) · 3.9 KB
Raw

Agent Instructions

This repo provides source-grounded retrieval infrastructure over a local corpus. It is not a chatbot application, and it is not asking agents to hand-parse PDFs.

When a user asks a question about the corpus, answer from the retrieval pipeline, not from memory and not from ad hoc file extraction.

First Command Choice

Run commands from the repo root.

Prefer the project virtualenv executable because it has the repo's PDF and retrieval dependencies:

.venv/bin/rag status
.venv/bin/rag context "<question>" --top-k 8

Use bare rag ... only after activating the virtualenv:

. .venv/bin/activate
rag status
rag context "<question>" --top-k 8

If .venv/bin/rag does not exist, set up the environment first:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install -e .
rag init

If the package is installed but the console script is unavailable, run the module with the virtualenv Python:

.venv/bin/python -m rag_pipeline.cli status
.venv/bin/python -m rag_pipeline.cli context "<question>" --top-k 8

Standard Answer Workflow

  1. Run .venv/bin/rag status to confirm the current database has documents, chunks, and vectors.
  2. If there are zero documents, rebuild the intended corpus before answering.
  3. Run .venv/bin/rag context "<question>" --top-k 8 --format json to get an LLM-ready context pack.
  4. Check answerability.status before answering. Treat low_confidence and no_results as insufficient evidence.
  5. Use only the returned results[*].text, summary, keywords, and metadata for claims about the corpus.
  6. Cite claims with the returned citation_markdown fields when present.
  7. If the context is insufficient, say what is missing and suggest a narrower or better query.
  8. If the first context pack is broad, run one narrower follow-up query using key terms found in the first results.

Fresh Corpus Workflow

When the user wants to evaluate a new production corpus:

  1. Put files in corpus/.
  2. Run .venv/bin/rag rebuild corpus --yes.
  3. Run .venv/bin/rag status.
  4. Run .venv/bin/rag eval after updating eval/questions.yml for the new corpus.
  5. Use .venv/bin/rag context "<question>" for downstream answering.

rag rebuild ... --yes deletes generated data and creates a clean database. It does not delete source files. If the input produces zero documents, rebuild fails unless --allow-empty is supplied.

For an isolated trial rebuild, use:

.venv/bin/rag --data-dir data-production-test rebuild corpus --yes
.venv/bin/rag --data-dir data-production-test status
.venv/bin/rag --data-dir data-production-test eval --file eval/your-production-questions.yml
.venv/bin/rag --data-dir data-production-test context "<question>"

If corpus/ only contains README.md, ask the user to add real source files before treating the corpus as production-ready.

Retrieval Rules

  • Prefer rag context over direct database queries for answering.
  • Use rag search when debugging ranking.
  • Use rag inspect-doc <document_id> and rag inspect-chunk <chunk_id> for traceability.
  • Use returned citation_markdown fields exactly when present.
  • Do not invent source links; use returned source_link, page_link, source_markdown, or citation_markdown fields.
  • Never assume the current corpus matches prior runs; always check rag status.

Do Not Do This

  • Do not run pdftotext, fitz, pypdf, or custom PDF parsing to answer normal corpus questions. The pipeline already extracted, chunked, enriched, indexed, and linked the sources.
  • Do not use system Python for repo commands unless the virtualenv is unavailable and you are setting it up.
  • Do not answer from a single file path just because a filename looks relevant. Retrieve ranked context first.
  • Do not cite raw filenames when citation_markdown is available.
  • Do not treat an old terminal result as current after rag rebuild, rag reset, or a data-directory change.