Skip to content

docs(cli): use hosted VL rerank defaults in CLI example (NVBug 6657245) - #2576

Draft
kheiss-uwzoo wants to merge 1 commit into
NVIDIA:mainfrom
kheiss-uwzoo:docs/6657245-cli-hosted-rerank-example
Draft

docs(cli): use hosted VL rerank defaults in CLI example (NVBug 6657245)#2576
kheiss-uwzoo wants to merge 1 commit into
NVIDIA:mainfrom
kheiss-uwzoo:docs/6657245-cli-hosted-rerank-example

Conversation

@kheiss-uwzoo

Copy link
Copy Markdown
Collaborator

Summary

  • Replace the CLI README rerank example that pointed at https://inference-api.nvidia.com/v1/rerank, NGC_INFERENCE_API_KEY, and the retired llama-3.2-nv-rerankqa-1b-v2 model.
  • Use the 26.08 hosted vision-language rerank defaults (ai.api.nvidia.com /reranking URL, nvidia/llama-nemotron-rerank-vl-1b-v2, NVIDIA_API_KEY) so customers can run the example with a self-service NVIDIA key.
  • Document that --rerank without --reranker-invoke-url uses the local GPU reranker, and that a Cohere-style /v1/rerank URL is a gateway route that needs a gateway-issued credential.

Fixes NVBug 6657245.

Test plan

  • Diff is docs-only: nemo_retriever/docs/cli/README.md
  • Endpoint and model match nemo_retriever/src/nemo_retriever/operators/rerank.py hosted VL defaults
  • QA: ingest a PDF to local LanceDB with hosted embeddings, then run the updated retriever query example with NVIDIA_API_KEY
  • Confirm ranking changes versus the no-rerank baseline

pre-draft: leakage, mkdocs --strict, ::a, ::p, ::r on the diff vs main

Base: upstream/main
Files: nemo_retriever/docs/cli/README.md

Check Result
Leakage (page roles + see [ CTAs) PASS — no see [ CTAs; CLI README is an allowed documentation path
Allowed paths PASS — 1 documentation file
mkdocs --strict PASS for this change — exit 1 from untracked leftover pages (custom-metadata.md, user-defined-stages.md), not this diff
::a audit PASS — hosted URL/model/key match product defaults; --rerank without invoke URL uses local GPU (verified in query/workflow.py)
::p polish Applied — customer-runnable hosted example first; gateway /v1/rerank guidance secondary
::r style 95% — no blocking issues

Code drift (not in this docs PR): nemo_retriever/src/nemo_retriever/operators/rerank.py:207 still documents https://inference-api.nvidia.com/v1/rerank in operator help.

PR: this draft

Replace the non-customer inference-api rerank example with the hosted llama-nemotron-rerank-vl-1b-v2 endpoint so NVIDIA_API_KEY works.
@kheiss-uwzoo kheiss-uwzoo self-assigned this Aug 24, 2026
@kheiss-uwzoo kheiss-uwzoo added 26.08 doc Improvements or additions to documentation labels Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

26.08 doc Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant