From 4b0d9c01065580289ab4165ac7fc69698fcde56d Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Mon, 24 Aug 2026 09:58:06 -0700 Subject: [PATCH] docs(extraction): pass NVIDIA_API_KEY explicitly in Live RAG examples (NVBug 6622594) LiteLLM nvidia_nim does not read NVIDIA_API_KEY from the environment, so the README and API-key docs now pass os.environ/NVIDIA_API_KEY. --- docs/docs/extraction/api-keys.md | 4 ++++ docs/docs/extraction/nemo-retriever-api-reference.md | 2 ++ nemo_retriever/README.md | 6 ++++++ 3 files changed, 12 insertions(+) diff --git a/docs/docs/extraction/api-keys.md b/docs/docs/extraction/api-keys.md index 257399b74..959c4a5b2 100644 --- a/docs/docs/extraction/api-keys.md +++ b/docs/docs/extraction/api-keys.md @@ -23,6 +23,8 @@ On Windows PowerShell you can use `$env:NVIDIA_API_KEY = "nvapi-..."`. The SDK and CLI do not load a `.env` file automatically. For a full list of related variables and how to source a `.env` file into the shell, refer to [Environment configuration variables](environment-config.md). +For `LiteLLMClient` and `LLMJudge` with a `nvidia_nim/...` model, pass `api_key="os.environ/NVIDIA_API_KEY"`. LiteLLM does not read `NVIDIA_API_KEY` for that provider. + Hosted object-detection NIMs (Page Elements, Table Structure, Graphic Elements) cap inline base64 image payloads at about 180,000 characters (roughly 180 KB). This key authorizes those hosted inference calls. For size limits and what to do when an image exceeds the cap, refer to [Hosted Page Elements NIM image size limits](troubleshoot.md#hosted-page-elements-nim-image-size-limits). !!! note @@ -39,6 +41,8 @@ api_key="os.environ/NVIDIA_API_KEY" Use the provider's own variable name, for example `os.environ/OPENAI_API_KEY` for an OpenAI model. The reference is stored in graph JSON and resolved only when the operator is constructed or invoked on the worker. +For LiteLLM `nvidia_nim/...` models, use `os.environ/NVIDIA_API_KEY`. The worker resolves that reference and passes the value as `api_key`. If you omit `api_key`, LiteLLM looks up `NVIDIA_NIM_API_KEY` instead. + Literal keys remain available for non-persisted local execution, but attempting to serialize one raises an error. This prevents graph persistence from silently substituting an NVIDIA credential for another provider's key. For how persisted graphs store credential references, refer to [Persisted graphs are trusted configuration](nemo-retriever-api-reference.md#persisted-graphs-are-trusted-configuration) in the Python API guide. diff --git a/docs/docs/extraction/nemo-retriever-api-reference.md b/docs/docs/extraction/nemo-retriever-api-reference.md index 35ec850e1..42e833ba0 100644 --- a/docs/docs/extraction/nemo-retriever-api-reference.md +++ b/docs/docs/extraction/nemo-retriever-api-reference.md @@ -257,6 +257,8 @@ QAGenerationOperator( ) ``` +For LiteLLM `nvidia_nim/...` models, including the default `LiteLLMClient` and `LLMJudge` models, use `os.environ/NVIDIA_API_KEY`. Refer to [Authentication and API keys](api-keys.md#credential-references-in-persisted-graphs). + Serializing a graph containing a literal API key fails with a contextual error instead of guessing which provider credential should be used on a worker. diff --git a/nemo_retriever/README.md b/nemo_retriever/README.md index 685effa08..65b7ce728 100644 --- a/nemo_retriever/README.md +++ b/nemo_retriever/README.md @@ -482,6 +482,10 @@ uv pip install "nemo-retriever[llm]" export NVIDIA_API_KEY=nvapi-... ``` +The default Live RAG model uses LiteLLM's `nvidia_nim` provider. LiteLLM does not +read `NVIDIA_API_KEY` for that provider. Pass `api_key="os.environ/NVIDIA_API_KEY"` +so the same key is forwarded on each request. + Single-query live RAG. Point `vdb_kwargs["uri"]` at any table built above; the embedding model in `embed_kwargs` must match the one used during ingestion so query vectors land in the same embedding space as the stored chunks. @@ -501,6 +505,7 @@ retriever = Retriever( ) llm = LiteLLMClient.from_kwargs( model="nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5", + api_key="os.environ/NVIDIA_API_KEY", temperature=0.0, max_tokens=512, ) @@ -522,6 +527,7 @@ from nemo_retriever.models.llm import LLMJudge judge = LLMJudge.from_kwargs( model="nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5", + api_key="os.environ/NVIDIA_API_KEY", temperature=0.1, max_tokens=4096, )