diff --git a/docs/docs/extraction/multimodal-extraction.md b/docs/docs/extraction/multimodal-extraction.md index c0d7e27bc..f9c27d406 100644 --- a/docs/docs/extraction/multimodal-extraction.md +++ b/docs/docs/extraction/multimodal-extraction.md @@ -49,17 +49,17 @@ NeMo Retriever Library detects tables as structured page elements, processes the Charts and infographic regions are classified with other page layout elements (tables, text blocks, titles) and processed through layout detection and OCR. `extract_charts` and `extract_infographics` are enabled by default. Outputs use the same metadata schema as other extracted objects. -!!! important "Chart modality requires the default layout path" - [Nemotron Parse v1.2](https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-v1.2) semantic classes do not include `Chart` or `Infographic`. The model labels regions as `Text`, `Table`, `Picture`, `Caption`, `List-item`, `Section-header`, and similar types instead. +!!! important "Chart modality and Nemotron Parse" + Local Hugging Face inference still uses [Nemotron Parse v1.2](https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-v1.2). That model does not emit `Chart` or `Infographic` semantic classes. It labels regions as `Text`, `Table`, `Picture`, `Caption`, `List-item`, `Section-header`, and similar types instead. - When you set `method="nemotron_parse"`: + The optional Helm self-hosted Parse NIM defaults to [Nemotron Parse v2.0](https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0), which adds chart-aware `` output. Even with v2.0, prefer the default **pdfium** layout path (page-elements detection and OCR) when you need reliable chart and infographic modality rows for filtered retrieval. + + When you set `method="nemotron_parse"` on the local v1.2 path: - The pipeline does not produce `chart` or `infographic` modality rows, even when `extract_charts=True` or `extract_infographics=True`. - Chart- and infographic-filtered retrieval (for example, queries scoped to figure or chart content) returns no hits. - Chart-heavy and infographic-heavy pages are typically emitted as `Picture` or other non-chart modalities. - For chart and infographic detection and modality-specific retrieval, use the default **pdfium** layout path (page-elements detection and OCR), not `method="nemotron_parse"`. - Chart-labeled PDF regions are **not** routed through the Omni caption stage; they remain on the layout-and-OCR path. For scope and validation guidance, refer to [Image captioning](#image-captioning). For natural-language infographic descriptions, optionally enable [image captioning](#image-captioning) and set `caption_infographics=True` when you need VLM captions on infographic regions. diff --git a/docs/docs/extraction/prerequisites-support-matrix.md b/docs/docs/extraction/prerequisites-support-matrix.md index 5347bbe24..6fe81c295 100644 --- a/docs/docs/extraction/prerequisites-support-matrix.md +++ b/docs/docs/extraction/prerequisites-support-matrix.md @@ -124,6 +124,8 @@ Optional advanced features (audio and video transcription, Nemotron Parse, Omni The production Helm chart reconciles NIM microservices through `nimOperator..enabled`. Four core NIMs are **enabled by default** and auto-wired into the retriever service; optional NIMs reconcile only when you opt in. For chart keys, image overrides, and enablement, refer to the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#nim-operator-sub-stack) and [Recommended minimal install](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#recommended-minimal-install-2608). +The chart service image defaults to `nvcr.io/nvstaging/nim/nrl-service:26.08-RC1` for this release candidate. Override `service.image.repository` and `service.image.tag` only when you pin a different build. + | Helm flag | NIM | Default image (`repository:tag`) | Role | Enabled by default | |-----------|-----|----------------------------------|------|--------------------| | `page_elements` | [nemotron-page-elements-v3](https://build.nvidia.com/nvidia/nemotron-page-elements-v3) | `nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1` | Page layout and element detection | Yes | @@ -131,10 +133,10 @@ The production Helm chart reconciles NIM microservices through `nimOperator. @@ -182,12 +184,12 @@ When you call [NVIDIA-hosted NIMs](deployment-options.md#when-to-use-nvidia-host NVIDIA Build and self-hosted Nemotron Parse use distinct request contracts: - **Hosted Build** (`https://integrate.api.nvidia.com/v1/chat/completions`) resolves to model ID `nvidia/nemotron-parse` and uses an image-only tool-call contract. - - **Self-hosted Parse v1.2** uses `nvidia/nemotron-parse-v1.2` and the tagged text-prompt contract. The Helm chart defaults to `nvcr.io/nim/nvidia/nemotron-parse-v1.2:1.7.0-variant`. - - **Self-hosted Parse v2.0** uses `nvidia/nemotron-parse-v2.0` and its v2.0 contract. It is an optional Helm image override, not the chart default. + - **Self-hosted Parse v2.0** uses `nvidia/nemotron-parse-v2.0` and its v2.0 contract. The Helm chart defaults to `nvcr.io/nim/nvidia/nemotron-parse-v2.0:2.0.8-variant`. + - **Self-hosted Parse v1.2** uses `nvidia/nemotron-parse-v1.2` and the tagged text-prompt contract. It is an optional Helm image override. To use hosted Build, set `nemotron_parse_invoke_url` to the Build chat-completions URL and set `method="nemotron_parse"`. You can normally omit `nemotron_parse_model` so the library selects the model automatically. If you set `nemotron_parse_model` explicitly, it must match the endpoint contract. Mixed Build and self-hosted endpoint lists require an explicit model. - When the chart manages the Parse NIM, it wires the model that matches the selected v1.2 or v2.0 image. To select Parse v2.0, enable Parse and set `nimOperator.nemotron_parse.image.repository=nvcr.io/nim/nvidia/nemotron-parse-v2.0` and `nimOperator.nemotron_parse.image.tag=2.0.8-variant`. For a direct external v2.0 endpoint, set `nemotron_parse_model="nvidia/nemotron-parse-v2.0"` explicitly with its invoke URL. + When the chart manages the Parse NIM, it wires the model that matches the selected v2.0 or v1.2 image. To select Parse v1.2, enable Parse and set `nimOperator.nemotron_parse.image.repository=nvcr.io/nim/nvidia/nemotron-parse-v1.2` and `nimOperator.nemotron_parse.image.tag=1.7.0-variant`. For a direct external Parse endpoint, set `nemotron_parse_model` explicitly with its invoke URL. For model/endpoint mismatch symptoms, refer to [Nemotron Parse model and endpoint mismatch](troubleshoot.md#nemotron-parse-model-endpoint-mismatch). @@ -237,7 +239,7 @@ Model repositories and NIM references are linked in [Core and Advanced Pipeline ⁴ Self-hosted [audio/video extraction](audio-video.md) through Parakeet ASR (`parakeet-1-1b-ctc-en-us:1.5.0`, `nimOperator.audio`) is **not supported** on **B200**, other **Blackwell** GPUs (compute capability 12.0), including RTX PRO 6000 Blackwell and RTX PRO 4500 Blackwell, or **H200 NVL**. Core PDF and multimodal extraction on those GPUs is unchanged. Video workflows that depend on Parakeet for speech transcription are affected the same way. `NIMService` for `nimOperator.audio` may stay not Ready or enter `CrashLoopBackOff` while building the Riva/TensorRT engine (for example ONNX Runtime IR version, cuDNN visibility, or FP8 tactic errors). Use a supported dedicated GPU (for example H100 or A100), [hosted Parakeet on build.nvidia.com](audio-video.md#parakeet-hosted-inference-build-nvidia), or set `nimOperator.audio.enabled=false`. -³ Opt-in Omni captioning uses the [nemotron-3-nano-omni-30b-a3b-reasoning](https://docs.api.nvidia.com/nim/reference/nvidia-nemotron-3-nano-omni-30b-a3b-reasoning) NIM (`nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:1.7.0-variant`). BF16 requires at least 80 GB total GPU memory for this NIM deployment; refer to the [VLM NIM support matrix](https://docs.nvidia.com/nim/vision-language-models/latest/support-matrix.html#nemotron-3-nano-omni-30b-a3b-reasoning). L40S requires two GPUs. A100 40GB, A10G, and RTX PRO 4500 are below the minimum. +³ Opt-in Omni captioning uses the [nemotron-3-nano-omni-30b-a3b-reasoning](https://docs.api.nvidia.com/nim/reference/nvidia-nemotron-3-nano-omni-30b-a3b-reasoning) NIM (`nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:2.0.4-variant`). BF16 requires at least 80 GB total GPU memory for this NIM deployment; refer to the [VLM NIM support matrix](https://docs.nvidia.com/nim/vision-language-models/latest/support-matrix.html#nemotron-3-nano-omni-30b-a3b-reasoning). L40S requires two GPUs. A100 40GB, A10G, and RTX PRO 4500 are below the minimum. ⁵ **Total GPUs** is combined VRAM co-residency for the four core models. The default Helm chart still requests `nvidia.com/gpu: 1` per NIMService. Without GPU sharing, plan for four allocatable GPU slots across eligible nodes. Refer to [Kubernetes Helm GPU scheduling](#kubernetes-helm-gpu-scheduling). diff --git a/docs/docs/extraction/releasenotes.md b/docs/docs/extraction/releasenotes.md index c504d1894..c545b49e9 100644 --- a/docs/docs/extraction/releasenotes.md +++ b/docs/docs/extraction/releasenotes.md @@ -12,17 +12,17 @@ Highlights for the 26.08 release include: ### Upgrade notes { #upgrade-notes } -- Text splitting for graph and library ingest moved into `.extract(split_config=...)` instead of standalone `.split()` on the graph ingest path (the service ingestor API may still expose `.split()` separately) -- Direct `Retriever(...)` construction uses `vdb_kwargs`, `embed_kwargs`, and `rerank` instead of flat `lancedb_uri`, `lancedb_table`, `embedder`, `embedding_endpoint`, `local_query_embed_backend`, and `reranker` arguments -- For Helm audio and video extraction, set `service.installFfmpeg: true` in `values.yaml` (or pass `--set service.installFfmpeg=true`) when images no longer bundle `ffmpeg` and `ffprobe` by default +- Text splitting for graph and library ingest moved into `.extract(split_config=...)` instead of standalone `.split()` on the graph ingest path (the service ingestor API may still expose `.split()` separately) +- Direct `Retriever(...)` construction uses `vdb_kwargs`, `embed_kwargs`, and `rerank` instead of flat `lancedb_uri`, `lancedb_table`, `embedder`, `embedding_endpoint`, `local_query_embed_backend`, and `reranker` arguments +- For Helm audio and video extraction, set `service.installFfmpeg: true` in `values.yaml` (or pass `--set service.installFfmpeg=true`) when images no longer bundle `ffmpeg` and `ffprobe` by default - Changing a Helm NIM image repository or tag on an existing release cannot patch `NIMCache` `spec.source.ngc.modelPuller`. Delete the `NIMCache` and its PVC, then upgrade. The affected NIM is unavailable while the operator re-caches weights. Refer to [Changing a NIM image repository or tag](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#changing-nim-image-repository-or-tag). -- `nemo_retriever` requires Python 3.12 +- `nemo_retriever` requires Python 3.12 ### Pipeline and ingestion { #pipeline-and-ingestion } - Legacy `nv-ingest` and compatibility pipeline CLI code paths removed; `retriever ingest` and the graph stage registry are the canonical ingestion paths -- Manifest-based ingest routing replaces input-type routing; `retriever ingest` is input-aware for PDF, image, audio, video, text, HTML, DOCX/PPTX, SVG, and related types -- `allow_no_gpu` option to skip GPU requirement during ingest for CPU-only experimentation +- Manifest-based ingest routing replaces input-type routing; `retriever ingest` is input-aware for PDF, image, audio, video, text, HTML, DOCX/PPTX, SVG, and related types +- `allow_no_gpu` option to skip GPU requirement during ingest for CPU-only experimentation ### CLI { #cli } @@ -31,51 +31,52 @@ Highlights for the 26.08 release include: ### Retriever Service and deployment { #retriever-service-and-deployment } -- Retriever Service v2 adds a scalable multi-pod architecture with gateway, process isolation, and VectorDB integration -- OpenTelemetry basic support for pipeline and service observability -- Expanded air-gapped deployment guidance in [deployment options](deployment-options.md) and the Helm chart README +- Retriever Service v2 adds a scalable multi-pod architecture with gateway, process isolation, and VectorDB integration +- OpenTelemetry basic support for pipeline and service observability +- Expanded air-gapped deployment guidance in [deployment options](deployment-options.md) and the Helm chart README ### Models, OCR, and captioning { #models-ocr-and-captioning } -- Nemotron OCR v2 is the default OCR engine for HuggingFace, with CLI language selectors and unified OCR actors. For Helm NIM deployments, Nemotron OCR v1 is the default. -- Nemotron Parse is available as an alternate PDF extraction method (v1.2 HTTP interface; optional Helm NIM; local inference via vLLM where configured) -- VLM image captioning via vLLM (including Omni caption model profiles) addresses the capability deferred in 26.03 -- vLLM-backed text and vision-language embedders, multimodal VL reranker, and torch 2.11 for local GPU installs +- Nemotron OCR v2 is the default OCR engine for HuggingFace and for Helm NIM deployments, with CLI language selectors and unified OCR actors. +- Nemotron Parse is available as an alternate PDF extraction method (self-hosted Helm NIM defaults to the v2.0 tagged text-prompt interface; local Hugging Face inference remains on v1.2 where configured) +- VLM image captioning via vLLM (including Omni caption model profiles) addresses the capability deferred in 26.03 +- vLLM-backed text and vision-language embedders, multimodal VL reranker, and torch 2.11 for local GPU installs ### Multimodal extraction { #multimodal-extraction } -- Video retrieval pipeline with frame extraction, OCR, audio-visual fusion, and text deduplication -- Long-audio Parakeet chunking with time-aligned segments; punctuation-based audio segmenting; ASR batch/streaming improvements +- Video retrieval pipeline with frame extraction, OCR, audio-visual fusion, and text deduplication +- Long-audio Parakeet chunking with time-aligned segments; punctuation-based audio segmenting; ASR batch/streaming improvements - Fixed an issue that could cause local Hugging Face batch audio extraction to hang in interactive terminals when FFmpeg inherited the parent process's standard input. ### Retrieval and RAG { #retrieval-and-rag } -- Live RAG SDK with `Retriever.retrieve()`, reference answer generation `Retriever.answer()`, and optional batch operator graphs via LiteLLM (`[llm]` extra) +- Live RAG SDK with `Retriever.retrieve()`, reference answer generation `Retriever.answer()`, and optional batch operator graphs via LiteLLM (`[llm]` extra) ### Vector database { #vector-database } -- Vector database operators integrated directly in the pipeline; custom metadata support; LanceDB hybrid search guidance updated -- LanceDB is documented as the first-party vector path for new deployments; Milvus/MinIO guidance removed from the primary extraction doc set +- Vector database operators integrated directly in the pipeline; custom metadata support; LanceDB hybrid search guidance updated +- LanceDB is documented as the first-party vector path for new deployments; Milvus/MinIO guidance removed from the primary extraction doc set ### Evaluation { #evaluation } -- BEIR-centric evaluation overhaul and `retriever skill-eval` benchmark CLI for the NeMo Retriever skill (experimental) +- BEIR-centric evaluation overhaul and `retriever skill-eval` benchmark CLI for the NeMo Retriever skill (experimental) -- Text-to-SQL agent graph and tabular tooling for structured data retrieval, including tabular data ingestion +- Text-to-SQL agent graph and tabular tooling for structured data retrieval, including tabular data ingestion ### Packaging and platform { #packaging-and-platform } -- Optional install extras (`[local]`, `[multimedia]`, `[llm]`, `[tabular]`, `[nemotron-parse]`, `[service]`, and others), including slim remote/NIM-only installs on Mac and Windows +- Optional install extras (`[local]`, `[multimedia]`, `[llm]`, `[tabular]`, `[nemotron-parse]`, `[service]`, and others), including slim remote/NIM-only installs on Mac and Windows ### Helm chart { #helm-chart } -- Helm chart refresh under `nemo_retriever/helm/` with VL embedder defaults and optional Nemotron Parse and Omni caption NIMs +- Helm chart refresh under `nemo_retriever/helm/` with VL embedder defaults and optional Nemotron Parse and Omni caption NIMs +- Chart defaults pin the 26.08-RC1 service image and the release-approved optional NIM tags (Omni `2.0.4-variant`, Nemotron Parse `v2.0:2.0.8-variant`, Answer LLM Super-49B `v1:1.10.1`) ### Documentation { #documentation } - Documentation aligned to a Helm-first supported path for NIM and service deployment -- Documentation consolidates extraction concepts, ingest workflow, embeddings, audio/video guides, prerequisites and support matrix, and UDF/custom stages in the [graph README](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph) +- Documentation consolidates extraction concepts, ingest workflow, embeddings, audio/video guides, prerequisites and support matrix, and UDF/custom stages in the [graph README](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph) ## Release Notes for Previous Versions { #previous-versions } diff --git a/docs/docs/extraction/troubleshoot.md b/docs/docs/extraction/troubleshoot.md index b5395b985..ef244a615 100644 --- a/docs/docs/extraction/troubleshoot.md +++ b/docs/docs/extraction/troubleshoot.md @@ -264,9 +264,9 @@ When you run PDF extraction with `method="nemotron_parse"`, a mismatched model a HTTP 400: Content cannot be a plain string. The model does not support text input. ``` -This can occur when you send a versioned self-hosted model (for example `nvidia/nemotron-parse-v1.2` or `nvidia/nemotron-parse-v2.0`) to the NVIDIA-hosted Build endpoint, which expects the image-only `nvidia/nemotron-parse` contract. It can also occur when the selected self-hosted Parse image and configured model use different versions. The library may replace the raw HTTP error with a targeted model/contract mismatch hint. +This can occur when you send a versioned self-hosted model (for example `nvidia/nemotron-parse-v2.0` or `nvidia/nemotron-parse-v1.2`) to the NVIDIA-hosted Build endpoint, which expects the image-only `nvidia/nemotron-parse` contract. It can also occur when the selected self-hosted Parse image and configured model use different versions. The library may replace the raw HTTP error with a targeted model/contract mismatch hint. -To use hosted Build, omit `nemotron_parse_model` so the library selects `nvidia/nemotron-parse` automatically, or set `nemotron_parse_model="nvidia/nemotron-parse"` explicitly. Send `nvidia/nemotron-parse-v1.2` or `nvidia/nemotron-parse-v2.0` only to its matching self-hosted chat endpoint. For direct external Parse v2.0 endpoints, set `nemotron_parse_model="nvidia/nemotron-parse-v2.0"` explicitly. For more information, refer to [Nemotron Parse: hosted Build and self-hosted NIM contracts](prerequisites-support-matrix.md#nemotron-parse-hosted-vs-self-hosted). +To use hosted Build, omit `nemotron_parse_model` so the library selects `nvidia/nemotron-parse` automatically, or set `nemotron_parse_model="nvidia/nemotron-parse"` explicitly. Send `nvidia/nemotron-parse-v2.0` or `nvidia/nemotron-parse-v1.2` only to its matching self-hosted chat endpoint. The library and Helm chart default self-hosted chat endpoints to `nvidia/nemotron-parse-v2.0`. For more information, refer to [Nemotron Parse: hosted Build and self-hosted NIM contracts](prerequisites-support-matrix.md#nemotron-parse-hosted-vs-self-hosted). ## Hosted Page Elements NIM image size limits { #hosted-page-elements-nim-image-size-limits } diff --git a/nemo_retriever/helm/Chart.yaml b/nemo_retriever/helm/Chart.yaml index 8f377aac8..1a54f6bf9 100644 --- a/nemo_retriever/helm/Chart.yaml +++ b/nemo_retriever/helm/Chart.yaml @@ -18,8 +18,8 @@ description: | work and status history. type: application -version: "26.5.0" -appVersion: "26.5.0" +version: "26.08-RC1" +appVersion: "26.08-RC1" kubeVersion: ">=1.25.0-0" home: https://github.com/NVIDIA/NeMo-Retriever sources: diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index 072fe42c3..3f63859e8 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -321,10 +321,10 @@ If `helm install` already succeeded and NIM pods stay `Pending` on ### 1. Service image { #1-service-image } -The chart defaults to the image published to NGC: +The chart defaults to the 26.08-RC1 image published for release qualification: ``` -nvcr.io/nvidia/nemo-microservices/nrl-service:26.5.0 +nvcr.io/nvstaging/nim/nrl-service:26.08-RC1 ``` Pulling from `nvcr.io` requires an NGC pull secret — either set @@ -463,8 +463,7 @@ helm install retriever ./nemo_retriever/helm \ --set ngcImagePullSecret.create=true \ --set ngcImagePullSecret.password=$NGC_API_KEY \ --set ngcApiSecret.create=true \ - --set ngcApiSecret.password=$NGC_API_KEY \ - --set service.image.tag=26.8.0 + --set ngcApiSecret.password=$NGC_API_KEY ``` > The VL reranker (`rerankqa`), Nemotron Parse, the Nemotron 3 Nano Omni 30B caption NIM, the generic answer-generation LLM (`answer_llm`, Super-49B defaults), and the Parakeet `audio` ASR NIM are **all off by default** — they only reconcile when you explicitly opt in. Opt-in flags: @@ -480,24 +479,24 @@ helm install retriever ./nemo_retriever/helm \ ### Nemotron Parse (optional) Enable `nimOperator.nemotron_parse` to use `method="nemotron_parse"`. The -default self-hosted image is `nvcr.io/nim/nvidia/nemotron-parse-v1.2:1.7.0-variant`. -The chart wires its matching `nvidia/nemotron-parse-v1.2` model into the +default self-hosted image is `nvcr.io/nim/nvidia/nemotron-parse-v2.0:2.0.8-variant`. +The chart wires its matching `nvidia/nemotron-parse-v2.0` model into the retriever service. -To deploy the optional Parse v2.0 NIM instead, override both image fields: +To deploy the optional Parse v1.2 NIM instead, override both image fields: ```bash helm upgrade --install retriever ./nemo_retriever/helm \ --set nimOperator.nemotron_parse.enabled=true \ - --set nimOperator.nemotron_parse.image.repository=nvcr.io/nim/nvidia/nemotron-parse-v2.0 \ - --set nimOperator.nemotron_parse.image.tag=2.0.8-variant + --set nimOperator.nemotron_parse.image.repository=nvcr.io/nim/nvidia/nemotron-parse-v1.2 \ + --set nimOperator.nemotron_parse.image.tag=1.7.0-variant ``` -With this override, the chart wires `nvidia/nemotron-parse-v2.0` into the +With this override, the chart wires `nvidia/nemotron-parse-v1.2` into the retriever service. Do not set `serviceConfig.nimEndpoints.nemotronParseModel` unless you need a custom model value; an explicit service value takes precedence -over the image-selected model. For direct external Parse v2.0 endpoints, set -`nemotron_parse_model="nvidia/nemotron-parse-v2.0"` in the SDK extraction +over the image-selected model. For direct external Parse endpoints, set +`nemotron_parse_model` to the matching model ID in the SDK extraction configuration. The chart auto-wires the operator-managed in-cluster URLs of the three @@ -587,8 +586,8 @@ short list of knobs you'll touch first. | Path | Default | Notes | |-------------------------------|------------------------------------|-------| -| `service.image.repository` | `nvcr.io/nvidia/nemo-microservices/nrl-service` | NGC image; override to pin a different build or use a local registry. | -| `service.image.tag` | `26.5.0` | | +| `service.image.repository` | `nvcr.io/nvstaging/nim/nrl-service` | NGC staging image for 26.08-RC1; override to pin a different build or use a local registry. | +| `service.image.tag` | `26.08-RC1` | | | `service.replicas` | `1` | Keep at 1 because standalone job and scheduler state are process-local. | | `service.installFfmpeg` | `false` | Install `ffmpeg`/`ffprobe` at container startup by setting `INSTALL_FFMPEG=true`. Requires network egress, writable root filesystem, and sudo/setuid allowed. Not for air-gapped clusters — use a custom image instead. | | `service.resources.requests` | `16 / 16Gi` | Tune in tandem with `serviceConfig.pipeline.*Workers`. | @@ -760,7 +759,7 @@ NIMCache/NIMService by default and writes this block into ```yaml llm: enabled: true - model: "openai/nvidia/llama-3.3-nemotron-super-49b-v1.5" + model: "openai/nvidia/llama-3.3-nemotron-super-49b-v1" api_base: "http://answer-llm:8000/v1" rag_system_prompt_prefix: null reasoning_enabled: true @@ -851,7 +850,7 @@ gated on three conditions ALL holding: | `nimOperator.nemotron_parse.enabled` | `false` | Structured-parse NIM (optional). Set `true` when using `method="nemotron_parse"`. Default `false` so chart installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md). The default image is Parse v1.2; you can select Parse v2.0 with its documented image override. Image tags follow the [image tag conventions](#image-tag-conventions). | | `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so chart installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | | `nimOperator.answer_llm.enabled` | `false` | Generic answer-generation LLM NIM (optional; Super-49B defaults). Set `true` to enable `/v1/answer` — refer to [Answer generation (operator-managed LLM)](#answer-generation-llm). Default `false` so installs do not silently claim answer-generation GPUs. | -| `nimOperator.answer_llm.model` | `openai/nvidia/llama-3.3-nemotron-super-49b-v1.5` | LiteLLM/OpenAI model id inherited by `serviceConfig.llm.model` when the operator-managed answer LLM is enabled and no explicit service model is set. | +| `nimOperator.answer_llm.model` | `openai/nvidia/llama-3.3-nemotron-super-49b-v1` | LiteLLM/OpenAI model id inherited by `serviceConfig.llm.model` when the operator-managed answer LLM is enabled and no explicit service model is set. | | `nimOperator.answer_llm.ragSystemPromptPrefix` | `""` | Optional prompt prefix inherited by `serviceConfig.llm.ragSystemPromptPrefix` only when explicitly set. Leave empty to keep the operator-managed LLM model-neutral and use `serviceConfig.llm.reasoningEnabled` for request-level reasoning control. | | `nimOperator.audio.enabled` | `false` | Parakeet ASR NIM (optional). Set `true` for audio/video transcription; pair with `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` so the retriever-service can reach it. | | `nimOperator..image.repository` | `nvcr.io/nim/nvidia/...` | Per-NIM image. | @@ -942,7 +941,7 @@ Every NIM in this chart pins an exact NGC image tag in `values.yaml` | Family | Example | Meaning | | ------ | ------- | ------- | | Plain semver | `nemotron-object-detection:2.0.1` | A standard NIM release, identical bytes on every pull. Used by the four core NIMs and the reranker / ASR NIMs. | -| `-variant` | `nemotron-parse-v1.2:1.7.0-variant`, `nemotron-3-nano-omni-30b-a3b-reasoning:1.7.0-variant` | The Nemotron Parse and Nemotron 3 Nano Omni 30B builds that ship per-GPU TensorRT engine variants the NIM Operator selects from at reconciliation time (refer to the Omni and Parse rows in the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table). The `-variant` suffix is the NGC tag that ships alongside this chart and matches footnote ³ of the support matrix. | +| `-variant` | `nemotron-parse-v2.0:2.0.8-variant`, `nemotron-3-nano-omni-30b-a3b-reasoning:2.0.4-variant` | The Nemotron Parse and Nemotron 3 Nano Omni 30B builds that ship per-GPU TensorRT engine variants the NIM Operator selects from at reconciliation time (refer to the Omni and Parse rows in the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table). The `-variant` suffix is the NGC tag that ships alongside this chart and matches footnote ³ of the support matrix. | For air-gapped mirror pipelines: mirror the *exact* tag — both the plain semver and the `-variant` form — and do not substitute `:latest`. @@ -1766,9 +1765,9 @@ your release tag). Defaults below match | OCR | `ocr` | `nvcr.io/nim/nvidia/nemotron-ocr-v2:2.0.1` | | VL embed | `vlm_embed` | `nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:2.3.0` | | VL reranker (optional) | `rerankqa` | `nvcr.io/nim/nvidia/llama-nemotron-rerank-vl-1b-v2:2.3.0` | -| Nemotron Parse (optional) | `nemotron_parse` | `nvcr.io/nim/nvidia/nemotron-parse-v1.2:1.7.0-variant` | -| Omni caption (optional) | `nemotron_3_nano_omni_30b_a3b_reasoning` | `nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:1.7.0-variant` | -| Answer LLM (optional, Super-49B default) | `answer_llm` | `nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1.5:2.0.5` | +| Nemotron Parse (optional) | `nemotron_parse` | `nvcr.io/nim/nvidia/nemotron-parse-v2.0:2.0.8-variant` | +| Omni caption (optional) | `nemotron_3_nano_omni_30b_a3b_reasoning` | `nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:2.0.4-variant` | +| Answer LLM (optional, Super-49B default) | `answer_llm` | `nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1:1.10.1` | | Parakeet ASR (optional) | `audio` | `nvcr.io/nim/nvidia/parakeet-1-1b-ctc-en-us:1.5.0` | GPU SKU support for `audio` is in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements). diff --git a/nemo_retriever/helm/openshift.md b/nemo_retriever/helm/openshift.md index 88977542b..bd24baee1 100644 --- a/nemo_retriever/helm/openshift.md +++ b/nemo_retriever/helm/openshift.md @@ -1,261 +1,261 @@ -# OpenShift deployment - -Use this guide when you install the [NeMo Retriever Helm chart](./README.md) on **OpenShift 4.x** with the default **restricted-v2** Security Context Constraint (SCC) and **Pod Security Admission (PSA) `restricted`** profile. - -For general Kubernetes and Helm deployment choices, refer to [Deployment options](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/). For chart values and NIM wiring, refer to the [Helm chart README](./README.md). - -## Overview { #openshift-deployment } - -The chart defaults target generic Kubernetes clusters that allow fixed numeric UIDs (`runAsUser` / `runAsGroup` / `fsGroup` **1000**). OpenShift namespaces under **restricted-v2** assign a per-namespace UID/GID range instead. A stock `helm install` without overrides therefore fails SCC validation, emits PSA warnings, or crashes on log paths the random UID cannot write. - -We do **not** change chart defaults for OpenShift-only behavior (that would affect other platforms). Use the overrides below, or save the YAML block into a local values file and pass `-f ` on every `helm install` / `helm upgrade`. - -### Cluster posture (typical hardened namespaces) - -| Control | Typical default on a new OpenShift project | -| --- | --- | -| SCC | **restricted-v2** (first match in priority order) | -| PSA | `pod-security.kubernetes.io/warn=restricted` (and often `audit=restricted`; `enforce` may be unset on dev clusters) | -| UID assignment | SCC injects `runAsUser` / `fsGroup` from the namespace range (for example `1000750000–1000759999`) | - -On clusters with **PSA `enforce=restricted`**, missing container `securityContext` fields become hard rejections, not warnings. - -### Override reference (maps to chart limitations) - -| Symptom on stock install | Cause | Helm override | -| --- | --- | --- | -| `FailedCreate`: UID/GID **1000** not in namespace range | Hardcoded `service.podSecurityContext` UID/GID/fsGroup | Omit `runAsUser`, `runAsGroup`, and `fsGroup`; keep only `runAsNonRoot: true` | -| PSA warning: `allowPrivilegeEscalation`, capabilities, `seccompProfile` | Empty `service.securityContext` | Set restricted baseline on `service.securityContext` (refer to the sample below) | -| `PermissionError` on `/var/lib/nemo-retriever/retriever-service.log` when `persistence.enabled=false` | Default log path is image-owned; random UID cannot write without a PVC | Point `serviceConfig.logging.file` at `/tmp/...` (chart mounts `emptyDir` at `/tmp`) | -| `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Set `serviceConfig.vectordb.enabled=false` for a minimal service-only install, or patch the vectordb Deployment after install (below) | -| PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | -| Audio/video fails or pod never gets `ffmpeg` | `service.installFfmpeg=true` runs sudo at startup; **restricted-v2** blocks privilege escalation (`no-new-privileges`) | Prebuild a service image with `ffmpeg`/`ffprobe` baked in (refer to [Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift)); leave `service.installFfmpeg=false` | -| `ImagePullBackOff` for a service image in the **internal OpenShift registry** | Chart-rendered `imagePullSecrets` may omit the namespace SA `dockercfg` secret required for internal-registry pulls | List every required pull secret under `imagePullSecrets` (refer to [Internal registry pull secrets](#internal-registry-pull-secrets)) | -| Optional NIM `CrashLoopBackOff` with missing `.so` in logs | GPU/CUDA libraries not on `LD_LIBRARY_PATH` for some NIM Operator stacks on OCP | Append paths through `nimOperator..env` (refer to [Optional NIM runtime environment](#optional-nim-runtime-environment)) | - -### Recommended value overrides - -Save the block below as a local values file (for example `openshift-restricted.yaml`) and pass `-f openshift-restricted.yaml` on every `helm install` / `helm upgrade`. Use this profile for **restricted-v2** / PSA **restricted** namespaces with the NIM Operator, a prebuilt `ffmpeg` service image when you need [audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md), and optional NIM env overrides from later sections. - -```yaml -# OpenShift overrides for nemo-retriever Helm chart (restricted-v2 / PSA restricted). -# helm install retriever ./nemo_retriever/helm -f openshift-restricted.yaml ... - -service: - podSecurityContext: - runAsNonRoot: true - # Do NOT set runAsUser, runAsGroup, or fsGroup — OpenShift SCC assigns them. - securityContext: - runAsNonRoot: true - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - seccompProfile: - type: RuntimeDefault - # Leave installFfmpeg false on restricted-v2; use a custom image for audio/video. - installFfmpeg: false - -serviceConfig: - logging: - # Writable without persistence PVC (chart always mounts emptyDir at /tmp). - file: /tmp/retriever-service.log - vectordb: - # Set false for a minimal service-only install; refer to the vectordb patch below if you enable it. - enabled: false - -topology: - otel: - enabled: false -``` - -When **`persistence.enabled=true`**, you can keep the default log path under `persistence.mountPath` (`/var/lib/nemo-retriever`) because the PVC is mounted and SCC-assigned `fsGroup` applies. When persistence is off, always relocate logs to `/tmp` (or another path backed by `service.extraVolumes`). - -### Audio and video (ffmpeg) on restricted OpenShift { #audio-and-video-ffmpeg-on-restricted-openshift } - -The Helm chart supports `service.installFfmpeg=true`, which installs `ffmpeg`/`ffprobe` at container startup through passwordless `sudo`. On OpenShift **restricted-v2** SCC, that path is blocked: PSA restricted sets `allowPrivilegeEscalation: false` and the SCC enforces **no-new-privileges**, so the entrypoint cannot elevate to install packages. - -For audio and video extraction on OpenShift, **do not** set `service.installFfmpeg=true`. Instead, extend the service image on a connected build host and point the chart at that tag (same pattern as [air-gapped custom service images](./README.md#1-service-image)): - -```dockerfile -FROM nvcr.io/nvidia/nemo-microservices/nrl-service: -USER root -RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \ - && rm -rf /var/lib/apt/lists/* -USER nemo -``` - -Push the result to NGC, your private registry, or the [OpenShift internal registry](#internal-registry-pull-secrets), then set `service.image.repository` / `service.image.tag` accordingly. - -### Internal registry pull secrets { #internal-registry-pull-secrets } - -When you rebuild the service image into the OpenShift internal registry (`image-registry.openshift-image-registry.svc:5000/...`), pods normally pull through the namespace ServiceAccount's automatic `kubernetes.io/dockercfg` secret. The chart renders an explicit `imagePullSecrets` list on every Pod, so you must list **every** secret each Pod needs—including the SA `dockercfg` secret when `service.image` points at the internal registry. - -If the rendered list contains only `ngc-secret` (for NGC NIM images) while `service.image` uses the internal registry, the service Pod can fail with `ImagePullBackOff`. - -**Internal-registry service image only** — clear the chart-managed NGC pull secret name so the helper does not inject `ngc-secret` alone: - -```yaml -ngcImagePullSecret: - create: false - name: "" # Explicitly empty — clears the default "ngc-secret" - -imagePullSecrets: - - name: default-dockercfg-xxxxx # replace with your SA secret (section below) -``` - -**Both NGC NIMs and an internal-registry service image** — list every secret the pods need: - -```yaml -ngcImagePullSecret: - create: false - name: ngc-secret - -imagePullSecrets: - - name: ngc-secret - - name: default-dockercfg-xxxxx -``` - -Look up the ServiceAccount pull secret name: - -```bash -oc get sa default -n nemo-retriever -o jsonpath='{.imagePullSecrets[*].name}{"\n"}' -``` - -Pre-create `ngc-secret` / `ngc-api` with `oc create secret` when `ngcImagePullSecret.create=false` and `ngcApiSecret.create=false`, as in the install examples below. - -### Optional NIM runtime environment { #optional-nim-runtime-environment } - -On some OpenShift clusters with the NIM Operator, **optional** NIM pods (for example Parakeet ASR or Omni caption) need an explicit `LD_LIBRARY_PATH` so CUDA/driver libraries from the GPU Operator stack are visible at runtime. Symptoms are `CrashLoopBackOff` and log lines referencing a missing `.so` (for example `libcudart` or `libcudnn`). - -Helm **replaces** the whole `env` list when you override `nimOperator..env` in a values file. Copy the chart defaults from `values.yaml` for that NIM, then append `LD_LIBRARY_PATH`: - -```yaml -nimOperator: - audio: - env: - # Retain chart defaults, then append LD_LIBRARY_PATH. - - name: NIM_TAGS_SELECTOR - value: "name=parakeet-1-1b-ctc-en-us,mode=ofl,vad=default,diarizer=disabled" - - name: NIM_TRITON_LOG_VERBOSE - value: "1" - - name: LD_LIBRARY_PATH - value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" - nemotron_3_nano_omni_30b_a3b_reasoning: - env: - # Retain chart defaults, then append LD_LIBRARY_PATH. - - name: NIM_HTTP_API_PORT - value: "8000" - - name: NIM_TRITON_LOG_VERBOSE - value: "1" - - name: LD_LIBRARY_PATH - value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" -``` - -Paths vary with GPU Operator version and node image. Inspect a healthy GPU workload on the same cluster (`oc exec` into a CUDA sample pod) or the failing NIM pod's filesystem before pinning production values. - -### Call the Omni NIM directly { #omni-caption-manual-smoke-testing } - -The retriever service caption profile already sends `chat_template_kwargs.enable_thinking=false` to the Omni NIM during ingest. To confirm the in-cluster Omni NIM returns caption text in `message.content`, call `/v1/chat/completions` directly (for example with `curl` against the `NIMService`) and include the same flag: - -```json -{ - "model": "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning", - "messages": [{"role": "user", "content": "..."}], - "chat_template_kwargs": {"enable_thinking": false} -} -``` - -For pipeline scope (PDF chart regions are not captioned), refer to [Image captioning (support matrix)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#image-captioning) and [Image captioning (pipeline scope)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/multimodal-extraction.md#image-captioning) in the extraction docs. - -### Example install (service only, no in-cluster NIMs) - -Use this flow when you want only the retriever service pod: disable the in-cluster NIM Operator stack (`nims.enabled=false`), skip the persistence PVC, and skip the results PVC. Pre-create NGC pull/API secrets, then install with the restricted OpenShift values file: - -```bash -oc new-project nemo-retriever - -oc create secret docker-registry ngc-secret -n nemo-retriever \ - --docker-server=nvcr.io --docker-username='$oauthtoken' \ - --docker-password="$NGC_API_KEY" - -oc create secret generic ngc-api -n nemo-retriever \ - --from-literal=NGC_API_KEY="$NGC_API_KEY" \ - --from-literal=NGC_CLI_API_KEY="$NGC_API_KEY" - -helm install retriever ./nemo_retriever/helm -n nemo-retriever \ - -f openshift-restricted.yaml \ - --set ngcImagePullSecret.create=false \ - --set ngcApiSecret.create=false \ - --set nims.enabled=false \ - --set persistence.enabled=false \ - --set retrieverResults.enabled=false \ - --set service.image.repository=nvcr.io/nvidia/nemo-microservices/nrl-service \ - --set service.image.tag=26.8.0 -``` - -Verify pods: - -```bash -oc get pods -n nemo-retriever -oc describe pod -l app.kubernetes.io/name=nemo-retriever -n nemo-retriever -``` - -You should see SCC-assigned numeric `runAsUser` on containers that declare a `securityContext` block, and no PSA warnings after overrides are applied. - -### Example install with NIM Operator (in-cluster NIMs) - -After you install the NIM Operator and GPU Operator, reuse `openshift-restricted.yaml` and the NGC secrets from the service-only example. Point `service.image` at a **ffmpeg-enabled** build when you use [audio and video](#audio-and-video-ffmpeg-on-restricted-openshift). Add [optional NIM `LD_LIBRARY_PATH`](#optional-nim-runtime-environment) overrides if ASR or Omni pods crash on missing libraries. - -```bash -helm install retriever ./nemo_retriever/helm -n nemo-retriever \ - -f openshift-restricted.yaml \ - --set ngcImagePullSecret.create=false \ - --set ngcApiSecret.create=false \ - --set service.image.repository=nvcr.io/nvidia/nemo-microservices/nrl-service \ - --set service.image.tag=26.8.0 -``` - -After install, confirm workloads reach Ready before you run ingest: - -```bash -oc get pods -n nemo-retriever -oc get nimservice -n nemo-retriever -``` - -When you enable optional NIMs, wait for Parakeet ASR and Omni caption pods to become Ready before you run [audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md) or caption extraction. If ASR or Omni pods crash with missing `.so` errors, apply the [LD_LIBRARY_PATH overrides](#optional-nim-runtime-environment) above and upgrade the release. - -### Enabling the vectordb Deployment on OpenShift - -`serviceConfig.vectordb.enabled=true` renders a **vectordb** container from the same image (`USER nemo`, non-numeric). The chart does not yet expose a `securityContext` value for that container. After `helm install`, patch the Deployment so OpenShift can inject a numeric UID into the container spec: - -```bash -RELEASE=retriever -NS=nemo-retriever -VDB_DEPLOY="${RELEASE}-nemo-retriever-vectordb" - -oc patch deployment "$VDB_DEPLOY" -n "$NS" --type=json -p='[ - {"op": "add", "path": "/spec/template/spec/containers/0/securityContext", "value": { - "allowPrivilegeEscalation": false, - "capabilities": {"drop": ["ALL"]}, - "runAsNonRoot": true, - "seccompProfile": {"type": "RuntimeDefault"} - }} -]' -``` - -Re-apply the patch after `helm upgrade` if the Deployment is recreated. A future chart release may add first-class `topology.vectordb.securityContext` values. - -### Enabling the OpenTelemetry collector on OpenShift - -The chart's otel-collector Deployment likewise lacks `securityContext` fields. Prefer `topology.otel.enabled=false` (as in the sample values) unless you operate your own collector or patch `*-otel` the same way as vectordb. - -### What we intentionally do not require on OpenShift - -Do **not** bind the namespace to **anyuid** SCC or set PSA `enforce=privileged` unless your security team explicitly approves it. The overrides above are intended to keep **restricted-v2** / PSA **restricted** posture. - -## Related topics - -- [Pre-Requisites & Support Matrix](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md) -- [Deployment options](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/) -- [Helm chart README](./README.md) -- [Audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md) +# OpenShift deployment + +Use this guide when you install the [NeMo Retriever Helm chart](./README.md) on **OpenShift 4.x** with the default **restricted-v2** Security Context Constraint (SCC) and **Pod Security Admission (PSA) `restricted`** profile. + +For general Kubernetes and Helm deployment choices, refer to [Deployment options](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/). For chart values and NIM wiring, refer to the [Helm chart README](./README.md). + +## Overview { #openshift-deployment } + +The chart defaults target generic Kubernetes clusters that allow fixed numeric UIDs (`runAsUser` / `runAsGroup` / `fsGroup` **1000**). OpenShift namespaces under **restricted-v2** assign a per-namespace UID/GID range instead. A stock `helm install` without overrides therefore fails SCC validation, emits PSA warnings, or crashes on log paths the random UID cannot write. + +We do **not** change chart defaults for OpenShift-only behavior (that would affect other platforms). Use the overrides below, or save the YAML block into a local values file and pass `-f ` on every `helm install` / `helm upgrade`. + +### Cluster posture (typical hardened namespaces) + +| Control | Typical default on a new OpenShift project | +| --- | --- | +| SCC | **restricted-v2** (first match in priority order) | +| PSA | `pod-security.kubernetes.io/warn=restricted` (and often `audit=restricted`; `enforce` may be unset on dev clusters) | +| UID assignment | SCC injects `runAsUser` / `fsGroup` from the namespace range (for example `1000750000–1000759999`) | + +On clusters with **PSA `enforce=restricted`**, missing container `securityContext` fields become hard rejections, not warnings. + +### Override reference (maps to chart limitations) + +| Symptom on stock install | Cause | Helm override | +| --- | --- | --- | +| `FailedCreate`: UID/GID **1000** not in namespace range | Hardcoded `service.podSecurityContext` UID/GID/fsGroup | Omit `runAsUser`, `runAsGroup`, and `fsGroup`; keep only `runAsNonRoot: true` | +| PSA warning: `allowPrivilegeEscalation`, capabilities, `seccompProfile` | Empty `service.securityContext` | Set restricted baseline on `service.securityContext` (refer to the sample below) | +| `PermissionError` on `/var/lib/nemo-retriever/retriever-service.log` when `persistence.enabled=false` | Default log path is image-owned; random UID cannot write without a PVC | Point `serviceConfig.logging.file` at `/tmp/...` (chart mounts `emptyDir` at `/tmp`) | +| `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Set `serviceConfig.vectordb.enabled=false` for a minimal service-only install, or patch the vectordb Deployment after install (below) | +| PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | +| Audio/video fails or pod never gets `ffmpeg` | `service.installFfmpeg=true` runs sudo at startup; **restricted-v2** blocks privilege escalation (`no-new-privileges`) | Prebuild a service image with `ffmpeg`/`ffprobe` baked in (refer to [Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift)); leave `service.installFfmpeg=false` | +| `ImagePullBackOff` for a service image in the **internal OpenShift registry** | Chart-rendered `imagePullSecrets` may omit the namespace SA `dockercfg` secret required for internal-registry pulls | List every required pull secret under `imagePullSecrets` (refer to [Internal registry pull secrets](#internal-registry-pull-secrets)) | +| Optional NIM `CrashLoopBackOff` with missing `.so` in logs | GPU/CUDA libraries not on `LD_LIBRARY_PATH` for some NIM Operator stacks on OCP | Append paths through `nimOperator..env` (refer to [Optional NIM runtime environment](#optional-nim-runtime-environment)) | + +### Recommended value overrides + +Save the block below as a local values file (for example `openshift-restricted.yaml`) and pass `-f openshift-restricted.yaml` on every `helm install` / `helm upgrade`. Use this profile for **restricted-v2** / PSA **restricted** namespaces with the NIM Operator, a prebuilt `ffmpeg` service image when you need [audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md), and optional NIM env overrides from later sections. + +```yaml +# OpenShift overrides for nemo-retriever Helm chart (restricted-v2 / PSA restricted). +# helm install retriever ./nemo_retriever/helm -f openshift-restricted.yaml ... + +service: + podSecurityContext: + runAsNonRoot: true + # Do NOT set runAsUser, runAsGroup, or fsGroup — OpenShift SCC assigns them. + securityContext: + runAsNonRoot: true + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + seccompProfile: + type: RuntimeDefault + # Leave installFfmpeg false on restricted-v2; use a custom image for audio/video. + installFfmpeg: false + +serviceConfig: + logging: + # Writable without persistence PVC (chart always mounts emptyDir at /tmp). + file: /tmp/retriever-service.log + vectordb: + # Set false for a minimal service-only install; refer to the vectordb patch below if you enable it. + enabled: false + +topology: + otel: + enabled: false +``` + +When **`persistence.enabled=true`**, you can keep the default log path under `persistence.mountPath` (`/var/lib/nemo-retriever`) because the PVC is mounted and SCC-assigned `fsGroup` applies. When persistence is off, always relocate logs to `/tmp` (or another path backed by `service.extraVolumes`). + +### Audio and video (ffmpeg) on restricted OpenShift { #audio-and-video-ffmpeg-on-restricted-openshift } + +The Helm chart supports `service.installFfmpeg=true`, which installs `ffmpeg`/`ffprobe` at container startup through passwordless `sudo`. On OpenShift **restricted-v2** SCC, that path is blocked: PSA restricted sets `allowPrivilegeEscalation: false` and the SCC enforces **no-new-privileges**, so the entrypoint cannot elevate to install packages. + +For audio and video extraction on OpenShift, **do not** set `service.installFfmpeg=true`. Instead, extend the service image on a connected build host and point the chart at that tag (same pattern as [air-gapped custom service images](./README.md#1-service-image)): + +```dockerfile +FROM nvcr.io/nvstaging/nim/nrl-service: +USER root +RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \ + && rm -rf /var/lib/apt/lists/* +USER nemo +``` + +Push the result to NGC, your private registry, or the [OpenShift internal registry](#internal-registry-pull-secrets), then set `service.image.repository` / `service.image.tag` accordingly. + +### Internal registry pull secrets { #internal-registry-pull-secrets } + +When you rebuild the service image into the OpenShift internal registry (`image-registry.openshift-image-registry.svc:5000/...`), pods normally pull through the namespace ServiceAccount's automatic `kubernetes.io/dockercfg` secret. The chart renders an explicit `imagePullSecrets` list on every Pod, so you must list **every** secret each Pod needs—including the SA `dockercfg` secret when `service.image` points at the internal registry. + +If the rendered list contains only `ngc-secret` (for NGC NIM images) while `service.image` uses the internal registry, the service Pod can fail with `ImagePullBackOff`. + +**Internal-registry service image only** — clear the chart-managed NGC pull secret name so the helper does not inject `ngc-secret` alone: + +```yaml +ngcImagePullSecret: + create: false + name: "" # Explicitly empty — clears the default "ngc-secret" + +imagePullSecrets: + - name: default-dockercfg-xxxxx # replace with your SA secret (section below) +``` + +**Both NGC NIMs and an internal-registry service image** — list every secret the pods need: + +```yaml +ngcImagePullSecret: + create: false + name: ngc-secret + +imagePullSecrets: + - name: ngc-secret + - name: default-dockercfg-xxxxx +``` + +Look up the ServiceAccount pull secret name: + +```bash +oc get sa default -n nemo-retriever -o jsonpath='{.imagePullSecrets[*].name}{"\n"}' +``` + +Pre-create `ngc-secret` / `ngc-api` with `oc create secret` when `ngcImagePullSecret.create=false` and `ngcApiSecret.create=false`, as in the install examples below. + +### Optional NIM runtime environment { #optional-nim-runtime-environment } + +On some OpenShift clusters with the NIM Operator, **optional** NIM pods (for example Parakeet ASR or Omni caption) need an explicit `LD_LIBRARY_PATH` so CUDA/driver libraries from the GPU Operator stack are visible at runtime. Symptoms are `CrashLoopBackOff` and log lines referencing a missing `.so` (for example `libcudart` or `libcudnn`). + +Helm **replaces** the whole `env` list when you override `nimOperator..env` in a values file. Copy the chart defaults from `values.yaml` for that NIM, then append `LD_LIBRARY_PATH`: + +```yaml +nimOperator: + audio: + env: + # Retain chart defaults, then append LD_LIBRARY_PATH. + - name: NIM_TAGS_SELECTOR + value: "name=parakeet-1-1b-ctc-en-us,mode=ofl,vad=default,diarizer=disabled" + - name: NIM_TRITON_LOG_VERBOSE + value: "1" + - name: LD_LIBRARY_PATH + value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" + nemotron_3_nano_omni_30b_a3b_reasoning: + env: + # Retain chart defaults, then append LD_LIBRARY_PATH. + - name: NIM_HTTP_API_PORT + value: "8000" + - name: NIM_TRITON_LOG_VERBOSE + value: "1" + - name: LD_LIBRARY_PATH + value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" +``` + +Paths vary with GPU Operator version and node image. Inspect a healthy GPU workload on the same cluster (`oc exec` into a CUDA sample pod) or the failing NIM pod's filesystem before pinning production values. + +### Call the Omni NIM directly { #omni-caption-manual-smoke-testing } + +The retriever service caption profile already sends `chat_template_kwargs.enable_thinking=false` to the Omni NIM during ingest. To confirm the in-cluster Omni NIM returns caption text in `message.content`, call `/v1/chat/completions` directly (for example with `curl` against the `NIMService`) and include the same flag: + +```json +{ + "model": "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning", + "messages": [{"role": "user", "content": "..."}], + "chat_template_kwargs": {"enable_thinking": false} +} +``` + +For pipeline scope (PDF chart regions are not captioned), refer to [Image captioning (support matrix)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#image-captioning) and [Image captioning (pipeline scope)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/multimodal-extraction.md#image-captioning) in the extraction docs. + +### Example install (service only, no in-cluster NIMs) + +Use this flow when you want only the retriever service pod: disable the in-cluster NIM Operator stack (`nims.enabled=false`), skip the persistence PVC, and skip the results PVC. Pre-create NGC pull/API secrets, then install with the restricted OpenShift values file: + +```bash +oc new-project nemo-retriever + +oc create secret docker-registry ngc-secret -n nemo-retriever \ + --docker-server=nvcr.io --docker-username='$oauthtoken' \ + --docker-password="$NGC_API_KEY" + +oc create secret generic ngc-api -n nemo-retriever \ + --from-literal=NGC_API_KEY="$NGC_API_KEY" \ + --from-literal=NGC_CLI_API_KEY="$NGC_API_KEY" + +helm install retriever ./nemo_retriever/helm -n nemo-retriever \ + -f openshift-restricted.yaml \ + --set ngcImagePullSecret.create=false \ + --set ngcApiSecret.create=false \ + --set nims.enabled=false \ + --set persistence.enabled=false \ + --set retrieverResults.enabled=false \ + --set service.image.repository=nvcr.io/nvstaging/nim/nrl-service \ + --set service.image.tag=26.08-RC1 +``` + +Verify pods: + +```bash +oc get pods -n nemo-retriever +oc describe pod -l app.kubernetes.io/name=nemo-retriever -n nemo-retriever +``` + +You should see SCC-assigned numeric `runAsUser` on containers that declare a `securityContext` block, and no PSA warnings after overrides are applied. + +### Example install with NIM Operator (in-cluster NIMs) + +After you install the NIM Operator and GPU Operator, reuse `openshift-restricted.yaml` and the NGC secrets from the service-only example. Point `service.image` at a **ffmpeg-enabled** build when you use [audio and video](#audio-and-video-ffmpeg-on-restricted-openshift). Add [optional NIM `LD_LIBRARY_PATH`](#optional-nim-runtime-environment) overrides if ASR or Omni pods crash on missing libraries. + +```bash +helm install retriever ./nemo_retriever/helm -n nemo-retriever \ + -f openshift-restricted.yaml \ + --set ngcImagePullSecret.create=false \ + --set ngcApiSecret.create=false \ + --set service.image.repository=nvcr.io/nvstaging/nim/nrl-service \ + --set service.image.tag=26.08-RC1 +``` + +After install, confirm workloads reach Ready before you run ingest: + +```bash +oc get pods -n nemo-retriever +oc get nimservice -n nemo-retriever +``` + +When you enable optional NIMs, wait for Parakeet ASR and Omni caption pods to become Ready before you run [audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md) or caption extraction. If ASR or Omni pods crash with missing `.so` errors, apply the [LD_LIBRARY_PATH overrides](#optional-nim-runtime-environment) above and upgrade the release. + +### Enabling the vectordb Deployment on OpenShift + +`serviceConfig.vectordb.enabled=true` renders a **vectordb** container from the same image (`USER nemo`, non-numeric). The chart does not yet expose a `securityContext` value for that container. After `helm install`, patch the Deployment so OpenShift can inject a numeric UID into the container spec: + +```bash +RELEASE=retriever +NS=nemo-retriever +VDB_DEPLOY="${RELEASE}-nemo-retriever-vectordb" + +oc patch deployment "$VDB_DEPLOY" -n "$NS" --type=json -p='[ + {"op": "add", "path": "/spec/template/spec/containers/0/securityContext", "value": { + "allowPrivilegeEscalation": false, + "capabilities": {"drop": ["ALL"]}, + "runAsNonRoot": true, + "seccompProfile": {"type": "RuntimeDefault"} + }} +]' +``` + +Re-apply the patch after `helm upgrade` if the Deployment is recreated. A future chart release may add first-class `topology.vectordb.securityContext` values. + +### Enabling the OpenTelemetry collector on OpenShift + +The chart's otel-collector Deployment likewise lacks `securityContext` fields. Prefer `topology.otel.enabled=false` (as in the sample values) unless you operate your own collector or patch `*-otel` the same way as vectordb. + +### What we intentionally do not require on OpenShift + +Do **not** bind the namespace to **anyuid** SCC or set PSA `enforce=privileged` unless your security team explicitly approves it. The overrides above are intended to keep **restricted-v2** / PSA **restricted** posture. + +## Related topics + +- [Pre-Requisites & Support Matrix](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md) +- [Deployment options](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/) +- [Helm chart README](./README.md) +- [Audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md) diff --git a/nemo_retriever/helm/values.yaml b/nemo_retriever/helm/values.yaml index aa62bce3f..5b587b868 100644 --- a/nemo_retriever/helm/values.yaml +++ b/nemo_retriever/helm/values.yaml @@ -67,12 +67,13 @@ imagePullSecrets: [] # ============================================================================= service: image: - # Default points at the GA image published to NGC. Override - # `repository` / `tag` to pin a different build, e.g. one produced by: + # Default pins the 26.08-RC1 service image from the release Test Plan. + # Override `repository` / `tag` to pin a different build, e.g. one + # produced by: # docker build -f Dockerfile --target service \ # -t /nemo-retriever-service: . - repository: nvcr.io/nvidia/nemo-microservices/nrl-service - tag: "26.5.0" + repository: nvcr.io/nvstaging/nim/nrl-service + tag: "26.08-RC1" pullPolicy: IfNotPresent # Optional image for in-pod Hugging Face models (service-gpu Docker target). @@ -553,7 +554,7 @@ serviceConfig: # unless this explicit endpoint is set. nemotronParseInvokeUrl: "" # Optional model override. Leave empty to infer the operator-managed - # Parse v1.2 or v2.0 model from its image repository. For external + # Parse v2.0 or v1.2 model from its image repository. For external # endpoints, set the matching model explicitly. nemotronParseModel: "" embedInvokeUrl: "" @@ -1042,7 +1043,7 @@ nimOperator: # Optional (enabled: false by default, NOT auto-wired): # - rerankqa (VL reranker, # ≈ 3.1 GiB GPU) - # - nemotron_parse (Parse v1.2 by default, ≈ 3.5 GiB GPU) + # - nemotron_parse (Parse v2.0 by default, ≈ 3.5 GiB GPU) # - nemotron_3_nano_omni_30b_a3b_reasoning (Omni 30B caption NIM, # ≈ 62 GiB BF16 weights) # - answer_llm (OpenAI-compatible answer LLM, @@ -1059,9 +1060,9 @@ nimOperator: # on alongside the core stack at install time would silently pull tens # of gigabytes of model weights (Omni 30B ≈ 62 GiB BF16) and consume # an additional dedicated GPU per NIM with no opt-in from the operator. - # See helm/README.md "Recommended minimal install (26.05)" for the + # See helm/README.md "Recommended minimal install (26.08)" for the # opt-in flags and "Image tag conventions" for what the - # ``1.7.0-variant`` Parse / Omni tags mean. + # ``*-variant`` Parse / Omni tags mean. # --------------------------------------------------------------------------- # Page-element detector. Uses the combined nemotron-object-detection NIM image @@ -1304,27 +1305,25 @@ nimOperator: # LiteLLM/OpenAI model id rendered into retriever-service.yaml when # this operator-managed LLM is enabled and serviceConfig.llm.model is # left empty. - model: "openai/nvidia/llama-3.3-nemotron-super-49b-v1.5" + model: "openai/nvidia/llama-3.3-nemotron-super-49b-v1" # Optional prompt prefix to apply when this operator-managed LLM # auto-wires /v1/answer. Leave empty to keep the LLM deployment and # prompt defaults model-neutral; use serviceConfig.llm.reasoningEnabled # for the portable request-level reasoning toggle. ragSystemPromptPrefix: "" image: - repository: nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 - tag: "2.0.5" + repository: nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1 + tag: "1.10.1" pullPolicy: IfNotPresent pullSecrets: - ngc-secret authSecret: ngc-api # Per-NIM NIMCache GPU/profile filter. See `nimOperator.modelProfile` # above for the shape; non-empty REPLACES the chart-wide default. - # Cache only the default BF16 2-GPU profile used by the Super-49B - # default resources. Override this when deploying a different model, - # profile, or GPU topology. - modelProfile: - profiles: - - "1146f49f84dff5dea09f5aa633cc70b92d7d972223d67878c841cd0fbccad4fb" + # Empty lets the NIM Operator select a compatible Super-49B v1 + # profile for the two-GPU resources below. Override when pinning a + # specific profile or GPU topology. + modelProfile: {} storage: pvc: create: true @@ -1416,7 +1415,7 @@ nimOperator: - name: NIM_TRITON_LOG_VERBOSE value: "1" - # Nemotron Parse v1.2. Optional structured document parser. + # Nemotron Parse v2.0. Optional structured document parser. # # Disabled by default per the 26.05 "optional and disabled by default" # contract (see docs/extraction/deployment-options.md). Opt in only when @@ -1424,20 +1423,17 @@ nimOperator: # NIM consumes one GPU and ~3.5 GiB of GPU memory at runtime, plus # ~16 GB of on-disk NIM model cache (see prerequisites-support-matrix). # - # The default image tag (`1.7.0-variant`) is the Nemotron Parse v1.2 - # NIM SKU shipped alongside the 26.05 release; the `-variant` suffix - # marks it as the build that ships with the per-GPU TensorRT engine - # variants required by NIM Operator profile reconciliation, matching - # `nemotron-3-nano-omni-30b-a3b-reasoning:1.7.0-variant` and the row - # for Nemotron Parse in - # docs/extraction/prerequisites-support-matrix.md "Model hardware - # requirements". Override to a different NGC tag only when validated - # against the matching service/Helm release. + # The default image tag (`2.0.8-variant`) is the Nemotron Parse v2.0 + # NIM SKU from the 26.08 Test Plan; the `-variant` suffix marks it as + # the build that ships with the per-GPU TensorRT engine variants + # required by NIM Operator profile reconciliation. Override to a + # different NGC tag only when validated against the matching + # service/Helm release. nemotron_parse: enabled: false image: - repository: nvcr.io/nim/nvidia/nemotron-parse-v1.2 - tag: "1.7.0-variant" + repository: nvcr.io/nim/nvidia/nemotron-parse-v2.0 + tag: "2.0.8-variant" pullPolicy: IfNotPresent pullSecrets: - ngc-secret @@ -1488,19 +1484,19 @@ nimOperator: # Operator from the model profile (override with `kubectl edit # nimservice` if necessary). # - # The default image tag (`1.7.0-variant`) is the Omni NIM SKU shipped - # alongside the 26.05 release; the `-variant` suffix marks it as the - # build that ships with the per-GPU TensorRT engine variants required - # by NIM Operator profile reconciliation, matching the row for the - # Omni caption NIM in - # docs/extraction/prerequisites-support-matrix.md "Model hardware - # requirements" (footnote ³). Override to a different NGC tag only - # when validated against the matching service/Helm release. + # The default image tag (`2.0.4-variant`) is the Omni NIM SKU from the + # 26.08 Test Plan; the `-variant` suffix marks it as the build that + # ships with the per-GPU TensorRT engine variants required by NIM + # Operator profile reconciliation, matching the row for the Omni + # caption NIM in docs/extraction/prerequisites-support-matrix.md + # "Model hardware requirements" (footnote ³). Override to a different + # NGC tag only when validated against the matching service/Helm + # release. nemotron_3_nano_omni_30b_a3b_reasoning: enabled: false image: repository: nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning - tag: "1.7.0-variant" + tag: "2.0.4-variant" pullPolicy: IfNotPresent pullSecrets: - ngc-secret diff --git a/nemo_retriever/src/nemo_retriever/operators/extract/parse/nemotron_parse.py b/nemo_retriever/src/nemo_retriever/operators/extract/parse/nemotron_parse.py index 973587f9c..7eda8b7ff 100644 --- a/nemo_retriever/src/nemo_retriever/operators/extract/parse/nemotron_parse.py +++ b/nemo_retriever/src/nemo_retriever/operators/extract/parse/nemotron_parse.py @@ -48,7 +48,7 @@ # Constants # --------------------------------------------------------------------------- -NEMOTRON_PARSE_REMOTE_DEFAULT_MODEL = "nvidia/nemotron-parse-v1.2" +NEMOTRON_PARSE_REMOTE_DEFAULT_MODEL = "nvidia/nemotron-parse-v2.0" NEMOTRON_PARSE_V2_MODEL = "nvidia/nemotron-parse-v2.0" NEMOTRON_PARSE_HOSTED_MODEL = "nvidia/nemotron-parse" NEMOTRON_PARSE_LOCAL_DEFAULT_MODEL = "nvidia/NVIDIA-Nemotron-Parse-v1.2" diff --git a/nemo_retriever/src/nemo_retriever/service/config.py b/nemo_retriever/src/nemo_retriever/service/config.py index be397dcf5..4de1a7514 100644 --- a/nemo_retriever/src/nemo_retriever/service/config.py +++ b/nemo_retriever/src/nemo_retriever/service/config.py @@ -165,8 +165,8 @@ class NimEndpointsConfig(RichModel): description=( "Model identifier passed to the remote Nemotron Parse endpoint. " "Use nvidia/nemotron-parse for NVIDIA-hosted inference and " - "nvidia/nemotron-parse-v1.2 for the default self-hosted NIM, or " - "nvidia/nemotron-parse-v2.0 for an explicitly selected Parse 2.0 NIM. " + "nvidia/nemotron-parse-v2.0 for the default self-hosted NIM, or " + "nvidia/nemotron-parse-v1.2 for an explicitly selected Parse 1.2 NIM. " "Server-owned — clients cannot override the deployed Parse SKU." ), ) diff --git a/nemo_retriever/tests/test_actor_operators.py b/nemo_retriever/tests/test_actor_operators.py index 21730e25f..081fb6e58 100644 --- a/nemo_retriever/tests/test_actor_operators.py +++ b/nemo_retriever/tests/test_actor_operators.py @@ -417,7 +417,6 @@ def test_process(self, mock_fn): def test_remote_chat_completions_uses_v1_2_protocol(self): from nemo_retriever.operators.extract.parse.nemotron_parse import ( NEMOTRON_PARSE_DEFAULT_TASK_PROMPT, - NEMOTRON_PARSE_REMOTE_DEFAULT_MODEL, nemotron_parse_pages, ) @@ -435,16 +434,51 @@ def invoke_chat_completions_images(self, **kwargs): result = nemotron_parse_pages( df, invoke_url="http://nemotron-parse:8000/v1/chat/completions", + nemotron_parse_model="nvidia/nemotron-parse-v1.2", extract_text=True, nim_client=client, ) assert result["text"].tolist() == ["Hello world"] - assert client.kwargs["model"] == NEMOTRON_PARSE_REMOTE_DEFAULT_MODEL + assert client.kwargs["model"] == "nvidia/nemotron-parse-v1.2" assert client.kwargs["task_prompt"] == NEMOTRON_PARSE_DEFAULT_TASK_PROMPT assert client.kwargs["extra_body"] == {"max_tokens": 8192} assert client.kwargs["repetition_penalty"] == 1.1 + def test_remote_chat_completions_defaults_to_v2_protocol(self): + from nemo_retriever.operators.extract.parse.nemotron_parse import ( + NEMOTRON_PARSE_REMOTE_DEFAULT_MODEL, + NEMOTRON_PARSE_V2_MODEL, + nemotron_parse_pages, + ) + + class _FakeNIMClient: + def __init__(self): + self.kwargs = None + + def invoke_chat_completions_images(self, **kwargs): + self.kwargs = kwargs + return ["Hello world"] + + client = _FakeNIMClient() + df = pd.DataFrame({"page_image": [{"image_b64": "aW1hZ2U="}]}) + + result = nemotron_parse_pages( + df, + invoke_url="http://nemotron-parse:8000/v1/chat/completions", + extract_text=True, + nim_client=client, + ) + + assert result["text"].tolist() == ["Hello world"] + assert NEMOTRON_PARSE_REMOTE_DEFAULT_MODEL == NEMOTRON_PARSE_V2_MODEL + assert client.kwargs["model"] == NEMOTRON_PARSE_V2_MODEL + assert client.kwargs["extra_body"] == { + "max_tokens": 9000, + "top_k": 1, + "skip_special_tokens": False, + } + def test_remote_chat_completions_uses_v2_protocol(self): from nemo_retriever.operators.extract.parse.nemotron_parse import ( NEMOTRON_PARSE_DEFAULT_TASK_PROMPT, @@ -549,7 +583,7 @@ def invoke_chat_completions_images(self, **kwargs): ("endpoint", "model", "expected_model", "expected_profile"), [ ("https://integrate.api.nvidia.com/v1/chat/completions", None, "nvidia/nemotron-parse", "hosted_tool_call"), - ("http://parse:8000/v1/chat/completions", None, "nvidia/nemotron-parse-v1.2", "v1_2_tagged"), + ("http://parse:8000/v1/chat/completions", None, "nvidia/nemotron-parse-v2.0", "v2_0_tagged"), ( "http://parse:8000/v1/chat/completions", "nvidia/nemotron-parse", diff --git a/nemo_retriever/tests/test_helm_answer_llm_generation.py b/nemo_retriever/tests/test_helm_answer_llm_generation.py index c962e303b..fcbff823d 100644 --- a/nemo_retriever/tests/test_helm_answer_llm_generation.py +++ b/nemo_retriever/tests/test_helm_answer_llm_generation.py @@ -19,10 +19,9 @@ _ANSWER_LLM_KEY = " answer_llm:" _ANSWER_LLM_SERVICE = "answer-llm" -_SUPER49B_REPOSITORY = "nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1.5" -_SUPER49B_TAG = "2.0.5" -_SUPER49B_MODEL = "openai/nvidia/llama-3.3-nemotron-super-49b-v1.5" -_SUPER49B_PROFILE = "1146f49f84dff5dea09f5aa633cc70b92d7d972223d67878c841cd0fbccad4fb" +_SUPER49B_REPOSITORY = "nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1" +_SUPER49B_TAG = "1.10.1" +_SUPER49B_MODEL = "openai/nvidia/llama-3.3-nemotron-super-49b-v1" _NANO_SERVICE = "nemotron-3-nano" _NANO_REPOSITORY = "nvcr.io/nim/nvidia/nemotron-3-nano" _NANO_TAG = "1.7.0-variant" @@ -78,7 +77,7 @@ def test_values_define_generic_answer_llm_with_super49b_defaults(self) -> None: self.assertIn(f'model: "{_SUPER49B_MODEL}"', block) self.assertIn("nvidia.com/gpu: 2", block) self.assertIn('size: "250Gi"', block) - self.assertIn(_SUPER49B_PROFILE, block) + self.assertIn("modelProfile: {}", block) self.assertIn("reasoningEnabled: true", values) self.assertIn('ragSystemPromptPrefix: ""', block) @@ -100,8 +99,6 @@ def test_answer_llm_opt_in_renders_default_super49b_nim_and_autowires_llm_config self.assertIn(f"repository: {_SUPER49B_REPOSITORY}", proc.stdout) self.assertIn(f"tag: {_SUPER49B_TAG}", proc.stdout) self.assertIn("nvidia.com/gpu: 2", proc.stdout) - self.assertIn("profiles:", proc.stdout) - self.assertIn(_SUPER49B_PROFILE, proc.stdout) self.assertIn("NIM_PASSTHROUGH_ARGS", proc.stdout) self.assertIn("--disable-custom-all-reduce", proc.stdout) self.assertIn("NCCL_IB_DISABLE", proc.stdout) @@ -172,7 +169,6 @@ def test_answer_llm_can_swap_to_nano_image_model_and_profile(self) -> None: self.assertIn("NIM_TENSOR_PARALLEL_SIZE", proc.stdout) self.assertIn(_NANO_A100_PROFILE, proc.stdout) self.assertIn("nvidia.com/gpu: 1", proc.stdout) - self.assertNotIn(_SUPER49B_PROFILE, proc.stdout) self.assertIn(f'api_base: "http://{_NANO_SERVICE}:8000/v1"', proc.stdout) self.assertIn(f'model: "{_NANO_MODEL}"', proc.stdout) self.assertIn("rag_system_prompt_prefix: null", proc.stdout) diff --git a/nemo_retriever/tests/test_helm_nemotron_parse_endpoint.py b/nemo_retriever/tests/test_helm_nemotron_parse_endpoint.py index 58176ae71..3d7eda70d 100644 --- a/nemo_retriever/tests/test_helm_nemotron_parse_endpoint.py +++ b/nemo_retriever/tests/test_helm_nemotron_parse_endpoint.py @@ -82,23 +82,23 @@ def test_helm_template_autowires_operator_parse_endpoint() -> None: api_versions=("apps.nvidia.com/v1alpha1",), ) assert f'nemotron_parse_invoke_url: "http://{PARSE_SERVICE}:8000{PARSE_PATH}"' in rendered - assert 'nemotron_parse_model: "nvidia/nemotron-parse-v1.2"' in rendered + assert 'nemotron_parse_model: "nvidia/nemotron-parse-v2.0"' in rendered -def test_helm_template_autowires_v2_model_from_parse_image_repository() -> None: +def test_helm_template_autowires_v1_2_model_from_parse_image_repository() -> None: rendered = _helm_template( ( "--set", "nimOperator.nemotron_parse.enabled=true", "--set", - "nimOperator.nemotron_parse.image.repository=registry.example/nemotron-parse-v2.0", + "nimOperator.nemotron_parse.image.repository=registry.example/nemotron-parse-v1.2", "--set", - "nimOperator.nemotron_parse.image.tag=2.0.8-variant", + "nimOperator.nemotron_parse.image.tag=1.7.0-variant", ), api_versions=("apps.nvidia.com/v1alpha1",), ) assert f'nemotron_parse_invoke_url: "http://{PARSE_SERVICE}:8000{PARSE_PATH}"' in rendered - assert 'nemotron_parse_model: "nvidia/nemotron-parse-v2.0"' in rendered + assert 'nemotron_parse_model: "nvidia/nemotron-parse-v1.2"' in rendered def test_helm_template_explicit_hosted_endpoint_and_model_win() -> None: diff --git a/nemo_retriever/tests/test_helm_nimcache_model_profile.py b/nemo_retriever/tests/test_helm_nimcache_model_profile.py index 70f7c69a0..c4b5e0a61 100644 --- a/nemo_retriever/tests/test_helm_nimcache_model_profile.py +++ b/nemo_retriever/tests/test_helm_nimcache_model_profile.py @@ -18,10 +18,9 @@ * ``values.yaml`` carries a chart-wide ``nimOperator.modelProfile`` default plus a per-NIM ``nimOperator..modelProfile`` override - for every NIMCache the chart provisions. Existing extraction NIMs - default their per-NIM override to ``{}``; ``answer_llm`` is the - intentional exception because its Super-49B default pins the bundled - two-GPU profile by default. + for every NIMCache the chart provisions. Every NIM, including + ``answer_llm``, defaults its per-NIM override to ``{}`` so the NIM + Operator can select a compatible profile unless an operator opts in. * A ``helm template`` with **no overrides** renders no ``model:`` block on default-empty-profile NIMCaches (preserves operator default). * ``--set nimOperator.modelProfile.gpus[0]...`` renders an identical @@ -57,9 +56,10 @@ _CHART_DIR = _REPO_ROOT / "nemo_retriever/helm" # Per-NIM keys whose NIMCache modelProfile values intentionally default -# to ``{}``, preserving pre-fix operator behaviour unless an operator -# opts into GPU/profile filtering. answer_llm is tracked separately because -# its Super-49B default ships with a pinned profile for the bundled two-GPU NIM. +# Every chart NIM defaults ``modelProfile`` to ``{}`` so the NIM Operator +# can select a compatible profile unless an operator opts into filtering. +# answer_llm is listed separately from the helm-template opted-in set +# because it stays disabled unless explicitly enabled. _EMPTY_MODEL_PROFILE_NIM_KEYS: tuple[str, ...] = ( "page_elements", "table_structure", @@ -71,7 +71,6 @@ "audio", ) _ANSWER_LLM_NIM_KEY = "answer_llm" -_SUPER49B_DEFAULT_PROFILE = "1146f49f84dff5dea09f5aa633cc70b92d7d972223d67878c841cd0fbccad4fb" _ALL_NIM_KEYS: tuple[str, ...] = _EMPTY_MODEL_PROFILE_NIM_KEYS + (_ANSWER_LLM_NIM_KEY,) @@ -186,13 +185,13 @@ def test_values_exposes_per_nim_override_for_every_nim(self) -> None: "operator opts in.", ) - def test_answer_llm_exposes_intentional_default_super49b_model_profile(self) -> None: - """Super-49B pins its bundled two-GPU NIM profile by default.""" + def test_answer_llm_defaults_to_empty_model_profile(self) -> None: + """Super-49B v1 lets the NIM Operator select a compatible profile.""" values = _read_required_file(_VALUES_YAML) loaded = yaml.safe_load(values) self.assertEqual( loaded["nimOperator"][_ANSWER_LLM_NIM_KEY]["modelProfile"], - {"profiles": [_SUPER49B_DEFAULT_PROFILE]}, + {}, ) # ------------------------------------------------------------------ @@ -294,8 +293,7 @@ def test_chart_wide_modelprofile_applies_to_default_empty_nimcaches(self) -> Non This is the exact customer ask for the existing extraction NIMs: one --set flag makes each rendered cache download only the H100 - profile. Super-49B is covered separately because it intentionally - pins its bundled default profile. + profile. """ proc = _helm_template( extra_args=( diff --git a/nemo_retriever/tests/test_helm_optional_nims_disabled_by_default.py b/nemo_retriever/tests/test_helm_optional_nims_disabled_by_default.py index e010542fe..cefb5f070 100644 --- a/nemo_retriever/tests/test_helm_optional_nims_disabled_by_default.py +++ b/nemo_retriever/tests/test_helm_optional_nims_disabled_by_default.py @@ -28,9 +28,9 @@ auto-wiring). * Explicit opt-in still reconciles them, so the documented ``--set nimOperator..enabled=true`` workflow keeps working. -* The README and ``values.yaml`` document the ``1.7.0-variant`` tag +* The README and ``values.yaml`` document the ``*-variant`` tags used by Parse + Omni so air-gapped mirror pipelines and - reproducibility audits can map it to the 26.05 release. + reproducibility audits can map them to the 26.08 release. The integration tests shell out to ``helm template`` when ``helm`` is on ``$PATH``; otherwise they skip cleanly. @@ -68,10 +68,12 @@ _PARSE_SERVICE_NAME = "name: nemotron-parse" _OMNI_SERVICE_NAME = "name: nemotron-3-nano-omni-30b-a3b-reasoning" -# Image tag the chart pins for both NIMs in 26.05. Documenting it on -# both ends (values.yaml comments + README) keeps air-gapped mirror -# pipelines pointed at the right NGC tag. -_VARIANT_TAG = "1.7.0-variant" +# Image tags the chart pins for Parse and Omni in 26.08. Documenting +# them on both ends (values.yaml comments + README) keeps air-gapped +# mirror pipelines pointed at the right NGC tags. +_PARSE_VARIANT_TAG = "2.0.8-variant" +_OMNI_VARIANT_TAG = "2.0.4-variant" +_PARSE_REPOSITORY = "nvcr.io/nim/nvidia/nemotron-parse-v2.0" # Repositories the rerank NIM may be pinned to. The chart MUST point at # the VL SKU — the text-only SKU silently degrades multimodal @@ -239,22 +241,30 @@ def test_values_rerankqa_image_is_vl_sku(self) -> None: ) def test_values_document_the_variant_tag(self) -> None: - """The ``1.7.0-variant`` tag must be explained in ``values.yaml``. - - Customer-facing pain (3) from the bug report: ``1.7.0-variant`` - is unsearchable on NGC and has no docs entry, so air-gapped - mirror pipelines and reproducibility audits cannot map the tag - to a known release. An inline comment in ``values.yaml`` is the - minimum bar — the README's "Image tag conventions" subsection - covers it in more depth. + """The Parse and Omni ``*-variant`` tags must be explained in ``values.yaml``. + + Customer-facing pain (3) from the bug report: ``*-variant`` + tags are unsearchable on NGC and have no docs entry, so + air-gapped mirror pipelines and reproducibility audits cannot + map the tag to a known release. An inline comment in + ``values.yaml`` is the minimum bar — the README's "Image tag + conventions" subsection covers it in more depth. """ values = _read_required_file(_VALUES_YAML) self.assertEqual( - values.count(f'tag: "{_VARIANT_TAG}"'), - 2, - "Expected exactly two NIM image tags to be pinned to " - f"{_VARIANT_TAG!r} (Parse + Omni). If you bumped one tag, " - "bump the other together or split this test.", + values.count(f'tag: "{_PARSE_VARIANT_TAG}"'), + 1, + f"Expected exactly one NIM image tag pinned to {_PARSE_VARIANT_TAG!r} (Parse).", + ) + self.assertEqual( + values.count(f'tag: "{_OMNI_VARIANT_TAG}"'), + 1, + f"Expected exactly one NIM image tag pinned to {_OMNI_VARIANT_TAG!r} (Omni).", + ) + self.assertIn( + f"repository: {_PARSE_REPOSITORY}", + values, + f"values.yaml must pin Parse to `{_PARSE_REPOSITORY}` per the 26.08 Test Plan.", ) # The comment must explain what `-variant` means, not just # mention the literal string. @@ -317,10 +327,10 @@ def test_readme_image_table_pins_vl_rerank_sku(self) -> None: ) def test_readme_documents_image_tag_conventions(self) -> None: - """A dedicated subsection must explain the ``1.7.0-variant`` tag. + """A dedicated subsection must explain the Parse and Omni ``*-variant`` tags. - Without this, the customer-facing complaint that ``1.7.0-variant`` - is undocumented stays valid even after the defaults flip. + Without this, the customer-facing complaint that ``*-variant`` + tags are undocumented stays valid even after the defaults flip. """ readme = _read_required_file(_README_MD) self.assertIn( @@ -330,9 +340,16 @@ def test_readme_documents_image_tag_conventions(self) -> None: "values.yaml entries and per-NIM table can link to it.", ) self.assertIn( - _VARIANT_TAG, + _PARSE_VARIANT_TAG, + readme, + f"README must mention the {_PARSE_VARIANT_TAG!r} tag verbatim so a " + "`grep` for the tag inside the chart docs returns the " + "explanation.", + ) + self.assertIn( + _OMNI_VARIANT_TAG, readme, - f"README must mention the {_VARIANT_TAG!r} tag verbatim so a " + f"README must mention the {_OMNI_VARIANT_TAG!r} tag verbatim so a " "`grep` for the tag inside the chart docs returns the " "explanation.", ) @@ -471,9 +488,9 @@ def test_helm_template_parse_opt_in_renders_nimservice(self) -> None: ) # The pinned tag must travel with the opt-in. self.assertIn( - f"tag: {_VARIANT_TAG}", + f"tag: {_PARSE_VARIANT_TAG}", proc.stdout, - f"Parse opt-in must render with the pinned {_VARIANT_TAG!r} tag.", + f"Parse opt-in must render with the pinned {_PARSE_VARIANT_TAG!r} tag.", ) def test_helm_template_omni_opt_in_renders_nimservice_and_caption(self) -> None: @@ -538,7 +555,7 @@ def test_helm_template_rerankqa_opt_in_renders_vl_nimservice(self) -> None: ) def test_helm_template_omni_image_tag_pins_to_variant(self) -> None: - """Opt-in Omni must render with the pinned ``1.7.0-variant`` tag, not ``:latest``. + """Opt-in Omni must render with the pinned ``2.0.4-variant`` tag, not ``:latest``. The bug report's reproducibility concern: substituting ``:latest`` would silently move to a different NIM build. @@ -551,9 +568,9 @@ def test_helm_template_omni_image_tag_pins_to_variant(self) -> None: ) _assert_helm_ok(self, proc) self.assertIn( - f"tag: {_VARIANT_TAG}", + f"tag: {_OMNI_VARIANT_TAG}", proc.stdout, - f"Omni opt-in must render with the pinned {_VARIANT_TAG!r} tag.", + f"Omni opt-in must render with the pinned {_OMNI_VARIANT_TAG!r} tag.", ) # And there is no stray `:latest` reference in the rendered # NIMCache/NIMService manifests for either heavy-weight NIM.