Add remote NIM request payload capture

docs/docs/extraction/nemo-retriever-api-reference.md

@@ -140,6 +140,79 @@ For a support-oriented mapping of extraction paths, error signals, corrective
 actions, and escalation criteria, refer to
 [Python API error triage](troubleshoot.md#python-api-error-triage).
 
+## Capture remote NIM requests { #capture-remote-nim-requests }
+
+Use `InferenceCaptureConfig` to persist the final outbound request payload for
+remote inference. This is useful when you want to replay representative
+extraction or retrieval requests against a compatible self-hosted NIM.
+
+The feature is disabled by default. It captures supported HTTP JSON NIM requests
+from the generic NIM client, embedding, and reranking paths, plus Triton gRPC
+requests. It works with self-hosted and NVIDIA-hosted endpoints. It does not
+capture inputs sent to local Transformers or vLLM models because those execution
+modes do not make a NIM network request.
+
+The following example enables capture for an in-process ingestion pipeline.
+
+```python
+from nemo_retriever import create_ingestor
+from nemo_retriever.common.inference_capture import InferenceCaptureConfig
+
+capture = InferenceCaptureConfig(
+    storage_uri="/var/lib/nemo-retriever/nim-captures",
+    failure_mode="required",
+    operations=("ingest",),
+)
+
+pipeline = create_ingestor(
+    run_mode="inprocess",
+    inference_capture=capture,
+)
+```
+
+Pass the same configuration to `GraphIngestor` or `Retriever` when you create
+those objects directly. Use `operations=("query",)` to capture query embedding
+and reranking requests, or omit `operations` to capture both ingestion and
+query requests. Use `stages` to restrict capture to named inference stages.
+
+Each captured attempt has its own directory. HTTP JSON captures contain
+`manifest.json` and `request.json`. Triton gRPC captures contain `manifest.json`
+and `request.bin`, a NumPy `savez_compressed` archive of the input tensors. Its
+manifest has `protocol: "grpc"` and records the input names, data types, requested
+output names, and inference parameters. All manifests record the operation,
+stage, sanitized endpoint, model when available, timestamp, and retry attempt.
+Request inputs are persisted for replay. Embedding HTTP captures also add an
+optional replay-metadata sidecar in `manifest.json` at `metadata.replay`. It does
+not change `request.json`, so the captured request remains the exact payload
+sent to the NIM.
+
+`metadata.replay.replay_version` is `1`. Its `records` list is aligned with the
+request `input` positions. Each item includes `input_index`, an `input_sha256`,
+and the source `record`. The source record preserves available row identity and
+context, including VectorDB fields or application query-identifying fields when
+they are present. It omits `_content` and any `metadata.embedding` value. Do not
+assume a normalized VectorDB identity or a dedicated query-ID field.
+
+The manifest `operation` labels ingestion as `ingest` and retrieval embedding as
+`query`. Use the operation, input position, and replay record together to map a
+captured embedding request back to its source row. The replay record can contain
+document metadata, retrieved content, and query identifiers. Treat it as
+sensitive data. The recorder does not persist authorization headers, API keys,
+or endpoint query strings.
+
+By default, `failure_mode="best_effort"` logs a capture write failure and still
+sends the inference request. Set `failure_mode="required"` when you generate
+replay fixtures and want a capture write failure to stop the request before it
+is sent. `storage_uri` accepts a local directory or an
+[fsspec-compatible](https://filesystem-spec.readthedocs.io/) URI.
+
+!!! warning "Treat capture artifacts as sensitive data"
+
+    Request payloads can contain document text, user queries, and base64-encoded
+    page images. Store captures only in an approved location. Configure access
+    controls, encryption, and retention policies before enabling capture in a
+    production environment.
+
 !!! note "Version-specific behavior"
 
     This reference describes the current NeMo Retriever Library. Older

nemo_retriever/dev/compose/service-mode.compose.yaml

@@ -99,11 +99,11 @@ services:
     restart: unless-stopped
 
   # Page-elements and table-structure are separate Compose services (matching
-  # Helm) but both run the combined nemotron-object-detection:2.0.0 image.
+  # Helm) but both run the combined nemotron-object-detection:2.0.1 image.
   nim-page-elements:
     <<: *nim-service
     profiles: [nims-core]
-    image: ${NIM_PAGE_ELEMENTS_IMAGE:-nvcr.io/nim/nvidia/nemotron-object-detection}:${NIM_PAGE_ELEMENTS_TAG:-2.0.0}
+    image: ${NIM_PAGE_ELEMENTS_IMAGE:-nvcr.io/nim/nvidia/nemotron-object-detection}:${NIM_PAGE_ELEMENTS_TAG:-2.0.1}
     ports: ["${NIM_PAGE_ELEMENTS_HOST_PORT:-8001}:8000"]
     environment:
       <<: *nim-environment
@@ -125,7 +125,7 @@ services:
   nim-table-structure:
     <<: *nim-service
     profiles: [nims-core]
-    image: ${NIM_TABLE_STRUCTURE_IMAGE:-nvcr.io/nim/nvidia/nemotron-object-detection}:${NIM_TABLE_STRUCTURE_TAG:-2.0.0}
+    image: ${NIM_TABLE_STRUCTURE_IMAGE:-nvcr.io/nim/nvidia/nemotron-object-detection}:${NIM_TABLE_STRUCTURE_TAG:-2.0.1}
     ports: ["${NIM_TABLE_STRUCTURE_HOST_PORT:-8002}:8000"]
     environment:
       <<: *nim-environment
@@ -147,7 +147,7 @@ services:
   nim-ocr:
     <<: *nim-service
     profiles: [nims-core]
-    image: ${NIM_OCR_IMAGE:-nvcr.io/nim/nvidia/nemotron-ocr-v2}:${NIM_OCR_TAG:-2.0.0}
+    image: ${NIM_OCR_IMAGE:-nvcr.io/nim/nvidia/nemotron-ocr-v2}:${NIM_OCR_TAG:-2.0.1}
     ports: ["${NIM_OCR_HOST_PORT:-8003}:8000"]
     environment:
       <<: *nim-environment
@@ -170,7 +170,7 @@ services:
   nim-embedding:
     <<: *nim-service
     profiles: [nims-core]
-    image: ${NIM_EMBED_IMAGE:-nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2}:${NIM_EMBED_TAG:-1.12.0}
+    image: ${NIM_EMBED_IMAGE:-nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2}:${NIM_EMBED_TAG:-2.3.0}
     ports: ["${NIM_EMBED_HOST_PORT:-8004}:8000"]
     environment:
       <<: *nim-environment

nemo_retriever/helm/README.md

@@ -182,6 +182,53 @@ helm install retriever ./nemo_retriever/helm \
 via `optional: true` `secretKeyRef`, so the install still succeeds when
 the secret is absent (useful for fully local NIM endpoints).
 
+### Capture outbound NIM requests
+
+The retriever service can persist final outbound request payloads for supported
+remote HTTP JSON and Triton gRPC NIM calls during service ingestion, plus
+VectorDB query embedding calls. This is useful when you want to collect replay
+fixtures for a compatible self-hosted NIM. It does not capture other query-phase
+traffic.
+
+The capture location is administrator-owned service configuration. It is not a
+field that an ingest request can override. Mount or otherwise make an approved
+location writable by both the retriever service and VectorDB query deployment,
+then configure the chart:
+
+```yaml
+serviceConfig:
+  inferenceCapture:
+    enabled: true
+    storageUri: /var/lib/nemo-retriever/nim-captures
+    failureMode: best_effort
+    # Optional filters. Empty lists capture all supported service-ingestion stages.
+    operations: []
+    stages: []
+```
+
+`storageUri` is required when capture is enabled. Use `failureMode: required`
+when every request must be captured before the service sends it to the NIM. The
+default, `best_effort`, logs capture write failures and continues ingestion.
+Each HTTP JSON attempt writes `manifest.json` and `request.json` in its own
+capture directory. A Triton gRPC attempt writes `manifest.json` and `request.bin`,
+a NumPy `savez_compressed` archive of input tensors; its manifest records the
+input/output names, data types, and inference parameters. Embedding HTTP
+captures also include an optional replay-metadata sidecar in `manifest.json` at
+`metadata.replay`; it does not alter the raw `request.json` payload. Version 1
+metadata has `records` aligned with request input positions. Each record includes
+the input index, a SHA-256 hash of the input, and its source row. The source row
+preserves available VectorDB identity, document context, or application query
+identity, but does not provide a normalized VDB schema or a dedicated query-ID
+field. `manifest.json` labels service ingestion as `operation: "ingest"` and
+VectorDB query embedding as `operation: "query"`. The recorder does not persist
+authorization headers, API keys, or endpoint query strings.
+
+Capture artifacts can include document text, base64-encoded page images, source
+row metadata, retrieved content, and query-identifying fields. Use an approved
+destination with suitable access controls, encryption, and retention policies.
+This feature is separate from pipeline `.store()`, which persists ingest output
+artifacts rather than outbound NIM requests.
+
 ### 3. Install with the NIM Operator (in-cluster NIMs)
 
 Install the [NIM Operator](https://docs.nvidia.com/nim-operator/) first so
@@ -324,6 +371,11 @@ The retriever service picks up the in-cluster ASR endpoint when `nimOperator.aud
 | `serviceConfig.pipeline.realtimeWorkers`          | `24`    | Per-pod realtime worker count. |
 | `serviceConfig.pipeline.batchWorkers`             | `48`    | Per-pod batch worker count. Refer to [Timeouts and alleviating ingest failures](#timeouts-and-alleviating-ingest-failures) if embed or pool errors appear under load. |
 | `serviceConfig.resources.maxUploadBytes`          | `500000000` | Maximum upload file size in bytes; requests exceeding the limit are rejected before buffering. |
+| `serviceConfig.inferenceCapture.enabled`          | `false` | Enables administrator-owned capture for supported outbound remote-NIM requests during ingestion and VectorDB query embedding. |
+| `serviceConfig.inferenceCapture.storageUri`       | `""` | Local directory or fsspec-compatible URI. Required when capture is enabled. |
+| `serviceConfig.inferenceCapture.failureMode`      | `best_effort` | Use `required` to fail before a request is sent when it cannot be captured. |
+| `serviceConfig.inferenceCapture.operations`       | `[]` | Optional operation filter. |
+| `serviceConfig.inferenceCapture.stages`           | `[]` | Optional inference-stage filter. |
 | `serviceConfig.nimEndpoints.*InvokeUrl`           | `""`    | Override the auto-resolved NIM Operator URL. Available knobs: `pageElementsInvokeUrl`, `tableStructureInvokeUrl`, `ocrInvokeUrl`, `embedInvokeUrl`, and `captionInvokeUrl` (refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b)). |
 | `serviceConfig.nimEndpoints.captionModelName`     | `""`    | Model id sent to the remote VLM. Auto-set to `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` whenever a caption URL is resolved. |
 | `serviceConfig.llm.enabled`                         | `false` | Enables `POST /v1/answer`. Auto-flips to true when `nimOperator.answer_llm` is enabled and the operator URL resolves. |

nemo_retriever/helm/templates/configmap.yaml

@@ -96,6 +96,13 @@ logging:
   file: {{ .Values.serviceConfig.logging.file | quote }}
   format: {{ .Values.serviceConfig.logging.format | quote }}
 
+inference_capture:
+  enabled: {{ .Values.serviceConfig.inferenceCapture.enabled }}
+  storage_uri: {{ if .Values.serviceConfig.inferenceCapture.storageUri }}{{ .Values.serviceConfig.inferenceCapture.storageUri | quote }}{{ else }}null{{ end }}
+  failure_mode: {{ .Values.serviceConfig.inferenceCapture.failureMode | quote }}
+  operations: {{ .Values.serviceConfig.inferenceCapture.operations | toJson }}
+  stages: {{ .Values.serviceConfig.inferenceCapture.stages | toJson }}
+
 nim_endpoints:
   page_elements_invoke_url: {{ .pageElementsURL | quote }}
   table_structure_invoke_url: {{ .tableStructureURL | quote }}

nemo_retriever/helm/templates/deployment-vectordb.yaml

@@ -137,6 +137,14 @@ spec:
               containerPort: {{ $vdb.port }}
               protocol: TCP
           env:
+            {{- if .Values.serviceConfig.inferenceCapture.enabled }}
+            - name: NEMO_RETRIEVER_INFERENCE_CAPTURE_URI
+              value: {{ .Values.serviceConfig.inferenceCapture.storageUri | quote }}
+            - name: NEMO_RETRIEVER_INFERENCE_CAPTURE_FAILURE_MODE
+              value: {{ .Values.serviceConfig.inferenceCapture.failureMode | quote }}
+            - name: NEMO_RETRIEVER_INFERENCE_CAPTURE_OPERATION
+              value: query
+            {{- end }}
             {{- if $internalAuth.enabled }}
             - name: NRL_INTERNAL_VDB_TOKEN
               valueFrom:

nemo_retriever/helm/values.yaml

@@ -538,6 +538,15 @@ serviceConfig:
     file: "/var/lib/nemo-retriever/retriever-service.log"
     format: "%(asctime)s | %(levelname)s | %(name)s | %(message)s"
 
+  # Administrator-owned persistence of outbound remote-NIM request bodies.
+  # Artifacts can contain document content; use an approved, writable location.
+  inferenceCapture:
+    enabled: false
+    storageUri: ""
+    failureMode: best_effort
+    operations: []
+    stages: []
+
   # External NIM endpoints. Used as-is when the operator NIM is disabled
   # or when the NIM Operator CRDs are absent. When the corresponding
   # `nimOperator.<key>.enabled` is true (and the CRDs exist), the