Skip to content

chore: update NVIDIA GRID driver to 570.237 (vGPU18.8) - #178

Closed
Ganeshkumar Ashokavardhanan (ganeshkumarashok) wants to merge 1 commit into
Azure:mainfrom
ganeshkumarashok:chore/grid-driver-570.237
Closed

chore: update NVIDIA GRID driver to 570.237 (vGPU18.8)#178
Ganeshkumar Ashokavardhanan (ganeshkumarashok) wants to merge 1 commit into
Azure:mainfrom
ganeshkumarashok:chore/grid-driver-570.237

Conversation

@ganeshkumarashok

Copy link
Copy Markdown
Collaborator

What

Bumps the GRID driver (used by NVadsA10_v5 / A10 SKUs) from 570.211.01 → 570.237, and points url at the new Microsoft-hosted runfile.

grid:
  version: "570.237"
  url: "https://download.microsoft.com/download/5e213ec5-834f-4b0a-87f7-772751353b06/NVIDIA-Linux-x86_64-570.237-grid-azure.run"

Why the version has only two components

570.237 looks wrong next to the usual three-component GRID versions (570.211.01, 595.58.03), so I verified it against the runfile itself rather than trusting the filename. The self-extracting header declares:

label="NVIDIA Accelerated Graphics Driver for Linux-x86_64 570.237"
version_string=570.237
targetdir=NVIDIA-Linux-x86_64-570.237-grid-azure

So the two-component form is how NVIDIA versions this particular vGPU18.8 build — not a truncated value. I added a short comment in driver_config.yml recording this so the next person doesn't "fix" it.

Validation

  • Source of truth — the Azure N-series Linux driver setup docs GRID table now lists NVadsA10_v5 → vGPU18.8 → NVIDIA-Linux-x86_64-570.237-grid-azure.run with this exact download URL. (The step-by-step wget snippet further down that page still shows 570.211.01 — stale docs, the table is the updated one.)
  • URL reachable — returns HTTP 200.
  • Name derivation is consistentdownload.sh and install.sh build RUNFILE="NVIDIA-Linux-x86_64-${DRIVER_VERSION}-grid-azure". With DRIVER_VERSION=570.237 that matches both the URL basename (so the mv ${RUNFILE}.run after curl -O succeeds) and the runfile's targetdir (so /opt/gpu/${RUNFILE}/nvidia-installer exists after extraction).
  • No 3-component assumptions anywhereDRIVER_VERSION is only ever used as an opaque string: runfile name, the DKMS marker driver_version= field and its string equality check in baked_marker_matches, the buildx cache key, and the image tag. paulhatch/semantic-version's version_format receives it as literal text, so the tag becomes 570.237-<timestamp>. fabricmanager version interpolation is CUDA-only and unaffected.
  • YAML parses and other entries (cuda, cuda_lts, grid_v20) are untouched.

Heads-up: the daily auto-update may try to revert this

update_grid_driver.yaml runs auto_update.py, which reads the GRID "Latest" entry from azhpc-extensions/Nvidia-GPU-Linux-Resources.json. That feed still reports 570.211.01, so the nightly job will likely open a PR moving grid back to 570.211.01 until the feed catches up. It opens a PR rather than pushing directly, so nothing reverts silently — just don't merge that one blindly.

Bump the GRID (NVadsA10_v5) driver from 570.211.01 to 570.237, matching the
vGPU18.8 build Microsoft now publishes on the Azure N-series driver setup docs.

Note the version is two-component (570.237) rather than the usual
three-component form -- this is how NVIDIA versions this build. Confirmed
against the runfile header, which declares:

  version_string=570.237
  targetdir=NVIDIA-Linux-x86_64-570.237-grid-azure

so the RUNFILE name derived from DRIVER_VERSION in download.sh and install.sh
still resolves correctly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e525ff6d-7072-4f71-92b9-4d5a87808b20
@ganeshkumarashok

Copy link
Copy Markdown
Collaborator Author

Heads-up on the red CI: the failures are pre-existing and unrelated to this change.

Every build job fails (including grid_v20, cuda, cuda_lts — none of which this PR touches) because fork PRs don't receive repo secrets, so REGISTRY_SERVER is empty and the tag becomes invalid:

+ docker buildx build ... -t /public/aks/aks-gpu-grid-v20:595.58.03-20260817182823
ERROR: failed to build: invalid tag "/public/aks/aks-gpu-grid-v20:...": invalid reference format

That's exactly what #173 fixes. The grid job itself was cancelled by matrix fail-fast from its sibling, not failed on its own.

What did pass / was verified:

  • lint-and-test ✅ and license/cla
  • The Load GRID config step ran and accepted the new values before cancellation — including main's version-format guard ^[0-9]+(\.[0-9]+)+$, which the two-component 570.237 satisfies:
    GRID_VERSION=570.237
    GRID_URL=https://download.microsoft.com/download/5e213ec5-834f-4b0a-87f7-772751353b06/NVIDIA-Linux-x86_64-570.237-grid-azure.run
    
  • I ran the equivalent build locally to cover what CI couldn't:
    docker buildx build --platform linux/amd64 \
      --build-arg DRIVER_URL=<new url> --build-arg DRIVER_KIND=grid --build-arg DRIVER_VERSION=570.237 .
    
    It completed successfully, and inside the resulting image:
    /opt/gpu/NVIDIA-Linux-x86_64-570.237-grid-azure/nvidia-installer
    .manifest -> 570.237
    libnvidia-ml.so.570.237, nvidia-gridd
    
    confirming the download, the mv ${RUNFILE}.run, and the extraction all resolve correctly from DRIVER_VERSION=570.237.

@ganeshkumarashok

Copy link
Copy Markdown
Collaborator Author

Superseded by #179 — identical change, reopened from a branch on this repo instead of my fork so CI gets repo secrets and can actually run the builds. GitHub can't repoint an existing PR's head repository, so this needed a new PR rather than an update to this one.

Closing in favour of #179.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants