Skip to content

feat(anc): direct HTTP download for hotfix with SHA-256 verification - #9233

Open
Abigail Liang (abigailliang-aks-sig-node) wants to merge 5 commits into
mainfrom
abigailliang/anc-hotfix-http-download
Open

feat(anc): direct HTTP download for hotfix with SHA-256 verification#9233
Abigail Liang (abigailliang-aks-sig-node) wants to merge 5 commits into
mainfrom
abigailliang/anc-hotfix-http-download

fix

a06a78b
Select commit
Loading
Failed to load commit list.
Azure Pipelines / Agentbaker GPU E2E failed Aug 18, 2026 in 30m 25s

Build #20260818.23 had test failures

Details

Tests

  • Failed: 3 (5.66%)
  • Passed: 50 (94.34%)
  • Other: 0 (0.00%)
  • Total: 53

Annotations

Check failure on line 1675 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Agentbaker GPU E2E

Build log #L1675

Script failed with exit code: 1

Check failure on line 1 in Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Agentbaker GPU E2E

Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU

Failed
Raw output
=== RUN   Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU
=== PAUSE Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU
=== CONT  Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU
    test_helpers.go:412: [5.286s] TAGS {Name:Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU ImageName:2404gen2containerd OS:ubuntu Arch:amd64 NetworkIsolated:false NonAnonymousACR:false GPU:true WASM:false Kata:false BootstrapTokenFallback:false KubeletCustomConfig:false Scriptless:false VHDCaching:false MockAzureChinaCloud:false RCV1PCertMode:false VMSeriesCoverageTest:false}
    test_helpers.go:222: [5.288s] → running scenario...
    cluster.go:79: [5.288s] → preparing cluster...
    cluster.go:319: [14.124s] → get or create cluster abe2e-kubenet-v5-6f4ab...
    cluster.go:327: [15.107s] ✓ get or create cluster abe2e-kubenet-v5-6f4ab done (1.0s)
    cluster.go:725: [15.210s] using shared bastion abe2e-shared-bastion in abe2e-westus3
    aks_model.go:316: [15.210s] → adding firewall rules...
    cluster.go:894: [15.482s] → setting up private DNS for API server...
    aks_model.go:362: [16.097s] Adding route "vnet-local" to AKS route table "aks-agentpool-19541795-routetable"
    cluster.go:933: [16.394s] private DNS zone "abe2e-kubenet-v5-wj74ke1x.hcp.westus3.azmk8s.io" already up to date
    cluster.go:934: [16.394s] ✓ setting up private DNS for API server done (0.9s)
    aks_model.go:362: [17.266s] Adding route "default-route-to-firewall" to AKS route table "aks-agentpool-19541795-routetable"
    aks_model.go:373: [17.857s] Successfully added firewall routes to AKS route table "aks-agentpool-19541795-routetable"
    aks_model.go:374: [17.857s] ✓ adding firewall rules done (2.6s)
    cluster.go:749: [17.857s] → collecting garbage VMSS...
    cluster.go:821: [17.925s] → collecting garbage K8s nodes...
    cluster.go:862: [18.049s] ✓ collecting garbage K8s nodes done (0.1s)
    cluster.go:813: [18.049s] ✓ collecting garbage VMSS done (0.2s)
    kube.go:397: [18.049s] Creating daemonset debug-mariner-tolerated with image mcr.microsoft.com/cbl-mariner/base/core:2.0
    kube.go:397: [18.258s] Creating daemonset debugnonhost-mariner-tolerated with image mcr.microsoft.com/cbl-mariner/base/core:2.0
    kube.go:580: [18.342s] Creating proxy daemonset e2e-proxy with image mcr.microsoft.com/cbl-mariner/base/python:3
    cluster.go:202: [18.539s] ✓ preparing cluster done (13.3s)
    test_helpers.go:239: [18.539s] using cluster abe2e-kubenet-v5-6f4ab in rg=abe2e-westus3 sub=8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8
    test_helpers.go:240: [18.539s] portal: https://portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/abe2e-westus3/providers/Microsoft.ContainerService/managedClusters/abe2e-kubenet-v5-6f4ab/overview
    test_helpers.go:272: [18.542s] → preparing AKS node...
    vmss.go:369: [29.534s] → creating VMSS 1gre-2026-08-18-ubuntu2404nvidiadevicepluginrunningmigmul...
    vmss.go:263: [30.034s] VMSS portal link: https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/MC_abe2e-westus3_abe2e-kubenet-v5-6f4ab_westus3/providers/Microsoft.Compute/virtualMachineScaleSets/1gre-2026-08-18-ubuntu2404nvidiadevicepluginrunningmigmul/overview
    vmss.go:269: [30.034s] Managed cluster portal link: https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/MC_abe2e-westus3_abe2e-kubenet-v5-6f4ab_westus3/providers/Microsoft.ContainerService/managedClusters/abe2e-kubenet-v5-6f4ab/overview
2026/08/18 21:07:09 Using VM extension version 1.492 for extension type Compute.AKS.Linux.AKSNode in region westus3
    vmss.go:401: [48.707s] VM will be automatically deleted after the test
... [The stack trace has been truncated as it exceeded the maximum allowed size. Please refer to the complete log available in the Test Run attachments for full details.]

Check failure on line 1 in Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Agentbaker GPU E2E

Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU

Failed
Raw output
=== RUN   Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU
=== PAUSE Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU
=== CONT  Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU
    azure.go:573: [0.000s] Looking up images in https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/c4c3550e-a965-4993-a50c-628fd38cd3e1/resourceGroups/aksvhdtestbuildrg/providers/Microsoft.Compute/galleries/PackerSigGalleryEastUS/images/aks-ubuntu-containerd-24.04-gen2/overview
    azure.go:662: [2.340s] Image version /subscriptions/c4c3550e-a965-4993-a50c-628fd38cd3e1/resourceGroups/aksvhdtestbuildrg/providers/Microsoft.Compute/galleries/PackerSigGalleryEastUS/images/2404gen2containerd/versions/1.1787040588.3463 is already in region westus3
    vhd.go:361: [2.340s] got version by tag branch=refs/heads/main: https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/c4c3550e-a965-4993-a50c-628fd38cd3e1/resourceGroups/aksvhdtestbuildrg/providers/Microsoft.Compute/galleries/PackerSigGalleryEastUS/images/aks-ubuntu-containerd-24.04-gen2/versions/1.1787040588.3463/overview
    test_helpers.go:412: [2.340s] TAGS {Name:Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU ImageName:2404gen2containerd OS:ubuntu Arch:amd64 NetworkIsolated:false NonAnonymousACR:false GPU:true WASM:false Kata:false BootstrapTokenFallback:false KubeletCustomConfig:false Scriptless:false VHDCaching:false MockAzureChinaCloud:false RCV1PCertMode:false VMSeriesCoverageTest:false}
    test_helpers.go:222: [11.418s] → running scenario...
    cluster.go:79: [11.418s] → preparing cluster...
    shared_infra.go:51: [11.418s] → ensuring shared infrastructure...
    shared_infra.go:80: [17.135s] ✓ ensuring shared infrastructure done (5.7s)
    cluster.go:319: [17.443s] → get or create cluster abe2e-kubenet-v5-6f4ab...
    cluster.go:327: [18.455s] ✓ get or create cluster abe2e-kubenet-v5-6f4ab done (1.0s)
    cluster.go:725: [18.513s] using shared bastion abe2e-shared-bastion in abe2e-westus3
    aks_model.go:316: [18.513s] → adding firewall rules...
    cluster.go:894: [18.812s] → setting up private DNS for API server...
    aks_model.go:362: [19.192s] Adding route "vnet-local" to AKS route table "aks-agentpool-19541795-routetable"
    cluster.go:933: [19.761s] private DNS zone "abe2e-kubenet-v5-wj74ke1x.hcp.westus3.azmk8s.io" already up to date
    cluster.go:934: [19.761s] ✓ setting up private DNS for API server done (0.9s)
    aks_model.go:362: [20.693s] Adding route "default-route-to-firewall" to AKS route table "aks-agentpool-19541795-routetable"
    aks_model.go:373: [21.153s] Successfully added firewall routes to AKS route table "aks-agentpool-19541795-routetable"
    aks_model.go:374: [21.153s] ✓ adding firewall rules done (2.6s)
    cluster.go:749: [21.153s] → collecting garbage VMSS...
    cluster.go:821: [21.216s] → collecting garbage K8s nodes...
    cluster.go:862: [21.311s] ✓ collecting garbage K8s nodes done (0.1s)
    cluster.go:813: [21.311s] ✓ collecting garbage VMSS done (0.2s)
    kube.go:397: [21.311s] Creating daemonset debug-mariner-tolerated with image mcr.microsoft.com/cbl-mariner/base/core:2.0
    kube.go:397: [21.498s] Creating daemonset debugnonhost-mariner-tolerated with image mcr.microsoft.com/cbl-mariner/base/core:2.0
    kube.go:580: [21.568s] Creating proxy daemonset e2e-proxy with image mcr.microsoft.com/cbl-mariner/base/python:3
    cluster.go:202: [21.769s] ✓ preparing cluster done (10.4s)
    test_helpers.go:239: [21.769s] using cluster abe2e-kubenet-v5-6f4ab in rg=abe2e-westus3 sub=8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8
    test_helpers.go:240: [21.769s] portal: https://portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/abe2e-westus3/providers/Microsoft.
... [The stack trace has been truncated as it exceeded the maximum allowed size. Please refer to the complete log available in the Test Run attachments for full details.]

Check failure on line 1 in Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Agentbaker GPU E2E

Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU

Failed
Raw output
=== RUN   Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU
=== PAUSE Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU
=== CONT  Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU
    azure.go:573: [0.000s] Looking up images in https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/c4c3550e-a965-4993-a50c-628fd38cd3e1/resourceGroups/aksvhdtestbuildrg/providers/Microsoft.Compute/galleries/PackerSigGalleryEastUS/images/aks-ubuntu-containerd-24.04-gen2/overview
    azure.go:662: [2.257s] Image version /subscriptions/c4c3550e-a965-4993-a50c-628fd38cd3e1/resourceGroups/aksvhdtestbuildrg/providers/Microsoft.Compute/galleries/PackerSigGalleryEastUS/images/2404gen2containerd/versions/1.1787040588.3463 is already in region westus3
    vhd.go:361: [2.257s] got version by tag branch=refs/heads/main: https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/c4c3550e-a965-4993-a50c-628fd38cd3e1/resourceGroups/aksvhdtestbuildrg/providers/Microsoft.Compute/galleries/PackerSigGalleryEastUS/images/aks-ubuntu-containerd-24.04-gen2/versions/1.1787040588.3463/overview
    test_helpers.go:412: [2.257s] TAGS {Name:Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU ImageName:2404gen2containerd OS:ubuntu Arch:amd64 NetworkIsolated:false NonAnonymousACR:false GPU:true WASM:false Kata:false BootstrapTokenFallback:false KubeletCustomConfig:false Scriptless:false VHDCaching:false MockAzureChinaCloud:false RCV1PCertMode:false VMSeriesCoverageTest:false}
    test_helpers.go:222: [10.782s] → running scenario...
    cluster.go:79: [10.782s] → preparing cluster...
    shared_infra.go:51: [10.782s] → ensuring shared infrastructure...
    shared_infra.go:80: [15.640s] ✓ ensuring shared infrastructure done (4.9s)
    cluster.go:319: [15.839s] → get or create cluster abe2e-kubenet-v5-6f4ab...
    cluster.go:327: [16.795s] ✓ get or create cluster abe2e-kubenet-v5-6f4ab done (1.0s)
    cluster.go:894: [17.179s] → setting up private DNS for API server...
    cluster.go:725: [17.647s] using shared bastion abe2e-shared-bastion in abe2e-westus3
    aks_model.go:316: [17.647s] → adding firewall rules...
    cluster.go:933: [18.147s] private DNS zone "abe2e-kubenet-v5-wj74ke1x.hcp.westus3.azmk8s.io" already up to date
    cluster.go:934: [18.147s] ✓ setting up private DNS for API server done (1.0s)
    aks_model.go:362: [18.201s] Adding route "vnet-local" to AKS route table "aks-agentpool-19541795-routetable"
    aks_model.go:362: [19.131s] Adding route "default-route-to-firewall" to AKS route table "aks-agentpool-19541795-routetable"
    aks_model.go:373: [19.436s] Successfully added firewall routes to AKS route table "aks-agentpool-19541795-routetable"
    aks_model.go:374: [19.436s] ✓ adding firewall rules done (1.8s)
    cluster.go:749: [19.436s] → collecting garbage VMSS...
    cluster.go:821: [19.497s] → collecting garbage K8s nodes...
    cluster.go:862: [19.616s] ✓ collecting garbage K8s nodes done (0.1s)
    cluster.go:813: [19.616s] ✓ collecting garbage VMSS done (0.2s)
    kube.go:397: [19.616s] Creating daemonset debug-mariner-tolerated with image mcr.microsoft.com/cbl-mariner/base/core:2.0
    kube.go:397: [19.812s] Creating daemonset debugnonhost-mariner-tolerated with image mcr.microsoft.com/cbl-mariner/base/core:2.0
    kube.go:580: [19.882s] Creating proxy daemonset e2e-proxy with image mcr.microsoft.com/cbl-mariner/base/python:3
    cluster.go:202: [20.072s] ✓ preparing cluster done (9.3s)
    test_helpers.go:239: [20.072s] using cluster abe2e-kubenet-v5-6f4ab in rg=abe2e-westus3 sub=8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8
    test_helpers.go:240: [20.072s] portal: https://portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/abe2e-westus3/providers/Microsoft.C
... [The stack trace has been truncated as it exceeded the maximum allowed size. Please refer to the complete log available in the Test Run attachments for full details.]