Skip to content

OCI chart pulls fail when default keychain credentials return Docker identity tokens #335

Description

@mateipinzaru-parloa

What happened?

A private OCI chart pull can fail with 401 unauthorized when provider-helm relies on the default credential chain instead of chart.pullSecretRef, even though the provider runtime identity is able to resolve registry credentials.

This affects Azure Container Registry workload identity because the ACR credential helper returns Docker identity-token credentials: username <token> plus an ACR refresh token. provider-helm does not preserve that identity token before handing credentials to Helm, so the OCI pull falls back to anonymous/no usable credentials.

Example failure:

failed to install release: failed to pull chart: failed to perform "FetchReference" on source: GET "https://example.azurecr.io/v2/helm/private/my-chart/manifests/1.0.0": GET "https://example.azurecr.io/oauth2/token?scope=repository%3Ahelm%2Fprivate%2Fmy-chart%3Apull&service=example.azurecr.io": response status code 401: unauthorized: authentication required

Expected behavior

When the default keychain returns an identity token, provider-helm should preserve it and pass it to Helm/ORAS as an OCI registry refresh token. Registries that already materialize credentials as username plus password/access token should continue to work through the existing username/password path.

The bug is in the credential data shape passed between the default keychain and Helm:

The result is that AuthConfig.IdentityToken is resolved successfully by the default keychain, then discarded before Helm pulls the chart. This is not limited to ACR as a data-shape issue; any Docker credential helper that returns identity-token credentials can hit the same drop.

How can we reproduce it?

  1. Run provider-helm v1.2.0, or current main at 9e411cf68a2d5f39c96ea73f3f07838f97151e47, with Azure Workload Identity environment variables available to the provider runtime.
  2. Grant that identity pull access to a private Azure Container Registry OCI chart repository.
  3. Create a Release that references the private ACR OCI chart without chart.pullSecretRef, for example:
apiVersion: helm.m.crossplane.io/v1beta1
kind: Release
metadata:
  name: private-chart
  namespace: default
spec:
  forProvider:
    namespace: default
    chart:
      repository: oci://example.azurecr.io/helm/private
      name: my-chart
      version: 1.0.0
  providerConfigRef:
    name: default
    kind: ClusterProviderConfig
  1. Observe that the chart pull fails with ACR 401 unauthorized.

The default keychain can resolve a token through the ACR helper, but the token is stored in AuthConfig.IdentityToken and discarded before Helm pulls the chart.

What environment did it happen in?

Provider Helm version: v1.2.0

Code path also present on main at 9e411cf68a2d5f39c96ea73f3f07838f97151e47.

Registry: private Azure Container Registry OCI chart repository.

Authentication path: provider runtime Azure Workload Identity through the default credential chain, without chart.pullSecretRef.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions