[Feature] [Config] Add Azure Key Vault ConfigShade support#10962
[Feature] [Config] Add Azure Key Vault ConfigShade support#10962aakamara23 wants to merge 4 commits into
Conversation
DanielLeens
left a comment
There was a problem hiding this comment.
Thanks for adding a real external secret-manager direction here. I reviewed the latest head from the actual starter/config loading path, and I see two blockers before this should merge.
Runtime chain I checked:
Job startup / config loading
-> ConfigShadeUtils.decryptConfig(config)
-> processConfig(identifier, config, true, props)
-> CONFIG_SHADES.getOrDefault("azure-kv", ...)
-> configShade.open(props)
Azure Key Vault implementation
-> AzureKeyVaultConfigShade.open(props)
-> read vault.url
-> build SecretClient with DefaultAzureCredential
-> decrypt("${keyvault:azure:...}")
-> trim to last path segment
-> secretClient.getSecret(secretName).getValue()
Key files:
seatunnel-core/seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/utils/AzureKeyVaultConfigShade.java:1-62seatunnel-core/seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/utils/ConfigShadeUtils.java:61-69,166-170seatunnel-core/seatunnel-core-starter/pom.xml:70-80
Issue 1: the new Java file is missing the ASF license header
- Location:
seatunnel-core/seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/utils/AzureKeyVaultConfigShade.java:1 - Why this is blocking:
this is a brand new source file, and it currently starts directly withpackage .... The fork CI is already red onRun / License header, and the failing file reported there is exactly this one. - Severity: High
Issue 2: the Azure provider is wired directly into seatunnel-core-starter, so every starter distribution now pulls Azure SDKs whether the user needs them or not
- Location:
seatunnel-core/seatunnel-core-starter/pom.xml:70 - Why this is a blocker:
ConfigShadeis SPI-based already, so this is naturally extensible. But the current change putsazure-security-keyvault-secretsandazure-identitystraight into the core starter, which expands the default runtime footprint and dependency surface for every user, not only Azure users. - Better direction:
move the Azure Key Vault implementation into a dedicated optional module and let the existingServiceLoadermechanism discover it when present. - Severity: High
Issue 3: docs and regression coverage are not updated for a new built-in azure-kv capability
- Location:
docs/en/introduction/configuration/config-encryption-decryption.md:9 - Why it matters:
the current docs still describe only base64 and custom user implementations. There is no built-inazure-kvusage contract, novault.urlexplanation, and no test coverage for the actual new provider path. - Severity: Medium
Compatibility:
- Existing base64/default behavior remains intact.
- The main compatibility concern is packaging: the latest head changes the starter dependency footprint for everyone.
Tests / CI:
- I did not run this locally.
- The current
Buildfailure is directly related to this PR:Run / License headerfails and reportsAzureKeyVaultConfigShade.javaas the only invalid file. - This is not unrelated CI noise; it is a real branch-local blocker.
Merge conclusion: can merge after fixes
- Blocking items
- Issue 1 must be fixed first because the branch fails the ASF license header check.
- Issue 2 should be resolved before merge because the current packaging choice is too heavy for
seatunnel-core-starter.
- Suggested follow-up
- Add the missing docs and at least a minimal regression test for the new
azure-kvpath from Issue 3.
Overall, the feature idea is useful. The current concern is not Azure itself, but how it is being shipped: we should keep the core starter lean and keep the new provider properly documented and testable.
This PR is for issue #10309.
I added initial Azure Key Vault support to SeaTunnel using the existing ConfigShade framework.
azure-kvConfigShade providervault.urlThe goal is to allow users to store sensitive values in Azure Key Vault instead of directly in configuration files.
Does this PR introduce any user-facing change?
Yes.
Users can now configure
shade.identifier = "azure-kv"and reference secrets stored in Azure Key Vault.How was this patch tested?
mvn -pl seatunnel-core/seatunnel-core-starter test -DskipITs -DskipShade -Dspotless.check.skip=true -U