Forward catalog static FileIO properties to server-side FileIO#4696
Forward catalog static FileIO properties to server-side FileIO#4696jbonofre wants to merge 1 commit into
Conversation
`IcebergCatalog.loadFileIOForTableLike` only passed `tableDefaultProperties` (the `table-default.*`-prefixed subset of catalog properties) to the `FileIOFactory`. For S3-compatible storage configured with `stsUnavailable: true`, catalogs typically carry static credentials (`s3.access-key-id` / `s3.secret-access-key`) as catalog properties. These were dropped before reaching `S3FileIO`, which then fell back to the AWS default credentials provider chain and failed when no chain credential was available. Merge `catalogProperties` as the base layer under `tableProperties`. `DefaultFileIOFactory` still overlays `StorageAccessConfig` last, so STS-vended subscoped credentials continue to take precedence over static catalog credentials when STS is available.
Isn't this a bad practice? In the REST Federation doc page, we say:
|
|
I agree with @adutra 's comment above that passwords should not be stored in catalog properties. I believe general-purpose Catalog properties should probably not be forwarded to clients automatically. That will create scope confusion between the Server and the Client. If a particular property has a reason for being forwarded, it should be ok on a case-by-case basis (as for |
IcebergCatalog.loadFileIOForTableLikeonly passedtableDefaultProperties(thetable-default.*-prefixed subset of catalog properties) to theFileIOFactory. For S3-compatible storage configured withstsUnavailable: true, catalogs typically carry static credentials (s3.access-key-id/s3.secret-access-key) as catalog properties. These were dropped before reachingS3FileIO, which then fell back to the AWS default credentials provider chain and failed when no chain credential was available.Merge
catalogPropertiesas the base layer undertableProperties.DefaultFileIOFactorystill overlaysStorageAccessConfiglast, so STS-vended subscoped credentials continue to take precedence over static catalog credentials when STS is available.Checklist
CHANGELOG.md(if needed)site/content/in-dev/unreleased(if needed)