Skip to content

fix(bigquery): enable storage read api for appends/changes methods - #4824

Open
dtunikov wants to merge 4 commits into
bq/select-only-mirror-columnsfrom
bq/project-internal-col-names-to-enable-storage-read-api
Open

dtunikov wants to merge 4 commits into
bq/select-only-mirror-columnsfrom
bq/project-internal-col-names-to-enable-storage-read-api

Conversation

@dtunikov

@dtunikov dtunikov commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Currently appends/changes methods never use storage read api, because bigquery rejected it with:

InvalidArgument: Reading change stream columns is not supported:
_CHANGE_TIMESTAMP, _CHANGE_TYPE

This PR adds a workaround to project these columns to with PEERDB prefix to unblock storage read api usage.
It also removes ORDER BY _CHANGE_TIMESTAMP from the CHANGES method path to ensure that storage read api can be used.

Extend validation as well to perform a real Read from the storage read api stream.

@dtunikov
dtunikov requested a review from a team as a code owner September 18, 2026 14:34
@dtunikov
dtunikov requested a review from a team as a code owner September 18, 2026 14:36
@claude

claude Bot commented Sep 18, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

1 similar comment
@claude

claude Bot commented Sep 18, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant