Skip to content

Require gfw-common>=0.16.1 for the create_view column-order fix - #144

Merged
tomaslink merged 1 commit into
mainfrom
hotfix-gfw-common-create-view-fix
Oct 2, 2026
Merged

tomaslink merged 1 commit into
mainfrom
hotfix-gfw-common-create-view-fix

Conversation

@tomaslink

@tomaslink tomaslink commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Depends on gfw-common#93: BigQueryHelper.create_view() could silently swap column values when its schema argument's field order didn't match the view query's actual output order.
  • This affected pipe-gaps' raw_gaps view directly: start_seg_id and start_msgid ended up swapped, because the view's SELECT list followed Gap's NamedTuple declaration order (queries/gaps.py) while schema followed gaps.json's order, and those two orders disagreed for this one pair.
  • Bumps gfw-common[bq,beam] from ~=0.16.0 to ~=0.16.1 and pipe-gaps to 0.14.1.

Blocked on release: gfw-common#93 is open but not yet merged/released. This PR is ready to go but can't actually install/pass CI until gfw-common 0.16.1 is published to PyPI.

Test plan

  • Full pipe-gaps test suite passes locally against the fixed gfw-common code (112 passed).
  • Re-run CI once gfw-common 0.16.1 is released and this branch can install the real dependency.
  • Once merged and released, production's gfw-int-ais-datalake.gaps_v1.raw_gaps view should be recreated (next scheduled pipeline run, or manually) to pick up the fix.

gfw-common 0.16.0's BigQueryHelper.create_view() could silently swap
column values when its schema argument's field order didn't match the
view query's actual output order (see GlobalFishingWatch/gfw-common#93).
This affected pipe-gaps' raw_gaps view, where start_seg_id and
start_msgid ended up swapped.
@tomaslink
tomaslink force-pushed the hotfix-gfw-common-create-view-fix branch from c7b48f6 to f5bf861 Compare October 2, 2026 04:44
@tomaslink tomaslink self-assigned this Oct 2, 2026
@tomaslink
tomaslink merged commit 2ddefe6 into main Oct 2, 2026
5 of 6 checks passed
@tomaslink
tomaslink deleted the hotfix-gfw-common-create-view-fix branch October 2, 2026 04:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant