gfw-common

Common place for GFW reusable Python components.

Introduction

The following table shows a summary of the current supported features:

Module/Package	Description
gfw.common.beam	Common utilities and wrappers for Apache Beam pipelines.
gfw.common.cli	Lightweight framework around argparse for building CLIs more easily.
gfw.common.bigquery	Common utilities and wrappers for BigQuery interactions.
gfw.common.datetime	Simple helper functions around stdlib datetime module.
gfw.common.decorators.py	Basic function decorators.
gfw.common.dictionaries.py	Simple helper functions for dictionary manipulation.
gfw.common.io.py	Basic IO functions.
gfw.common.iterables.py	Iterables utilities.
gfw.common.logging.py	Basic logging configuration.
gfw.common.serialization.py	Basic serialization utilities.

For more information and API reference, check the documentation pages.

Installation

The package offers extras to avoid installing unnecessary dependencies:

• beam: includes dependencies for the gfw.common.beam package.
• bq: includes dependencies for BigQuery utilities.

For a default installation without extras, run:

pip install gfw-common

To install all extras, run:

pip install gfw-common[bq,beam]

Usage

You can see examples in the examples folder.

How to Contribute

Preparing the environment

First, clone the repository.

git clone https://github.com/GlobalFishingWatch/gfw-common.git

Install uv for faster installs:

./install-uv.sh

Create virtual environment and activate it:

make venv
source .venv/bin/activate

Install pre-commit hooks for local development:

make install-pre-commit

Install dependencies and package in editable mode for local development:

make install

Make sure you can run unit tests:

make test

Development Workflow

Regarding the git workflow, we follow GitHub Flow. See GITHUB-FLOW.md for a quick summary.

Try to write good commit messages. See How to Write a Git Commit Message guide for details.

The pre-commit hooks will take care of validating your code before a commit in terms of PEP8 standards, type-checking, miss-pellings, missing documentation, etc. If you want/need to do it manually, you have commands in the Makefile. To see options, type make.

How to Release

Creating a tag will automatically trigger a GitHub Action (release.yaml) to publish the package to PyPI. The tag must match the version declared in pyproject.toml; this will be validated by the action. The tag name must follow the format vX.Y.Z.

Note

In this context, X, Y and Z refer to MAJOR, MINOR and PATCH of Semantic Versioning.