Skip to content

Latest commit

 

History

4 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

spatial-fixture-gen

Generate synthetic geospatial test fixtures — vectors, rasters, and a catalogue of pathological edge cases — instead of committing sample data to your repository.

Most GIS projects end up with a tests/data/ directory that nobody dares to touch: a few hundred megabytes of shapefiles and GeoTIFFs, half of them added years ago to reproduce a bug whose details are long forgotten. The data is opaque, it bloats every clone, and it still doesn't cover the case that breaks you next.

This package replaces that directory with code. Every fixture is a pure function of its arguments — including its seed — so instead of storing a file you store the two lines that regenerate it. The interesting part is the edge-case registry: thirty-nine named, genuinely malformed fixtures (bowtie polygons, unclosed rings, antimeridian crossings, all-nodata rasters, rotated transforms, NaN coordinates) that your pipeline should survive, each one documented with what it tends to break.

Documentation and background articles: batch-processing.com.

Install

Not published to PyPI — clone and install from source:

git clone https://github.com/batch-processing-geospatial-cli-tools/spatial-fixture-gen.git
cd spatial-fixture-gen
uv sync                       # or: python -m venv .venv && .venv/bin/pip install -e ".[dev]"
uv run spatial-fixture-gen --help

Requires Python 3.11+. All dependencies (shapely, pyogrio, rasterio, pyproj, numpy, typer, rich) ship manylinux wheels, so there is no GDAL build step.

Usage

From the command line

# Vector layers — format inferred from the extension (.geojson, .gpkg, .shp, .fgb)
uv run spatial-fixture-gen points   --out points.gpkg   --count 5000 --bounds -180,-90,180,90
uv run spatial-fixture-gen polygons --out polys.fgb     --count 200 --vertices 12 --holes
uv run spatial-fixture-gen lines    --out lines.geojson --count 100 --crs EPSG:3857 --seed 7
uv run spatial-fixture-gen mixed    --out mixed.gpkg    --count 30

# Rasters — .cog.tif selects GDAL's COG driver
uv run spatial-fixture-gen raster --out dem.tif      --width 1024 --height 1024 \
                                  --dtype float32 --pattern gaussian --nodata -9999
uv run spatial-fixture-gen raster --out tiles.cog.tif --width 512 --height 512 --bands 3
$ uv run spatial-fixture-gen points --out points.gpkg --count 5000
wrote 5000 features to points.gpkg

The edge-case catalogue

$ uv run spatial-fixture-gen edge-cases list --kind vector
                                        Edge-case catalogue
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name                       ┃ kind   ┃ breaks                                                     ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ antimeridian-crossing      │ vector │ bbox computation, clipping and anything that assumes x     │
│                            │        │ increases westward                                         │
│ bowtie-polygon             │ vector │ shapely.is_valid, area calculations, GEOS overlay          │
│                            │        │ operations                                                 │
│ collinear-vertices         │ vector │ convex-hull and angle computations that divide by segment  │
│                            │        │ cross products                                             │
│ coords-outside-crs-bounds  │ vector │ pyproj transforms, which return inf rather than raising    │
│ crs-less-vector            │ vector │ transform pipelines that assume a source CRS is always     │
│                            │        │ present                                                    │
...
uv run spatial-fixture-gen edge-cases write bowtie-polygon --out bowtie.gpkg
uv run spatial-fixture-gen edge-cases write-all --out ./edge-cases/

A whole corpus with a manifest

uv run spatial-fixture-gen suite --out ./fixtures --preset full

That writes vector layers in three formats, rasters at two sizes, every writable edge case, and a manifest.json describing each file:

{
  "generator": "spatial-fixture-gen",
  "preset": "full",
  "seed": 0,
  "file_count": 68,
  "files": [
    {
      "path": "vector/points-250.gpkg",
      "category": "vector",
      "generator": "points",
      "feature_count": 250,
      "crs": "EPSG:4326",
      "bounds": [-10.0, -10.0, 10.0, 10.0],
      "seed": 0
    },
    {
      "path": "edge-cases/bowtie-polygon.gpkg",
      "category": "edge-case",
      "edge_case": "bowtie-polygon",
      "kind": "vector",
      "writable": true,
      "breaks": "shapely.is_valid, area calculations, GEOS overlay operations",
      "description": "A four-vertex ring whose edges cross, forming two lobes joined at a point."
    }
  ]
}

The manifest is what makes the corpus testable: your suite can iterate it and assert against declared expectations rather than hard-coded filenames.

From Python

from spatial_fixture_gen import points, polygons, raster, write_vector, write_raster

fc = polygons(50, bounds=(-1.0, -1.0, 1.0, 1.0), crs="EPSG:4326", seed=42, allow_holes=True)
write_vector(fc, "polys.gpkg")

write_raster(raster(width=256, height=256, bands=3, dtype="uint16", pattern="checkerboard"),
             "checker.tif")

Generators return GeoJSON-like dicts, so most assertions never need a file at all:

>>> fc = points(3, bounds=(0, 0, 1, 1), seed=1)
>>> fc["features"][0]["geometry"]
{'type': 'Point', 'coordinates': [0.511821625, 0.948649447]}

In your pytest suite

The package registers a pytest11 entry point, so installing it is enough — no conftest.py needed:

from spatial_fixture_gen import points, raster

def test_reader_counts_features(spatial_fixtures):
    path = spatial_fixtures.geojson(points(100, seed=1))
    assert my_reader(path).feature_count == 100

def test_survives_every_known_pathology(spatial_fixtures):
    for case in spatial_fixtures.edge_cases("vector"):
        if case.writable:
            my_pipeline(spatial_fixtures.edge_case(case.name))  # must not crash

Everything is written under pytest's tmp_path, so cleanup is automatic and two tests never collide on a filename. Available writers: geojson, gpkg, shapefile, flatgeobuf, geotiff, cog, edge_case, suite.

How it works

The determinism guarantee

The same seed produces byte-identical coordinates — in a fresh interpreter, on another machine, after a NumPy upgrade. Two things make that true:

  1. A stable bit-stream. Randomness comes from numpy.random.default_rng(seed) (PCG64). NumPy's stream-compatibility policy commits to that generator producing the same values across releases, which random.Random and the legacy numpy.random global state do not.
  2. Rounded ordinates. Every coordinate is rounded to 9 decimal places before it leaves a generator. Polygon vertices go through sin/cos, and different libm implementations disagree in the last bit or two; rounding erases that difference well above any precision a fixture needs (9 decimal degrees is about 0.1 mm).

The test suite asserts this across a subprocess boundary, not just within one process, because the interesting failure mode is process-global state.

Why raw WKB for the nasty cases

Shapely is the obvious way to build geometries, but it repairs exactly what this package wants to reproduce: it closes open rings, refuses M coordinates, and normalises degenerate input. So spatial_fixture_gen._wkb assembles the bytes directly — an unvalidating ISO WKB writer covering Point, LineString, Polygon and GeometryCollection with optional Z and M.

Those geometries reach write_vector through an escape hatch: a geometry dict of {"type": "WKB", "hex": "..."} is passed through verbatim instead of going via shapely. Fixtures stay JSON-serialisable and there is still only one write path.

The result is genuinely broken data. The unclosed-ring fixture, read back from the GeoPackage it was written to, makes GEOS raise IllegalArgumentException: Points of LinearRing do not form a closed linestring — which is the point. If your reader swallows that silently, you have a bug.

The registry is the test contract

Each edge case is registered with its name, kind, a one-line description, and what it tends to break. The test suite holds a table of pathology checkers keyed by name and asserts that the checker table and the registry have identical key sets, so adding an edge case without a test that proves the pathology is real fails CI. Every case is then parametrised three ways: the pathology holds, the fixture writes (or refuses honestly), and its metadata is well-formed.

zero-size-raster is the one case flagged writable=False. GDAL cannot create a 0×0 dataset, so the fixture exists in memory only; write-all and suite skip it and record it in the manifest with a null path rather than pretending it isn't there.

Layer-level CRS

RFC 7946 removed the CRS member from GeoJSON, but a fixture that cannot say "this is UTM 33N" is not much use. Collections carry the CRS under a private crs_ref key that write_vector passes to OGR and every GeoJSON reader ignores. The same mechanism carries an optional OGR layer geometry type, which is how the measured (M) fixtures keep their measure ordinate — GDAL only stores M when the layer itself is declared measured.

API reference

Vector generators

All return a GeoJSON-like FeatureCollection dict and accept bounds=(minx, miny, maxx, maxy), crs, and seed.

Function Extra arguments Notes
points(n, ...) — Uniform within bounds.
linestrings(n, ...) vertices=5 Random walk clamped to bounds.
polygons(n, ...) vertices=6, allow_holes=False Star-shaped rings; always valid.
mixed_collection(n, ...) — Cycles point / line / polygon.

write_vector(collection, path, layer=None, driver=None, geometry_type=None) writes to GeoJSON, GeoPackage, ESRI Shapefile or FlatGeobuf, inferring the driver from the extension. write_geojson_text(collection, path) serialises the dict with json instead, when you want exactly what you generated.

Raster generator

raster(width=64, height=64, bands=1, dtype="uint8", crs="EPSG:4326", transform=None, pattern="gradient", nodata=None, seed=0, bounds=(0, 0, 1, 1)) returns a RasterFixture (data, crs, transform, nodata, metadata, plus count/width/height/dtype properties). Patterns: gradient, checkerboard, gaussian, noise, constant.

write_raster(fixture, path, driver=None) writes GeoTIFF, or COG when the path ends in .cog.tif.

Edge cases

  • list_edge_cases(kind=None) -> tuple[EdgeCase, ...] — sorted; filter by "vector" or "raster".
  • edge_case(name) -> EdgeCase — raises UnknownEdgeCaseError with close-match suggestions.
  • write_edge_case(name, path) -> Path
  • write_all_edge_cases(directory, kind=None) -> dict[str, Path]

An EdgeCase has .name, .kind, .breaks, .description, .writable, .suffix and .build().

Suite

build_suite(directory, preset="small", seed=0, bounds=..., crs="EPSG:4326") -> dict writes the corpus and returns the manifest. small is a quick smoke corpus; full adds larger layers and the whole edge-case catalogue.

Errors

FixtureError for vector problems, RasterFixtureError for raster ones, UnknownEdgeCaseError for registry lookups. The CLI turns all of them into a one-line message on stderr and exit code 1; nothing escapes as a traceback.

Development

uv sync --extra dev
uv run --extra dev ruff check src tests
uv run --extra dev ruff format --check src tests
uv run --extra dev mypy
uv run --extra dev coverage run --source=src/spatial_fixture_gen -m pytest
uv run --extra dev coverage report

244 tests, 96 % statement coverage. Coverage is started before pytest rather than through pytest-cov, because the pytest11 entry point imports the package before a pytest plugin could begin measuring it.

CI runs the same steps on Python 3.11, 3.12 and 3.13.

Further reading

Background on the surrounding practices, from the guides this tooling grew out of:

License

MIT — see LICENSE.

About

Generate synthetic geospatial test fixtures — vectors, rasters and a catalogue of pathological edge cases

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages