[clampr]

Summary

This replaces the slow sqlite3mysql import from #518. Building stations.db and refreshing the inventory are now separate workflows. Importing into MySQL now happens on the server, with a PHP script (stations-import.php) that is not part of this repo.

sync-database.yml (on pushes to stations/*.json, or started manually)

1. Builds stations.db with meteo station build (meteostat-cli[contrib]; Python 3.12).
2. Copies the inventory table from the current stations.db into the new file, since the build creates a fresh file without it.
3. Uploads to FTP as stations.db.tmp and renames it to stations.db in the same call. Readers on the server never see a partial file.
4. Commits and pushes.

sync-inventory.yml (weekly, Mondays 02:00 UTC, or started manually)

• Replaces only the inventory table in the existing stations.db (mysql2sqlite -t inventory), then uploads the same way and commits.

Both workflows share one concurrency group, so they never run at the same time or push conflicting commits.

Removed: database.py and tables.sql. requirements.txt is back to mysql-to-sqlite3 and pandas.

Notes

• meteo station build skips stations that fail validation. It currently skips 49, mostly because two stations share an identifier. These stations will be missing from stations.db and from MySQL after the server import. Fix the data, or add --skip-validation.
• locations.csv.gz is no longer generated. Only database.py generated it.
• The FTP server must allow renaming onto an existing file. This was tested against vsftpd. Please check the first run.

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🔵 Needs a closer look

It reworks a production deployment pipeline that depends on unverifiable external pieces (the meteo station build CLI, FTP rename-onto-existing-file behavior, and a server-side PHP import webhook) that the author explicitly flags as untested.

Review effort: Balanced
Findings: None

What changed in this PR

This PR reworks how the published stations.db is produced and synced, replacing the previous sqlite3mysql-based MySQL export (from #518) with a simpler, split-workflow approach. Building the database from stations/*.json and refreshing the Meteostat inventory table are now decoupled into two workflows, and the actual MySQL import is delegated to a server-side PHP webhook outside this repo. The legacy JSON→SQLite conversion script and its SQL schema are removed in favor of the meteo station build CLI.

Changes:

• Rebuilt sync-database.yml to build stations.db via meteo station build, copy the existing inventory table into the fresh file, upload atomically over FTP (temp file + rename), commit/push, and trigger a server-side import webhook; trigger changed from a daily cron to pushes on stations/*.json.
• Added sync-inventory.yml (weekly) that replaces only the inventory table via mysql2sqlite and uploads the same way; both workflows share one stations-db concurrency group.
• Removed database.py, tables.sql, and requirements.txt (local SQLite build artifacts no longer needed).

File	Description
.github/​workflows/​sync-database.yml	Builds DB with meteo CLI, preserves inventory, atomic FTP upload, and server import webhook; new push trigger + concurrency group.
.github/​workflows/​sync-inventory.yml	New weekly workflow that refreshes only the inventory table from MySQL and uploads/commits.
database.py	Removed legacy JSON→SQLite conversion script (also generated locations.csv.gz).
tables.sql	Removed SQL schema used by the deleted database.py.
requirements.txt	Removed; dependencies are now installed inline per workflow.

I verified that no remaining code, workflow, or documentation references the removed database.py, tables.sql, or requirements.txt, that the inventory table is self-contained (no foreign keys) so the .dump inventory → pipe-into-fresh-DB step works, and that the FTP temp-upload-then-rename and shared concurrency group are internally consistent across both workflows. I did not find concrete code-level defects to comment on.

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.