Skip to content

fix(ci): bump db-dump so the crates index accepts tar_sha256 - #321

Merged
Folyd merged 1 commit into
masterfrom
cursor/fix-crates-index-db-dump-034d
Oct 29, 2026
Merged

Folyd merged 1 commit into
masterfrom
cursor/fix-crates-index-db-dump-034d

Conversation

@Folyd

@Folyd Folyd commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

The scheduled crates-index workflow has been failing in ./scripts/deploy-crates-index.sh on every daily run (latest: https://github.com/huhu/rust-search-extension/actions/runs/40390726860). db-dump 0.7.x refuses the current https://static.crates.io/db-dump.tar.gz:

db_dump::Error("versions.csv: CSV deserialize error: record 1 (line: 2, byte: 258): unknown field `tar_sha256`, expected one of `id`, `crate_id`, `num`, ...")

crates.io replaced versions.checksum with versions.tar_sha256 (and added zip_sha256 / zip_json_sha256). db-dump 0.7.x deserializes versions.csv with deny_unknown_fields and still expects checksum, so the first data row fails. The same runs also logged non-fatal unimplemented lines for oauth_github.csv and reserved_usernames.csv.

db-dump 0.8.0 (2026-07-03) learned tar_sha256 plus those two tables. 0.10.0 (2026-10-12) is the current release and matches the public columns in crates.io's dump config, including reserved_crate_names.csv. It requires Rust 1.88 / edition 2024. The failing runner image (ubuntu-24.04 20261012.334.1) already ships Rust 1.99.0, so the workflow does not need a toolchain pin.

rust/src/tasks/crates.rs did not need changes. It still only uses crate_downloads.crate_id / downloads, versions.crate_id / created_at / num, and crates.id / name / description, which are unchanged.

Cargo.lock stays gitignored, which is how this repo already resolves dependencies in CI. db-dump = "0.10.0" is the same caret-style requirement as the old 0.7.1 pin, so the workflow resolves 0.10.x.

Verification

Built with Rust 1.99.1 (cargo build) and ran the crates subcommand against the live dump (last-modified 2026-10-27, 2,086,809,539 bytes):

target/debug/rust-search-extension crates -p /tmp/db-dump.tar.gz -d /tmp/index.js

That is the same arguments as ./scripts/deploy-crates-index.sh (cargo run --release crates -p ... -d ...). Exit 0 in 195s. No unimplemented CSV warnings. Wrote /tmp/index.js (1,442,976 bytes, about 20,985 crates, same shape as extension/index/crates.js). Spot checks from the generated index: serde 1.0.229, tokio 1.54.1, regex 1.13.1, clap 4.6.7, axum 0.8.9.

The upload half of deploy-crates-index.sh was not run; it force-pushes the now branch and needs GITHUB_TOKEN.

Open in Web Open in Cursor 

crates.io versions.csv replaced checksum with tar_sha256, and db-dump
0.7.x rejects that column. 0.10.0 parses the current dump; the loader
call sites are unchanged.

Co-authored-by: Folyd <Folyd@users.noreply.github.com>
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying rust-search-extension with  Cloudflare Pages  Cloudflare Pages

Latest commit: c2f325d
Status: ✅  Deploy successful!
Preview URL: https://0f677850.rust-search-extension.pages.dev
Branch Preview URL: https://cursor-fix-crates-index-db-d.rust-search-extension.pages.dev

View logs

@Folyd
Folyd merged commit 1163925 into master Oct 29, 2026
4 checks passed
@Folyd
Folyd deleted the cursor/fix-crates-index-db-dump-034d branch October 29, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants