Problem
CSV enrichment currently processes multiple books within a web Worker invocation. Worker exceeded CPU time limit has been observed in the PR preview and previously in production. Accumulated CPU usage on Workers Free is the suspected cause; the failing endpoint and CPU hotspot still need verification.
Follow up on #280 after #309 is merged. Implement this in a separate PR and task.
Proposed approach
- After persisting imported books and durable enrichment jobs, immediately publish dispatch messages to Cloudflare Queues. Do not wait for cron or depend on an open browser.
- Keep the Queue consumer lightweight: start a Workflow using a stable job/instance identity, then acknowledge successful dispatch. Handle duplicate delivery and already-created instances safely.
- Execute enrichment in a dedicated regular Worker with Workflows, using bounded per-book steps for metadata lookup, persistence and cover storage. Do not wrap the existing multi-book HTTP batch in one Workflow step.
- Keep D1 as the durable source of progress and R2 for cover bytes; return references rather than large binary step results. Have the UI poll persisted progress.
- Reuse existing ownership checks, leases, deduplication, canonical-book coordination, user-tag preservation and shared OpenLibrary rate limiting. Define one owner for processing retries so Workflow and existing job retries do not compete.
- Cover the gap between database persistence and Queue publication with durable reconciliation. Retain scheduled recovery as a dispatcher/reconciler rather than heavy processing in the web Worker.
- Preserve the self-hosted processing path behind the existing runtime abstraction and route/service/repository architecture.
Free-plan constraint
Workflows Free currently permits 10 ms of active CPU per step. Queue/Workflow dispatch does not itself remove CPU limits. Measure representative steps, keep the jobs entry point lightweight, and split further where needed. If a required operation cannot reliably fit, document the measured limitation and required plan/configuration rather than claiming Free compatibility.
References: Workflow limits, Queue limits.
Acceptance criteria
- CSV import dispatches enrichment immediately after persistence; work continues after navigation, reload or closing the browser.
- Duplicate messages, dispatch failures, retries and worker restarts neither lose jobs nor apply enrichment twice unsafely.
- UI progress and terminal states remain accurate, and enrichment cannot overwrite user tag edits or introduce suggested tags on library cards.
- Representative imports and individual processing steps are validated against the target plan's CPU limits. Logs identify import, dispatch and processing failures separately.
- Production and PR previews have isolated Queue/Workflow bindings and appropriate provisioning, deployment and preview cleanup. No production data is processed by previews.
- Self-hosted deployments retain working enrichment without Cloudflare bindings.
- Add regression coverage for dispatch/reconciliation, idempotency, failure recovery and progress; run lint, typecheck and tests.
Scope boundary
This issue covers enrichment dispatch and execution. If profiling shows CSV parsing/persistence itself exceeds the request budget, capture that separately for chunked imports staged in R2. Broader email, storage reconciliation and image-product migrations are outside this issue.
Problem
CSV enrichment currently processes multiple books within a web Worker invocation.
Worker exceeded CPU time limithas been observed in the PR preview and previously in production. Accumulated CPU usage on Workers Free is the suspected cause; the failing endpoint and CPU hotspot still need verification.Follow up on #280 after #309 is merged. Implement this in a separate PR and task.
Proposed approach
Free-plan constraint
Workflows Free currently permits 10 ms of active CPU per step. Queue/Workflow dispatch does not itself remove CPU limits. Measure representative steps, keep the jobs entry point lightweight, and split further where needed. If a required operation cannot reliably fit, document the measured limitation and required plan/configuration rather than claiming Free compatibility.
References: Workflow limits, Queue limits.
Acceptance criteria
Scope boundary
This issue covers enrichment dispatch and execution. If profiling shows CSV parsing/persistence itself exceeds the request budget, capture that separately for chunked imports staged in R2. Broader email, storage reconciliation and image-product migrations are outside this issue.