Skip to content

tracking(storage): task storage visibility, reclamation and scoped cleanup #5825

Description

@liugddx

Step 1 of the task lifecycle plan agreed in #5776. It makes local storage visible, reclaims space that deletion already frees logically, and lets users clean up archived tasks by scope. It adds no automatic archiving or deletion; those are later, opt-in steps tracked in #5776. The Host computes and executes; Desktop only presents.

Baseline: main@0323714.

Current state

  • Nothing reports how much space Maka uses, or which tasks use it.
  • runtime.sqlite never shrinks.
    • It is WAL with auto_vacuum=NONE (sqlite-runtime-schema.ts).
    • Nothing calls wal_checkpoint, sets journal_size_limit, or runs VACUUM.
    • Deleted sessions leave free pages that SQLite reuses but never returns to the OS.
  • context-offload.sqlite is set up for reclamation but never reclaims.
    • It is created with auto_vacuum=INCREMENTAL (sqlite-context-offload-schema.ts).
    • No code runs incremental_vacuum.
  • Sessions have no archive time. session_metadata and SessionHeader have no archive time, and committed_at is overwritten on every metadata update, so it can't serve as one.
  • "Delete all archived" runs in the renderer.
    • It calls session.remove once per session, serially (session-row-actions.ts).
    • Its confirmation shows a count, not a preview of what will be removed.
  • fix(desktop): managed artifact previews outlive deletion and share one global quota #5341 is an in-memory preview lease that outlives deletion, not disk residue. fix(desktop): invalidate artifact previews after deletion #5394 fixes it at the Host retirement hook, and this issue depends on that fix rather than redoing it.

Plan: three PRs, each usable on its own

M1: storage visibility (read-only)

  • New Host query storage.usage.query, query mode, Ready only.
  • Per-task sizes are computed on demand for a bounded list of session IDs (the visible page) and cached in memory.
  • Desktop presents the results.
    • The Data page gets a "Storage" group with a breakdown and the reclaimable space.
    • The Tasks page shows a size on each row.
  • Epoch bump.

M2: reclamation

  • A bounded incremental_vacuum lane for context-offload.sqlite in the existing HostStorageMaintenance loop.
  • runtime.sqlite conversion to auto_vacuum=INCREMENTAL. It needs one full VACUUM, which needs roughly 2x the database size in free disk, so it runs only on explicit user request ("Compact database"). The Host persists the request and runs it at the next start, before Ready, next to migrations, after a free-disk check.
  • After conversion, bounded incremental vacuum plus a passive checkpoint run in the maintenance loop.
  • The Data page reports what was actually freed.
  • Epoch bump.

M3: scoped cleanup

  • Durable archived_at.
    • It is set when a session is archived and cleared when it is restored.
    • It is projected into the session catalog.
    • Sessions archived before the field exists have NULL, show "archive time unknown", and are excluded from age filters.
  • session.remove.preview takes a list of sessions and reports tasks, child tasks, bytes and worktrees that would be removed.
  • The Tasks page adds filters for "archived more than N days" and project. The confirmation lists the preview. Execution keeps using session.remove.
  • Epoch bump.

Questions for maintainers

  1. Is the explicit-request, pre-Ready conversion of runtime.sqlite acceptable? The alternative is to only report the reclaimable space and never convert.
  2. Should archived_at land as a small standalone PR first, since step 2 (retention) reuses it?
  3. Should bulk delete become a Host-side batch command (possibly together with fix(storage): batch retired Session artifact cleanup #4984), or stay a renderer loop over session.remove?

Risks

Refs #5776, #5341, #5394, #5038, #4984, #5605.

Activity

  1. garvit-arora commented on Sep 29, 2026

    @garvit-arora
    Contributor

    take

  2. garvit-arora commented on Sep 29, 2026

    @garvit-arora
    Contributor

    Plan (M2 reclamation): add bounded incremental_vacuum lane for context-offload.sqlite in HostStorageMaintenance; support explicit user-requested runtime.sqlite INCREMENTAL conversion + compact at next Host start (pre-Ready, after free-disk check); passive wal_checkpoint in maintenance loop; wire Data settings to show space reclaimed. Branch: feat/5825-m2-storage-reclamation. M1 visibility (#5832) stays separate.

  3. liugddx commented on Sep 30, 2026

    @liugddx
    MemberAuthor

    Status update:

    Open question 1 now blocks the rest of M2: where should a full VACUUM of runtime.sqlite run? #5855 has measured both placements:

    • before listen, it pushes startup toward the 75 s deadline (~55 s for a 1 GB DB);
    • after Ready, it blocks the serving event loop (~13 s for ~1 GB), past the 8 s client liveness timeout.

    The options I see:

    1. a worker thread with its own connection;
    2. run only while no client is attached;
    3. pre-listen, with a startup progress state and an extended deadline;
    4. don't convert, and only report reclaimable space.

    A maintainer call here would unblock the follow-up.

  4. liugddx commented on Oct 2, 2026

    @liugddx
    MemberAuthor

    Status update:

    One open item. Converting runtime.sqlite to incremental vacuum still has no decision on where its one full VACUUM should run:

    • before listen: about 55 s for a 1 GB database, against a 75 s startup deadline;
    • after Ready: blocks the serving event loop for about 13 s per GB;
    • in a worker;
    • or never, and only report the reclaimable space.

    Once #5855 lands I'll close this tracker, and move the runtime.sqlite question to its own issue so it isn't lost.

    Next steps from #5776 continue elsewhere: retention is #5899 / #5902.

  5. liugddx commented on Oct 8, 2026

    @liugddx
    MemberAuthor

    Closing. All three steps have landed:

    The one open item, where runtime.sqlite's one-time full VACUUM should run before it can switch to incremental vacuum, now lives in #6000 with the measurements and options. Retention, the next step from #5776, landed in #5902, with notices in #5961.

    One note for the record: the #5855 squash commit (c542bc757) went in subject-only, so it does not carry the Generated-by: Claude Code trailer that the follow-up commits had. Claude Code made a substantive contribution to that PR's final revision, as its description states.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions