Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
3146cc2
chore(tasks): validation round 2 — mutations aimed at files their own…
Aug 6, 2026
2a4568a
chore(tasks): task decomposition approved for 017-feat-wal-archiver
Aug 6, 2026
2577f46
chore(exec): execution plan for 017-feat-wal-archiver
Aug 6, 2026
9ec944b
feat: task 04 — report -W becomes a string flag (w|a)
Aug 6, 2026
7022058
docs: task 04 decisions entry and review reports
Aug 6, 2026
7a523a1
feat: task 02 — PG 19 fpi,KiB column on the wal screen
Aug 6, 2026
23a7b1f
chore(exec): correct the exit-code claim and assign cmd/help.go
Aug 6, 2026
814a21b
docs: task 02 decisions entry and review reports
Aug 6, 2026
ae50197
feat: task 01 — archiver query, selector and the shared test-role helper
Aug 6, 2026
db495c0
docs: task 01 decisions entry and review reports
Aug 6, 2026
e9d7b20
chore(exec): go build ./cmd does not work — correct the build command
Aug 6, 2026
13c33f2
feat: task 05 — register the archiver view and update the layout-pinn…
Aug 6, 2026
e11e2fe
docs: task 05 decisions entry and review reports
Aug 6, 2026
e3ac49d
feat: task 03 — verbose backlog via a pg_monitor-accessible function
Aug 6, 2026
3d8886b
docs: task 03 decisions entry and review reports
Aug 6, 2026
a35ada7
docs: task 09 — 0.12.0 release notes and the report -W help line
Aug 6, 2026
c34c78c
docs: task 09 decisions entry
Aug 6, 2026
91f4cf8
test: task 08 — golden replay coverage for archiver and the wal screen
Aug 6, 2026
d8f6ea0
docs: task 08 decisions entry
Aug 6, 2026
e7192fd
feat: task 07 — describe text for the archiver screen and the wal FPI…
Aug 6, 2026
3f41ee3
docs: task 07 decisions entry and review reports
Aug 6, 2026
18fb822
feat: task 06 — w cycle, W menu and help entries for the archiver screen
Aug 6, 2026
d3adc44
docs: task 06 decisions log entry
Aug 6, 2026
b47ff76
chore: complete waves 1-3 — all implementation tasks done
Aug 6, 2026
4530f04
docs: task 10 pre-deploy QA report — automated half green, manual gat…
Aug 6, 2026
27d9a8f
chore: wave 4 — automated QA green, manual gate blocked on stand access
Aug 6, 2026
be80fd2
docs(017): manual QA half — stand run complete, Decision 9 applied
Aug 6, 2026
d772c67
chore(techspec): Decision 20 — throttle the verbose backlog, with the…
Aug 6, 2026
e298313
chore(techspec): Decision 20 — measured and accepted, no throttling
Aug 6, 2026
aa86588
docs: finalize 017-feat-wal-archiver — PK, features catalog, ADR log,…
Aug 6, 2026
bbefb33
chore: go mod tidy — pflag is a direct dependency
Aug 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 43 additions & 1 deletion .claude/skills/project-knowledge/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ Version-specific query selectors in `internal/query/`:
- `SelectStatReplicationQuery(version, track)` — branches at PG 10
- `SelectStatDatabaseGeneralQuery(version)` — branches at PG 12
- `SelectStatStatementsTimingQuery(version)` — branches at PG 13, PG 17
- `SelectStatWALQuery(version)` — branches at PG 18 (columns removed)
- `SelectStatWALQuery(version)` — branches at PG 18 (columns removed) and PG 19 (`fpi,KiB` from `wal_fpi_bytes` inserted right after the `fpi` counter, so 8 cols / `DiffIntvl{2,6}`; the new column lands *inside* the diffed range and pushes its upper bound out by one, while `stats_age` must stay outside it — a `date_trunc` text value inside the range aborts the whole sample in `ParseInt`)
- `SelectStatArchiverQuery(_ int)` — version-independent: `pg_stat_archiver` is schema-identical on PG 14–19, so one query serves all, returning `(query, 9, [2]int{0,0})`
- `SelectStatBgwriterQuery(version)` — branches at PG 17 (`pg_stat_checkpointer` split off `pg_stat_bgwriter`) and PG 18 (`slru_written` added). Returns `(query, Ncols, DiffIntvl)` — DiffIntvl also differs per version.
- `SelectStatReplicationSlotsQuery(_ int)` — version-independent on PG 14–19 (chosen column subset is stable), returns `(query, 15, [2]int{6,13})`; the `version` param is kept for selector-signature symmetry. Single hybrid `pg_replication_slots LEFT JOIN pg_stat_replication_slots` query.
- `SelectStatProgressVacuumQuery(version)` / `SelectStatProgressAnalyzeQuery(version)` / `SelectStatProgressBasebackupQuery(version)` — branch at PG 19, which adds `started_by`+`mode` to the vacuum screen, `started_by` to analyze and `backup_type` to basebackup. All three return `(query, Ncols, DiffIntvl)`: the columns are inserted before `state`, so the diffed pairs shift (vacuum 13/`{10,11}` → 15/`{12,13}`; analyze 12 → 13, `DiffIntvl` stays `{0,0}`; basebackup 11/`{9,9}` → 12/`{10,10}`). `UniqueKey` stays 0 — `pid` remains column 0 in every layout, so the [007] 4-tuple is not needed.
Expand All @@ -76,6 +77,46 @@ The `pg_stat_io` screen (hotkey `j`/`J`, `internal/query/io.go`) is split into *

> **Note (009-feat-horizontal-scroll):** the main stats table now *has* horizontal column scroll (see "Horizontal Column Scroll" below), so the historical "no horizontal scroll" framing in the [006-feat-pg-stat-io] / [007] ADRs no longer holds as a constraint. The two-screen `pg_stat_io` split, the seven `pg_stat_statements` sub-screens, and the synthetic `io_key` are kept deliberately — they are a product decision (logical grouping and isolation of related data), not a workaround for a missing feature. Scroll exists for narrow terminals; it is not meant to collapse the sub-screens into one wide view.

## WAL and Archiving Area (017-feat-wal-archiver)

The `archiver` view (`internal/query/archiver.go`) is a single-row screen over `pg_stat_archiver`
plus one privileged sub-select — the count of `.ready` entries from `pg_ls_archive_statusdir()`.

- **Nothing is diffed.** `DiffIntvl{0,0}` makes `calculateDelta` short-circuit before `diff()`, so
the whole row passes through untouched. That is what makes the literal `'Archiver'` at column 0
safe (it is never parsed) and what lets the four nullable columns — both WAL names and both ages —
stay uncoalesced: a blank cell is the honest rendering of "this cluster has never archived", the
same reasoning as `backend_xid` on the activity screen. Coalescing is required only *inside* a
diffed range.
- **`MinRequiredVersion: PostgresV14` is load-bearing, not cosmetic.** There is no common version
floor in the registry — it still serves down to PG 9.4 — so a zero value would offer the screen on
PG ≤ 11, where `pg_ls_archive_statusdir()` (PG 12+) does not exist. The TUI would error every
tick, and `pgcenter record` aborts the **entire** recording on the first failing view query.
- **The privileged call is unconditional and takes the whole screen down without `pg_monitor`** —
deliberately, the same shape the `wal` screen already has with `pg_ls_waldir()`. See the ADR log
for why a `has_function_privilege()` guard cannot work at all.
- **Navigation.** `w` cycles `wal` ↔ `archiver` through `walNextView` (`top/config_view.go`); `W`
opens the two-item `menuWAL` (`top/menu.go`), whose branch calls `viewSwitchHandler` directly and
therefore never passes through `switchViewTo`. This is the only cycle whose group name *is* a view
name: `"wal"` cannot be renamed — it is the `report -W w` report type and the tar entry prefix in
recorded archives — so the dispatch case carries a comment saying so. `walNextView`'s default arm
returns `"wal"`, which is why `w` from any other screen behaves exactly as before the cycle existed.
- **record/report needed no recorder change** (the [008] pure-SQL rule). The CLI side is where the
work was: `-W` became a string flag with a **closed** `w`/`a` whitelist and no default arm —
`ReportType` is both the tar-entry filter in `isFilenameOK` and the key into the view map, so a
leaked value would select a zero-value `view.View` and print a silently empty report instead of an
error.

**The verbose panel's archiving backlog moved to `pg_ls_archive_statusdir()`** in the same pass
(`internal/query/overview.go`). Its predecessor `pg_ls_dir('pg_wal/archive_status')` has ACL
`{postgres}` — superuser only — while `pg_ls_waldir` and `pg_ls_archive_statusdir` are
`{postgres, pg_monitor}`, so the field was `n/a` for exactly the monitoring role the panel serves.
Output (bytes) and the degrade-to-`n/a` path are unchanged. Two consequences to know: the new
function is `missing_ok=true`, so a cluster whose `archive_status` directory is absent now shows a
confident `0` (a bare `0` — the size formatter's zero case returns the digit alone — not `0 B`);
and it stats every file instead of listing names, so the panel, which rides every screen, pays that
walk on every screen. Measured cost and why it is not throttled are in the ADR log.

## Horizontal Column Scroll (009-feat-horizontal-scroll)

The main stats table (the `dbstat` area, shared by every stat screen) scrolls horizontally by column. Hotkeys `]` (right) and `[` (left) move a sliding window over the columns; the first column is **frozen** (always rendered) so the row identifier — PID / database / table name — never scrolls off. Closes issue #14 (open since 2015). Scope is the main table only; side extra-panels (iostat/netdev/fsstats/logtail) and the record/report pipeline are untouched.
Expand Down Expand Up @@ -193,6 +234,7 @@ Integration tests require a running PostgreSQL instance.
Test helpers in `internal/postgres/testing.go`:
- `NewTestConnect()` — connects to PG 17 (port 21917, default)
- `NewTestConnectVersion(version)` — connects to specific version; returns an error for a version with no port mapping (it used to fall back to the oldest cluster, which made a forgotten entry invisible) and for unavailable versions (callers use `t.Skipf`)
- `SetupTestRole(db, name, pgMonitor)` (017-feat-wal-archiver) — creates a `NOLOGIN` role idempotently, optionally grants `pg_monitor`, and does `SET ROLE`; used by tests that must prove a query's privilege behaviour in both directions. It returns an `error` and takes no `*testing.T` on purpose: `testing.go` carries no build tag, so it links into the release binary and must not import `testing`. The roles are never dropped (their reusability is the point) — correct for ephemeral CI containers, worth knowing on a long-lived cluster.

Port map: PG14=21914, PG15=21915, PG16=21916, PG17=21917, PG18=21918, PG19=21919.
EOL entries (PG 9.5–13) kept in map but connections will fail gracefully.
Expand Down
5 changes: 3 additions & 2 deletions .claude/skills/project-knowledge/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ It reads PostgreSQL internal statistics views and presents them in a top-like in
|-----------|---------|
| `top` | Real-time monitoring (main feature) — live stats with refresh; the main stats table scrolls horizontally by column (`[`/`]`) with a frozen first column for narrow terminals (009-feat-horizontal-scroll); hotkey `v` expands the top `sysstat`/`pgstat` summary panels into a verbose instance-health overview (+3/+5 rows), persistent across screens (010-feat-overview-dashboard) |
| `record` | Collect stats to tar files ("poor man's monitoring") |
| `report` | Build reports from recorded files |
| `report` | Build reports from recorded files. **Breaking change in 0.12.0:** `-W` is no longer a boolean — it takes `w` (pg_stat_wal) or `a` (pg_stat_archiver), like `-J c\|t`. A legacy `report -W -f dump.tar` fails with `report type is not specified, quit` because pflag eats `-f` as the flag's value; documented in `doc/release-notes/v0.12.0.md` |
| `profile` | Wait events profiler — shows what queries are waiting on |

## Supported PostgreSQL Statistics
Expand All @@ -21,7 +21,8 @@ It reads PostgreSQL internal statistics views and presents them in a top-like in
- `pg_stat_bgwriter` (+ `pg_stat_checkpointer` on PG 17+) — background writer / checkpointer screen (hotkey `b`; PG 14–19; recordable via `record`/`report -B`)
- `pg_replication_slots` (+ `pg_stat_replication_slots`) — replication slots screen (hotkey `o`; PG 14–19; multi-row, all slots; retained WAL + wal_status + spill/stream; recordable via `record`/`report -L`)
- `pg_stat_io` — unified IO breakdown by backend_type × object × context (hotkey `j` toggles count↔time sub-screens, `J` opens the mode menu; PG 16+; multi-row; this is where `buffers_backend`/`buffers_backend_fsync` went on PG 17+ and WAL IO timings on PG 18; recordable via `record`/`report -J c|t`)
- `pg_stat_wal` — WAL generation stats (PG 14+; reduced schema in PG 18 — WAL IO timings moved to `pg_stat_io`)
- `pg_stat_wal` — WAL generation stats (hotkey `w`; PG 14+; reduced schema in PG 18 — WAL IO timings moved to `pg_stat_io`; on PG 19 a `fpi,KiB` column from `wal_fpi_bytes` sits next to the `fpi` counter, so how much WAL full-page images actually cost is readable beside how many there were; recordable via `record`/`report -W w`)
- `pg_stat_archiver` — archiver screen (hotkey `w` cycles `wal` ↔ `archiver`, `W` opens the two-item menu; PG 14+; single row: the `.ready` backlog count plus the archiver's own success/failure counters, last WAL name and ages on each side; recordable via `record`/`report -W a`). Answers "when did archiving stop, on which segment, and how much has piled up"; the first signal that it stopped comes earlier, from the verbose panel's backlog in bytes
- `pg_stat_statements` — top queries by various metrics (requires extension); 7 sub-screens under the `X` menu / `x` cycle: timings, general, IO, temp files, local (temp tables), WAL, and **JIT** (compilation cost per query — generation/inlining/optimization/emission phase times + functions, `+deform` on PG 17+; PG 15+; rows filtered to `jit_functions > 0`; recordable via `record`/`report -X j`)
- `pg_stat_progress_*` — progress of vacuum, analyze, cluster, create index, basebackup and copy (hotkey `p` cycles, `P` opens the menu); on PG 19 the vacuum screen also shows `started_by` and `mode`, analyze shows `started_by`, and basebackup shows `backup_type`
- System stats — CPU, memory, disk, network (read from /proc or via PL/Perl schema)
Expand Down
38 changes: 37 additions & 1 deletion .claude/skills/project-knowledge/patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,34 @@ invariant: name the mutation the test must fail on, run it, and see red before b
When the property genuinely cannot be reached without a forbidden seam, say so in the test comment
and defer it to the stand run — do not dress inspection up as a red test.

## Running the mutation, and reading its result honestly (017-feat-wal-archiver)

The rule above ("name the mutation, see it red") is necessary but not sufficient. Three ways it was
observed to lie in 017, all found by doing it rather than by reasoning about it:

- **A mutation that reddens for the wrong reason proves nothing.** Name the mutation *and* check
which assertion goes red. Dropping `pg_monitor` from the positive privilege test reddened on the
test's own membership guard — before the query ran — so the SQLSTATE 42501 the criterion asked
about was never reached, and the claim ended up resting on a permanently-present negative test
instead. Same class, recorded as a deliberate negative control: deleting the `archiver` case from
`Views.Configure()` leaves the package green, because `New()` already seeds the same
`QueryTmpl`/`Ncols`/`DiffIntvl` — so those `TestViews_Configure` asserts guard drift between the
selector and the static entry, not the wiring they look like they guard. Write that boundary next
to the assertion; the next reader will otherwise assume the stronger claim.
- **"No FAIL lines" is not "the test passed".** A mutation that breaks compilation produces
neither — one in 017 left a variable unused, the package did not build, and grepping the output
for `FAIL` read as green. Mutate through a declaration that stays used (or otherwise keep the
package compiling), and confirm the run by counting PASS/RUN for the named subtest rather than by
the absence of FAIL. The sibling trap: `go test -run` is case-sensitive, so a filter that silently
matches nothing looks identical to a clean pass.
- **"This cannot be unit-tested" was false twice in one feature.** A zero-value `&gocui.Gui{}` is
enough to drive `menuSelect`'s branches to completion and to exercise keybinding registration —
the gocui constraint recorded above applies to `g.Update` closures and `*gocui.View`, not to
everything that mentions gocui. What gocui does not give you is the registered binding: the
handlers are unexported and cannot be fetched or invoked, so `keybindings()` was split into a
`keybindingsList(app) []key` table plus the registration loop, which makes *which handler a key
carries* assertable. Before that split, binding `W` to the wrong menu left the whole suite green.

## Verbose display-mode toggle (010-feat-overview-dashboard)

When adding an on/off *display mode* that layers extra rows over the current screen (not a new screen),
Expand Down Expand Up @@ -153,7 +181,15 @@ Registering a view in `view.New()` couples to count-based tests that fail in CI
- `internal/view/view_test.go: TestNew` pins the total view count. `TestView_VersionOK` pins per-version availability — its row at a version **≥ the new view's `MinRequiredVersion`** also increases by one (feature 007's PG15+ view bumped only the `160000` row, not the `≤140000` rows).
- `record/record_test.go: Test_filterViews` pins, per version, how many views `filterViews` drops vs keeps. A `NotRecordable: true` view is always dropped, so every `wantN` row increases by the number of new `NotRecordable` views (feature 006 added 2 → `+2` each row; feature 007 added 1 → `+1`; `wantV` unchanged). This test runs without Postgres, so a stale count is a real failure even though the rest of the `record` package skips/fails on a missing PG fixture — do not assume a red `record` package is only the connection-refused tests.

Adding a `pg_stat_statements` **sub-screen** (or any `menuPgss`/cycle entry) additionally breaks `top` tests — `Test_selectMenuStyle` (pins each menu's item count), `Test_statementsNextView`, and `Test_switchViewTo` (pin the `x`-cycle transitions). These `top` tests DO run locally without Postgres, so they catch the miss in `make test` — but feature 007's code-research overlooked them (the task wrongly assumed the TUI layer had no tests). When touching `top/menu.go` or `top/config_view.go`, grep `top/*_test.go` for the function you changed before assuming it is untested.
Counts alone are a weak guard, and 017 measured it: with `Test_filterViews` pinning only numbers, a
view that fell out of the kept set could be masked by an arithmetic coincidence, so the table gained
a per-row "is *this* view kept" field. The same feature also found that nothing pins the registry's
own invariants for any other view — `key == v.Name`, and non-nil `ColsWidth`/`Filters`. The maps
matter beyond tidiness: three writers in `top/config_view.go` write into them **in place**, so a nil
map is a panic on the first column-width change or filter, not a wrong number. Pin them for any view
you add.

Adding a `pg_stat_statements` **sub-screen** (or any `menuPgss`/cycle entry) additionally breaks `top` tests — `Test_selectMenuStyle` (pins each menu's item count), `Test_statementsNextView`, and `Test_switchViewTo` (pin the `x`-cycle transitions). These `top` tests DO run locally without Postgres, so they catch the miss in `make test` — but feature 007's code-research overlooked them (the task wrongly assumed the TUI layer had no tests). When touching `top/menu.go` or `top/config_view.go`, grep `top/*_test.go` for the function you changed before assuming it is untested. A **new** hotkey group (017's `w`/`W`) breaks the same three plus the help-screen tests, and needs one thing more: the binding itself, which only `keybindingsList` makes assertable (see the mutation section above).

## Error Wrapping

Expand Down
3 changes: 2 additions & 1 deletion cmd/help.go
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,8 @@ Report options:
-I, --indexes show pg_stat_user_indexes and pg_statio_user_indexes statistics
-S, --sizes show statistics about tables sizes
-F, --functions show pg_stat_user_functions statistics
-W, --wal show pg_stat_wal statistics
-W, --wal SELECTOR show pg_stat_wal / pg_stat_archiver statistics, use additional selector to choose stats:
'w' - wal; 'a' - archiver
-N, --proc-stats show per-process system stats (procpidstat); local recordings only
-D, --databases SELECTOR show pg_stat_database statistics, use additional selector to choose stats:
'g' - general; 's' - sessions
Expand Down
16 changes: 12 additions & 4 deletions cmd/report/report.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ type options struct {
showIndexes bool // Show stats from pg_stat_user_indexes, pg_statio_user_indexes
showSizes bool // Show tables sizes
showFunctions bool // Show stats from pg_stat_user_functions
showWAL bool // Show stats from pg_stat_wal
showWAL string // Show stats from pg_stat_wal / pg_stat_archiver
showBgwriter bool // Show stats from pg_stat_bgwriter, pg_stat_checkpointer
showReplSlots bool // Show stats from pg_replication_slots, pg_stat_replication_slots
showStatIO string // Show stats from pg_stat_io
Expand Down Expand Up @@ -67,7 +67,7 @@ func init() {
CommandDefinition.Flags().BoolVarP(&opts.showIndexes, "indexes", "I", false, "show pg_stat_user_indexes and pg_statio_user_indexes report")
CommandDefinition.Flags().BoolVarP(&opts.showSizes, "sizes", "S", false, "show tables sizes report")
CommandDefinition.Flags().BoolVarP(&opts.showFunctions, "functions", "F", false, "show pg_stat_user_functions report")
CommandDefinition.Flags().BoolVarP(&opts.showWAL, "wal", "W", false, "show pg_stat_wal report")
CommandDefinition.Flags().StringVarP(&opts.showWAL, "wal", "W", "", "show pg_stat_wal / pg_stat_archiver report (w - wal, a - archiver)")
CommandDefinition.Flags().BoolVarP(&opts.showBgwriter, "bgwriter", "B", false, "show pg_stat_bgwriter / pg_stat_checkpointer report")
CommandDefinition.Flags().BoolVarP(&opts.showReplSlots, "replslots", "L", false, "show pg_replication_slots / pg_stat_replication_slots report")
CommandDefinition.Flags().StringVarP(&opts.showStatIO, "io", "J", "", "show pg_stat_io report (c - count, t - time)")
Expand Down Expand Up @@ -148,8 +148,16 @@ func selectReport(opts options) string {
return "indexes"
case opts.showFunctions:
return "functions"
case opts.showWAL:
return "wal"
case opts.showWAL != "":
// Closed whitelist: no default arm on purpose. An unmatched value falls out of both switches
// to the final 'return ""', so validate() rejects it instead of letting an unknown report
// type reach report.Config and select a zero-value view (a silently empty report).
switch opts.showWAL {
case "w":
return "wal"
case "a":
return "archiver"
}
case opts.showBgwriter:
return "bgwriter"
case opts.showReplSlots:
Expand Down
Loading
Loading