diff --git a/.claude/skills/project-knowledge/architecture.md b/.claude/skills/project-knowledge/architecture.md index 4dd7469d..a66c286e 100644 --- a/.claude/skills/project-knowledge/architecture.md +++ b/.claude/skills/project-knowledge/architecture.md @@ -62,7 +62,8 @@ Version-specific query selectors in `internal/query/`: - `SelectStatReplicationQuery(version, track)` — branches at PG 10 - `SelectStatDatabaseGeneralQuery(version)` — branches at PG 12 - `SelectStatStatementsTimingQuery(version)` — branches at PG 13, PG 17 -- `SelectStatWALQuery(version)` — branches at PG 18 (columns removed) +- `SelectStatWALQuery(version)` — branches at PG 18 (columns removed) and PG 19 (`fpi,KiB` from `wal_fpi_bytes` inserted right after the `fpi` counter, so 8 cols / `DiffIntvl{2,6}`; the new column lands *inside* the diffed range and pushes its upper bound out by one, while `stats_age` must stay outside it — a `date_trunc` text value inside the range aborts the whole sample in `ParseInt`) +- `SelectStatArchiverQuery(_ int)` — version-independent: `pg_stat_archiver` is schema-identical on PG 14–19, so one query serves all, returning `(query, 9, [2]int{0,0})` - `SelectStatBgwriterQuery(version)` — branches at PG 17 (`pg_stat_checkpointer` split off `pg_stat_bgwriter`) and PG 18 (`slru_written` added). Returns `(query, Ncols, DiffIntvl)` — DiffIntvl also differs per version. - `SelectStatReplicationSlotsQuery(_ int)` — version-independent on PG 14–19 (chosen column subset is stable), returns `(query, 15, [2]int{6,13})`; the `version` param is kept for selector-signature symmetry. Single hybrid `pg_replication_slots LEFT JOIN pg_stat_replication_slots` query. - `SelectStatProgressVacuumQuery(version)` / `SelectStatProgressAnalyzeQuery(version)` / `SelectStatProgressBasebackupQuery(version)` — branch at PG 19, which adds `started_by`+`mode` to the vacuum screen, `started_by` to analyze and `backup_type` to basebackup. All three return `(query, Ncols, DiffIntvl)`: the columns are inserted before `state`, so the diffed pairs shift (vacuum 13/`{10,11}` → 15/`{12,13}`; analyze 12 → 13, `DiffIntvl` stays `{0,0}`; basebackup 11/`{9,9}` → 12/`{10,10}`). `UniqueKey` stays 0 — `pid` remains column 0 in every layout, so the [007] 4-tuple is not needed. @@ -76,6 +77,46 @@ The `pg_stat_io` screen (hotkey `j`/`J`, `internal/query/io.go`) is split into * > **Note (009-feat-horizontal-scroll):** the main stats table now *has* horizontal column scroll (see "Horizontal Column Scroll" below), so the historical "no horizontal scroll" framing in the [006-feat-pg-stat-io] / [007] ADRs no longer holds as a constraint. The two-screen `pg_stat_io` split, the seven `pg_stat_statements` sub-screens, and the synthetic `io_key` are kept deliberately — they are a product decision (logical grouping and isolation of related data), not a workaround for a missing feature. Scroll exists for narrow terminals; it is not meant to collapse the sub-screens into one wide view. +## WAL and Archiving Area (017-feat-wal-archiver) + +The `archiver` view (`internal/query/archiver.go`) is a single-row screen over `pg_stat_archiver` +plus one privileged sub-select — the count of `.ready` entries from `pg_ls_archive_statusdir()`. + +- **Nothing is diffed.** `DiffIntvl{0,0}` makes `calculateDelta` short-circuit before `diff()`, so + the whole row passes through untouched. That is what makes the literal `'Archiver'` at column 0 + safe (it is never parsed) and what lets the four nullable columns — both WAL names and both ages — + stay uncoalesced: a blank cell is the honest rendering of "this cluster has never archived", the + same reasoning as `backend_xid` on the activity screen. Coalescing is required only *inside* a + diffed range. +- **`MinRequiredVersion: PostgresV14` is load-bearing, not cosmetic.** There is no common version + floor in the registry — it still serves down to PG 9.4 — so a zero value would offer the screen on + PG ≤ 11, where `pg_ls_archive_statusdir()` (PG 12+) does not exist. The TUI would error every + tick, and `pgcenter record` aborts the **entire** recording on the first failing view query. +- **The privileged call is unconditional and takes the whole screen down without `pg_monitor`** — + deliberately, the same shape the `wal` screen already has with `pg_ls_waldir()`. See the ADR log + for why a `has_function_privilege()` guard cannot work at all. +- **Navigation.** `w` cycles `wal` ↔ `archiver` through `walNextView` (`top/config_view.go`); `W` + opens the two-item `menuWAL` (`top/menu.go`), whose branch calls `viewSwitchHandler` directly and + therefore never passes through `switchViewTo`. This is the only cycle whose group name *is* a view + name: `"wal"` cannot be renamed — it is the `report -W w` report type and the tar entry prefix in + recorded archives — so the dispatch case carries a comment saying so. `walNextView`'s default arm + returns `"wal"`, which is why `w` from any other screen behaves exactly as before the cycle existed. +- **record/report needed no recorder change** (the [008] pure-SQL rule). The CLI side is where the + work was: `-W` became a string flag with a **closed** `w`/`a` whitelist and no default arm — + `ReportType` is both the tar-entry filter in `isFilenameOK` and the key into the view map, so a + leaked value would select a zero-value `view.View` and print a silently empty report instead of an + error. + +**The verbose panel's archiving backlog moved to `pg_ls_archive_statusdir()`** in the same pass +(`internal/query/overview.go`). Its predecessor `pg_ls_dir('pg_wal/archive_status')` has ACL +`{postgres}` — superuser only — while `pg_ls_waldir` and `pg_ls_archive_statusdir` are +`{postgres, pg_monitor}`, so the field was `n/a` for exactly the monitoring role the panel serves. +Output (bytes) and the degrade-to-`n/a` path are unchanged. Two consequences to know: the new +function is `missing_ok=true`, so a cluster whose `archive_status` directory is absent now shows a +confident `0` (a bare `0` — the size formatter's zero case returns the digit alone — not `0 B`); +and it stats every file instead of listing names, so the panel, which rides every screen, pays that +walk on every screen. Measured cost and why it is not throttled are in the ADR log. + ## Horizontal Column Scroll (009-feat-horizontal-scroll) The main stats table (the `dbstat` area, shared by every stat screen) scrolls horizontally by column. Hotkeys `]` (right) and `[` (left) move a sliding window over the columns; the first column is **frozen** (always rendered) so the row identifier — PID / database / table name — never scrolls off. Closes issue #14 (open since 2015). Scope is the main table only; side extra-panels (iostat/netdev/fsstats/logtail) and the record/report pipeline are untouched. @@ -193,6 +234,7 @@ Integration tests require a running PostgreSQL instance. Test helpers in `internal/postgres/testing.go`: - `NewTestConnect()` — connects to PG 17 (port 21917, default) - `NewTestConnectVersion(version)` — connects to specific version; returns an error for a version with no port mapping (it used to fall back to the oldest cluster, which made a forgotten entry invisible) and for unavailable versions (callers use `t.Skipf`) +- `SetupTestRole(db, name, pgMonitor)` (017-feat-wal-archiver) — creates a `NOLOGIN` role idempotently, optionally grants `pg_monitor`, and does `SET ROLE`; used by tests that must prove a query's privilege behaviour in both directions. It returns an `error` and takes no `*testing.T` on purpose: `testing.go` carries no build tag, so it links into the release binary and must not import `testing`. The roles are never dropped (their reusability is the point) — correct for ephemeral CI containers, worth knowing on a long-lived cluster. Port map: PG14=21914, PG15=21915, PG16=21916, PG17=21917, PG18=21918, PG19=21919. EOL entries (PG 9.5–13) kept in map but connections will fail gracefully. diff --git a/.claude/skills/project-knowledge/overview.md b/.claude/skills/project-knowledge/overview.md index 7bc4171d..1840a5f8 100644 --- a/.claude/skills/project-knowledge/overview.md +++ b/.claude/skills/project-knowledge/overview.md @@ -9,7 +9,7 @@ It reads PostgreSQL internal statistics views and presents them in a top-like in |-----------|---------| | `top` | Real-time monitoring (main feature) — live stats with refresh; the main stats table scrolls horizontally by column (`[`/`]`) with a frozen first column for narrow terminals (009-feat-horizontal-scroll); hotkey `v` expands the top `sysstat`/`pgstat` summary panels into a verbose instance-health overview (+3/+5 rows), persistent across screens (010-feat-overview-dashboard) | | `record` | Collect stats to tar files ("poor man's monitoring") | -| `report` | Build reports from recorded files | +| `report` | Build reports from recorded files. **Breaking change in 0.12.0:** `-W` is no longer a boolean — it takes `w` (pg_stat_wal) or `a` (pg_stat_archiver), like `-J c\|t`. A legacy `report -W -f dump.tar` fails with `report type is not specified, quit` because pflag eats `-f` as the flag's value; documented in `doc/release-notes/v0.12.0.md` | | `profile` | Wait events profiler — shows what queries are waiting on | ## Supported PostgreSQL Statistics @@ -21,7 +21,8 @@ It reads PostgreSQL internal statistics views and presents them in a top-like in - `pg_stat_bgwriter` (+ `pg_stat_checkpointer` on PG 17+) — background writer / checkpointer screen (hotkey `b`; PG 14–19; recordable via `record`/`report -B`) - `pg_replication_slots` (+ `pg_stat_replication_slots`) — replication slots screen (hotkey `o`; PG 14–19; multi-row, all slots; retained WAL + wal_status + spill/stream; recordable via `record`/`report -L`) - `pg_stat_io` — unified IO breakdown by backend_type × object × context (hotkey `j` toggles count↔time sub-screens, `J` opens the mode menu; PG 16+; multi-row; this is where `buffers_backend`/`buffers_backend_fsync` went on PG 17+ and WAL IO timings on PG 18; recordable via `record`/`report -J c|t`) -- `pg_stat_wal` — WAL generation stats (PG 14+; reduced schema in PG 18 — WAL IO timings moved to `pg_stat_io`) +- `pg_stat_wal` — WAL generation stats (hotkey `w`; PG 14+; reduced schema in PG 18 — WAL IO timings moved to `pg_stat_io`; on PG 19 a `fpi,KiB` column from `wal_fpi_bytes` sits next to the `fpi` counter, so how much WAL full-page images actually cost is readable beside how many there were; recordable via `record`/`report -W w`) +- `pg_stat_archiver` — archiver screen (hotkey `w` cycles `wal` ↔ `archiver`, `W` opens the two-item menu; PG 14+; single row: the `.ready` backlog count plus the archiver's own success/failure counters, last WAL name and ages on each side; recordable via `record`/`report -W a`). Answers "when did archiving stop, on which segment, and how much has piled up"; the first signal that it stopped comes earlier, from the verbose panel's backlog in bytes - `pg_stat_statements` — top queries by various metrics (requires extension); 7 sub-screens under the `X` menu / `x` cycle: timings, general, IO, temp files, local (temp tables), WAL, and **JIT** (compilation cost per query — generation/inlining/optimization/emission phase times + functions, `+deform` on PG 17+; PG 15+; rows filtered to `jit_functions > 0`; recordable via `record`/`report -X j`) - `pg_stat_progress_*` — progress of vacuum, analyze, cluster, create index, basebackup and copy (hotkey `p` cycles, `P` opens the menu); on PG 19 the vacuum screen also shows `started_by` and `mode`, analyze shows `started_by`, and basebackup shows `backup_type` - System stats — CPU, memory, disk, network (read from /proc or via PL/Perl schema) diff --git a/.claude/skills/project-knowledge/patterns.md b/.claude/skills/project-knowledge/patterns.md index cbe5c37d..dc01a325 100644 --- a/.claude/skills/project-knowledge/patterns.md +++ b/.claude/skills/project-knowledge/patterns.md @@ -88,6 +88,34 @@ invariant: name the mutation the test must fail on, run it, and see red before b When the property genuinely cannot be reached without a forbidden seam, say so in the test comment and defer it to the stand run — do not dress inspection up as a red test. +## Running the mutation, and reading its result honestly (017-feat-wal-archiver) + +The rule above ("name the mutation, see it red") is necessary but not sufficient. Three ways it was +observed to lie in 017, all found by doing it rather than by reasoning about it: + +- **A mutation that reddens for the wrong reason proves nothing.** Name the mutation *and* check + which assertion goes red. Dropping `pg_monitor` from the positive privilege test reddened on the + test's own membership guard — before the query ran — so the SQLSTATE 42501 the criterion asked + about was never reached, and the claim ended up resting on a permanently-present negative test + instead. Same class, recorded as a deliberate negative control: deleting the `archiver` case from + `Views.Configure()` leaves the package green, because `New()` already seeds the same + `QueryTmpl`/`Ncols`/`DiffIntvl` — so those `TestViews_Configure` asserts guard drift between the + selector and the static entry, not the wiring they look like they guard. Write that boundary next + to the assertion; the next reader will otherwise assume the stronger claim. +- **"No FAIL lines" is not "the test passed".** A mutation that breaks compilation produces + neither — one in 017 left a variable unused, the package did not build, and grepping the output + for `FAIL` read as green. Mutate through a declaration that stays used (or otherwise keep the + package compiling), and confirm the run by counting PASS/RUN for the named subtest rather than by + the absence of FAIL. The sibling trap: `go test -run` is case-sensitive, so a filter that silently + matches nothing looks identical to a clean pass. +- **"This cannot be unit-tested" was false twice in one feature.** A zero-value `&gocui.Gui{}` is + enough to drive `menuSelect`'s branches to completion and to exercise keybinding registration — + the gocui constraint recorded above applies to `g.Update` closures and `*gocui.View`, not to + everything that mentions gocui. What gocui does not give you is the registered binding: the + handlers are unexported and cannot be fetched or invoked, so `keybindings()` was split into a + `keybindingsList(app) []key` table plus the registration loop, which makes *which handler a key + carries* assertable. Before that split, binding `W` to the wrong menu left the whole suite green. + ## Verbose display-mode toggle (010-feat-overview-dashboard) When adding an on/off *display mode* that layers extra rows over the current screen (not a new screen), @@ -153,7 +181,15 @@ Registering a view in `view.New()` couples to count-based tests that fail in CI - `internal/view/view_test.go: TestNew` pins the total view count. `TestView_VersionOK` pins per-version availability — its row at a version **≥ the new view's `MinRequiredVersion`** also increases by one (feature 007's PG15+ view bumped only the `160000` row, not the `≤140000` rows). - `record/record_test.go: Test_filterViews` pins, per version, how many views `filterViews` drops vs keeps. A `NotRecordable: true` view is always dropped, so every `wantN` row increases by the number of new `NotRecordable` views (feature 006 added 2 → `+2` each row; feature 007 added 1 → `+1`; `wantV` unchanged). This test runs without Postgres, so a stale count is a real failure even though the rest of the `record` package skips/fails on a missing PG fixture — do not assume a red `record` package is only the connection-refused tests. -Adding a `pg_stat_statements` **sub-screen** (or any `menuPgss`/cycle entry) additionally breaks `top` tests — `Test_selectMenuStyle` (pins each menu's item count), `Test_statementsNextView`, and `Test_switchViewTo` (pin the `x`-cycle transitions). These `top` tests DO run locally without Postgres, so they catch the miss in `make test` — but feature 007's code-research overlooked them (the task wrongly assumed the TUI layer had no tests). When touching `top/menu.go` or `top/config_view.go`, grep `top/*_test.go` for the function you changed before assuming it is untested. +Counts alone are a weak guard, and 017 measured it: with `Test_filterViews` pinning only numbers, a +view that fell out of the kept set could be masked by an arithmetic coincidence, so the table gained +a per-row "is *this* view kept" field. The same feature also found that nothing pins the registry's +own invariants for any other view — `key == v.Name`, and non-nil `ColsWidth`/`Filters`. The maps +matter beyond tidiness: three writers in `top/config_view.go` write into them **in place**, so a nil +map is a panic on the first column-width change or filter, not a wrong number. Pin them for any view +you add. + +Adding a `pg_stat_statements` **sub-screen** (or any `menuPgss`/cycle entry) additionally breaks `top` tests — `Test_selectMenuStyle` (pins each menu's item count), `Test_statementsNextView`, and `Test_switchViewTo` (pin the `x`-cycle transitions). These `top` tests DO run locally without Postgres, so they catch the miss in `make test` — but feature 007's code-research overlooked them (the task wrongly assumed the TUI layer had no tests). When touching `top/menu.go` or `top/config_view.go`, grep `top/*_test.go` for the function you changed before assuming it is untested. A **new** hotkey group (017's `w`/`W`) breaks the same three plus the help-screen tests, and needs one thing more: the binding itself, which only `keybindingsList` makes assertable (see the mutation section above). ## Error Wrapping diff --git a/cmd/help.go b/cmd/help.go index 5a83a296..b5648cef 100644 --- a/cmd/help.go +++ b/cmd/help.go @@ -167,7 +167,8 @@ Report options: -I, --indexes show pg_stat_user_indexes and pg_statio_user_indexes statistics -S, --sizes show statistics about tables sizes -F, --functions show pg_stat_user_functions statistics - -W, --wal show pg_stat_wal statistics + -W, --wal SELECTOR show pg_stat_wal / pg_stat_archiver statistics, use additional selector to choose stats: + 'w' - wal; 'a' - archiver -N, --proc-stats show per-process system stats (procpidstat); local recordings only -D, --databases SELECTOR show pg_stat_database statistics, use additional selector to choose stats: 'g' - general; 's' - sessions diff --git a/cmd/report/report.go b/cmd/report/report.go index 4e3614a0..d008b549 100644 --- a/cmd/report/report.go +++ b/cmd/report/report.go @@ -22,7 +22,7 @@ type options struct { showIndexes bool // Show stats from pg_stat_user_indexes, pg_statio_user_indexes showSizes bool // Show tables sizes showFunctions bool // Show stats from pg_stat_user_functions - showWAL bool // Show stats from pg_stat_wal + showWAL string // Show stats from pg_stat_wal / pg_stat_archiver showBgwriter bool // Show stats from pg_stat_bgwriter, pg_stat_checkpointer showReplSlots bool // Show stats from pg_replication_slots, pg_stat_replication_slots showStatIO string // Show stats from pg_stat_io @@ -67,7 +67,7 @@ func init() { CommandDefinition.Flags().BoolVarP(&opts.showIndexes, "indexes", "I", false, "show pg_stat_user_indexes and pg_statio_user_indexes report") CommandDefinition.Flags().BoolVarP(&opts.showSizes, "sizes", "S", false, "show tables sizes report") CommandDefinition.Flags().BoolVarP(&opts.showFunctions, "functions", "F", false, "show pg_stat_user_functions report") - CommandDefinition.Flags().BoolVarP(&opts.showWAL, "wal", "W", false, "show pg_stat_wal report") + CommandDefinition.Flags().StringVarP(&opts.showWAL, "wal", "W", "", "show pg_stat_wal / pg_stat_archiver report (w - wal, a - archiver)") CommandDefinition.Flags().BoolVarP(&opts.showBgwriter, "bgwriter", "B", false, "show pg_stat_bgwriter / pg_stat_checkpointer report") CommandDefinition.Flags().BoolVarP(&opts.showReplSlots, "replslots", "L", false, "show pg_replication_slots / pg_stat_replication_slots report") CommandDefinition.Flags().StringVarP(&opts.showStatIO, "io", "J", "", "show pg_stat_io report (c - count, t - time)") @@ -148,8 +148,16 @@ func selectReport(opts options) string { return "indexes" case opts.showFunctions: return "functions" - case opts.showWAL: - return "wal" + case opts.showWAL != "": + // Closed whitelist: no default arm on purpose. An unmatched value falls out of both switches + // to the final 'return ""', so validate() rejects it instead of letting an unknown report + // type reach report.Config and select a zero-value view (a silently empty report). + switch opts.showWAL { + case "w": + return "wal" + case "a": + return "archiver" + } case opts.showBgwriter: return "bgwriter" case opts.showReplSlots: diff --git a/cmd/report/report_test.go b/cmd/report/report_test.go index d804c6cb..b6d5c645 100644 --- a/cmd/report/report_test.go +++ b/cmd/report/report_test.go @@ -2,7 +2,10 @@ package report import ( "github.com/lesovsky/pgcenter/report" + "github.com/spf13/pflag" "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "io" "testing" "time" ) @@ -18,6 +21,7 @@ func Test_options_validate(t *testing.T) { {valid: false, opts: options{tsStart: "2021-01-01 12:00:00", tsEnd: "2021-01-01 13:00:00"}}, // no report type specified {valid: false, opts: options{showActivity: true, tsStart: "2021-01-32"}}, // invalid report start timestamp {valid: false, opts: options{showActivity: true, filter: `colname:"["`}}, // invalid regexp + {valid: false, opts: options{showWAL: "-f"}}, // unmapped -W value, see Test_selectReport_WALWhitelistIsClosed } for _, tc := range testcases { @@ -43,7 +47,8 @@ func Test_selectReport(t *testing.T) { {opts: options{showTables: true}, want: "tables"}, {opts: options{showIndexes: true}, want: "indexes"}, {opts: options{showFunctions: true}, want: "functions"}, - {opts: options{showWAL: true}, want: "wal"}, + {opts: options{showWAL: "w"}, want: "wal"}, + {opts: options{showWAL: "a"}, want: "archiver"}, {opts: options{showSizes: true}, want: "sizes"}, {opts: options{showStatements: "m"}, want: "statements_timings"}, {opts: options{showStatements: "g"}, want: "statements_general"}, @@ -72,6 +77,117 @@ func Test_selectReport(t *testing.T) { } } +// Test_selectReport_WALWhitelistIsClosed proves the -W mapping is a closed whitelist: only 'w' and +// 'a' map, everything else yields "". This matters because ReportType is not an inert label - it is +// the tar-entry filter in report.isFilenameOK and the key into the view map in report.newApp, so a +// value leaking through would select a zero-value view and print a silently empty report instead of +// erroring. +func Test_selectReport_WALWhitelistIsClosed(t *testing.T) { + testcases := []struct { + value string + why string + }{ + {value: "c", why: "valid for -J (pg_stat_io count) - a user mixing up flags would get a wal report"}, + {value: "t", why: "valid for -J (pg_stat_io time) and is also -t (strlimit) - same mix-up"}, + {value: "g", why: "valid for -D and -X - and is also -g (grep) shorthand"}, + {value: "W", why: "case variant of the flag letter itself - no case normalisation is done on purpose"}, + {value: "wal", why: "spelled-out report type - no full-word aliases on purpose"}, + {value: "archiver", why: "spelled-out report type - no full-word aliases on purpose"}, + {value: "x", why: "arbitrary typo - the plain unknown-value case"}, + {value: "-f", why: "the value pflag assigns on the legacy 'pgcenter report -W -f dump.tar' invocation"}, + {value: "w ", why: "trailing whitespace - no trimming is done on purpose, so a quoted '-W \"w \"' must not map"}, + {value: " a", why: "leading whitespace - the other half of the no-trimming rule; both sides must stay closed"}, + } + + for _, tc := range testcases { + assert.Equal(t, "", selectReport(options{showWAL: tc.value}), "value %q must not map: %s", tc.value, tc.why) + } + + // The exact error message users see on the legacy '-W -f dump.tar' shape; it is quoted verbatim + // in the release notes. Test_options_validate's table has no field for a message, so the literal + // is pinned here. + _, err := options{showWAL: "-f"}.validate() + assert.EqualError(t, err, "report type is not specified, quit") + + // The mapped values must survive validate() into Config.ReportType unchanged. selectReport being + // correct is not enough on its own: ReportType is what keys the view map and filters tar entries, + // so anything rewriting it in between (a stray 'archiver' -> 'wal' alias, say) is the same + // silently-wrong-report failure the whitelist exists to prevent. + cfg, err := options{showWAL: "a"}.validate() + assert.NoError(t, err) + assert.Equal(t, "archiver", cfg.ReportType) + + cfg, err = options{showWAL: "w"}.validate() + assert.NoError(t, err) + assert.Equal(t, "wal", cfg.ReportType) +} + +// Test_selectReport_WALPrecedence pins both boundaries of the -W arm's slot in the first-match-wins +// switch chain: it must stay below showFunctions and above showBgwriter. Asserting only that -A wins +// would leave the arm free to move anywhere below showActivity without reddening a test. +func Test_selectReport_WALPrecedence(t *testing.T) { + testcases := []struct { + opts options + want string + why string + }{ + {opts: options{showActivity: true, showWAL: "a"}, want: "activity", why: "-A heads the chain and must keep beating -W"}, + {opts: options{showDatabases: "g", showWAL: "a"}, want: "databases_general", why: "the arm may not rise above showDatabases"}, + {opts: options{showFunctions: true, showWAL: "a"}, want: "functions", why: "the arm may not rise above showFunctions, its upper neighbour"}, + {opts: options{showWAL: "a", showBgwriter: true}, want: "archiver", why: "the arm may not sink below showBgwriter, its lower neighbour"}, + } + + for _, tc := range testcases { + assert.Equal(t, tc.want, selectReport(tc.opts), "flag precedence changed: %s", tc.why) + } +} + +// Test_walFlagDefinition guards the user-visible shape of -W: its type, shorthand, default and the +// help text printed by 'pgcenter report --help'. It is also what keeps the local FlagSet mirror in +// Test_walFlagPflagFailureShapes honest. +func Test_walFlagDefinition(t *testing.T) { + f := CommandDefinition.Flags().Lookup("wal") + // require, not assert: the assertions below dereference f, so a renamed flag must stop this test + // rather than panic and take the rest of the package's tests down with it. + require.NotNil(t, f) + assert.Equal(t, "string", f.Value.Type()) + assert.Equal(t, "W", f.Shorthand) + assert.Equal(t, "", f.DefValue) + assert.Equal(t, "show pg_stat_wal / pg_stat_archiver report (w - wal, a - archiver)", f.Usage) + // NoOptDefVal is what decides whether -W demands an argument at all, so it is the property the + // pflag mirror below actually rests on. A non-empty value is the cobra shim Decision 7 rejected: + // it keeps bare '-W' working while making '-W a' silently drop the 'a' and report wal. + assert.Equal(t, "", f.NoOptDefVal, "-W must demand an argument; a non-empty NoOptDefVal is the rejected shim from Decision 7") +} + +// Test_walFlagPflagFailureShapes pins the two user-visible failure shapes of the breaking change +// from a boolean -W to a string one. Parsing runs through a locally constructed FlagSet that mirrors +// the -W and -f definitions - CommandDefinition's flags are bound to the package-level opts var, so +// parsing through them would mutate shared state and make the suite order-dependent. +// Test_walFlagDefinition is what keeps this mirror honest. +func Test_walFlagPflagFailureShapes(t *testing.T) { + newMirror := func() (*pflag.FlagSet, *string) { + fs := pflag.NewFlagSet("report", pflag.ContinueOnError) + fs.SetOutput(io.Discard) + wal := fs.StringP("wal", "W", "", "show pg_stat_wal / pg_stat_archiver report (w - wal, a - archiver)") + fs.StringP("file", "f", "pgcenter.stat.tar", "read stats from file") + return fs, wal + } + + // '-W' as the last token: pflag errors out before RunE ever runs. The message is pflag's own. + fs, _ := newMirror() + assert.EqualError(t, fs.Parse([]string{"-W"}), "flag needs an argument: 'W' in -W") + + // '-W -f dump.tar' - the shape legacy scripts have. A string flag consumes the next token + // unconditionally, so '-f' lands in -W's value even though -f is itself a defined flag: there is + // no flag error at all and the whitelist is the only thing that catches it. ('-f' is defined in + // the mirror to keep the scenario realistic, not because the outcome depends on it.) + fs, wal := newMirror() + assert.NoError(t, fs.Parse([]string{"-W", "-f", "dump.tar"})) + assert.Equal(t, "-f", *wal) + assert.Equal(t, "", selectReport(options{showWAL: *wal})) +} + func Test_setReportInterval(t *testing.T) { today := time.Now().Format("2006-01-02") diff --git a/doc/release-notes/v0.12.0.md b/doc/release-notes/v0.12.0.md new file mode 100644 index 00000000..c5444d13 --- /dev/null +++ b/doc/release-notes/v0.12.0.md @@ -0,0 +1,76 @@ +## Release 0.12.0 + +Release date: TBD + +This release changes the `-W` flag of `report` utility in a way that breaks existing scripts, and +carries one known replay limitation for statistics recorded on Postgres 19. + +### Overview + +Pgcenter 0.12.0 contains the following user-visible changes: +- `-W` flag of `report` utility is now a string flag and requires a report selector: `-W w` or `-W a` +- `wal` statistics recorded on Postgres 19 by pgcenter older than 0.12 no longer replay +- archiving backlog in the verbose panel of `top` utility is now visible to `pg_monitor` roles + +The above items are explained in more detail in the sections below. + +### Breaking changes + +1. `-W` flag of `report` utility requires a report selector. + +Until 0.12.0 `-W`/`--wal` was a boolean flag which selected the `pg_stat_wal` report. Now it is a +string flag which takes a selector: `-W w` shows the `pg_stat_wal` report, `-W a` shows the new +`pg_stat_archiver` report. Scripts which pass a bare `-W` have to be updated: + +``` +pgcenter report -W -f pgcenter.stat.tar # 0.11.0 and earlier +pgcenter report -W w -f pgcenter.stat.tar # 0.12.0 +``` + +The old invocation does not complain about `-W`. The command line parser takes the next token as the +value of the flag, so `pgcenter report -W -f dump.tar` is understood as `-W` with the value `-f`, the +report type stays unknown, and the command prints: + +``` +report type is not specified, quit +``` + +That message mentions neither `-W` nor `-f`, and says nothing about a flag having changed its type — +if you see it after upgrading, a bare `-W` is the reason. The same message is printed for any +selector other than `w` and `a`, e.g. `pgcenter report -W x`. + +**The failure is loud on a terminal and silent to a script.** pgcenter prints the message and exits +with code **0**, so a wrapper like: + +``` +pgcenter report -W -f dump.tar > wal-report.txt || alert +``` + +never fires the alert and leaves an empty `wal-report.txt` behind. Grep your recording and reporting +scripts for a bare `-W` before upgrading, rather than waiting for a report to come out empty. + +The other shape of the mistake, `-W` as the last token on the command line, fails earlier and names +the flag: `flag needs an argument: 'W' in -W`. The exit code is 0 in that case too. + +### Known limitations + +1. A `wal` recording made on Postgres 19 by pgcenter older than 0.12 does not replay. + +Postgres 19 added a `wal_fpi_bytes` column to `pg_stat_wal`, and 0.12 reads that column. A recording +made **on a Postgres 19 cluster** by pgcenter 0.11.0 or earlier holds the older, narrower set of +columns, while 0.12 replays it against the new layout — the diffed range no longer matches, and +`pgcenter report -W w` fails with an error beginning `diff failed`, followed by the details of the +mismatch. + +This is a known and accepted limitation, not a bug to wait out: the practical answer is to re-record +the statistics with 0.12. Recordings made on Postgres 14 through 18 are unaffected — the column +layout is chosen from the Postgres version stored in each sample, and those layouts did not change. +Postgres 19 is still in beta, which is why this narrow case is documented rather than fixed. + +### Other + +- Archiving backlog in the verbose panel (`v` shortcut) of `top` utility is now read through + `pg_ls_archive_statusdir()` instead of `pg_ls_dir()`. `pg_ls_dir()` is available to superusers only, + so a role holding just `pg_monitor` — the usual role for monitoring — never saw the backlog and got + `n/a` in its place; now it sees a real value. As a side effect, on a cluster whose `archive_status` + directory is missing the panel reports a backlog of zero instead of `n/a`. diff --git a/docs/decisions-log.md b/docs/decisions-log.md index 3f91d557..a472af04 100644 --- a/docs/decisions-log.md +++ b/docs/decisions-log.md @@ -655,7 +655,7 @@ Used by tech-spec planning and code research to avoid repeating mistakes and re- **Date:** 2026-06-25 **Feature:** 010-feat-overview-dashboard -**Status:** Accepted +**Status:** Superseded in part by [017-feat-wal-archiver] "The verbose backlog moves off `pg_ls_dir`" — the aggregate's shape (count of `.ready` × `wal_segment_size`, its own `QueryRow`, degrade to `n/a`) stands; the **function choice and the privilege claim below do not**. Measurement on live PG 14 and PG 18 showed `pg_ls_dir` has ACL `{postgres}` — superuser only — so instead of degrading gracefully for `pg_monitor`, this aggregate never worked for that role at all, which is the most common monitoring role and the one the panel exists to serve. **Context:** The replication row needs a WAL-archiving backlog signal that works over the network (no PL/Perl) and degrades cleanly when archiving is off. @@ -1114,3 +1114,205 @@ is an invariant defended by inspection alone. **Alternatives considered:** Documenting the invariants in comments and relying on review (this is what failed twice). A fake `*gocui.Gui` (not possible outside the package). Deferring everything to the stand run (too coarse — the stand cannot isolate a single branch). + +--- + +## [017-feat-wal-archiver] `has_function_privilege()` cannot guard a privileged call — measured, not assumed + +**Date:** 2026-08-06 +**Feature:** 017-feat-wal-archiver +**Status:** Accepted + +**Context:** The `archiver` screen calls `pg_ls_archive_statusdir()`, which needs superuser or +`pg_monitor`. The obvious way to keep the rest of the screen alive for a role without it is to hide +the privileged call behind `has_function_privilege()` and let the column degrade instead of the +screen. + +**Decision:** Do not attempt it in SQL. The privileged call is unconditional and a role without +`pg_monitor` loses the whole screen, which then retries next tick — the same shape the `wal` screen +already has with its unconditional `pg_ls_waldir()`. + +**Rationale:** PostgreSQL checks EXECUTE at **function-node initialisation**, not when a row would +need the value, so every guarding form fails alike: `CASE` with an uncorrelated subquery (which +becomes an InitPlan and is evaluated first), `CASE` with a correlated subquery, and `LEFT JOIN +LATERAL … ON has_function_privilege(...)`. All three were run on a live PG 18 under a purpose-made +privilege-less role and all three raised the permission error. This closes a whole class of future +attempts: the guard has to live outside the statement, not inside it. + +**Alternatives considered:** two query variants of equal width, chosen in Go from a connect-time +privilege probe — technically sound and the only surviving option, rejected here because it would fix +half the WAL area for a role that cannot use the other half anyway. If per-column degradation is ever +wanted, it must be done for both screens at once, as its own change. + +--- + +## [017-feat-wal-archiver] The verbose backlog moves off `pg_ls_dir` to `pg_ls_archive_statusdir()` + +**Date:** 2026-08-06 +**Feature:** 017-feat-wal-archiver +**Status:** Accepted +**Supersedes (in part):** [010-feat-overview-dashboard] "Archiving backlog via `count(.ready) × +wal_segment_size`" — its function choice and its claim about `pg_monitor`. + +**Context:** ADR [010] assumed `pg_monitor` was enough to run the archiving-backlog aggregate and +that insufficient privileges would merely degrade the field to `n/a`. + +**Decision:** `OverviewArchivingBacklog` counts `.ready` entries via `pg_ls_archive_statusdir()`. +The output (bytes) and the degrade-to-`n/a` path are unchanged. + +**Rationale:** The assumption was wrong, and measurement on live PG 14 and PG 18 is what showed it: +`pg_ls_dir` has ACL `{postgres}` — superuser only — while `pg_ls_waldir` and +`pg_ls_archive_statusdir` are `{postgres, pg_monitor}`. So the most common monitoring role saw `n/a` +permanently and never got the first signal that archiving had stopped. Two consequences are accepted +knowingly: the new function is `missing_ok=true`, so a cluster whose `archive_status` directory is +gone now reports a confident `0` instead of `n/a` (a damaged data directory, where the backlog is the +least of the operator's problems) — and it renders as a bare `0`, not `0 B`, because the size +formatter's zero case returns the digit alone. And it returns `SETOF record`, stating every file, +where `pg_ls_dir` returned names only; since the verbose panel rides every screen, that walk is now +paid on every screen. Showing the signal is worth more than saving the walk. + +**Alternatives considered:** leaving the panel alone and correcting the user-spec's framing — rejected +by the roadmap owner in favour of fixing the signal. Doing it as a separate task later — rejected: the +roadmap's mandate was to enter the WAL area exactly once. + +--- + +## [017-feat-wal-archiver] The report always discards the first sample, so a pass-through screen loses real data + +**Date:** 2026-08-06 +**Feature:** 017-feat-wal-archiver +**Status:** Accepted + +**Context:** `report -W a` over an N-tick recording prints N−1 rows. The user-spec had promised one +row per tick. + +**Decision:** Accept it and document it rather than fix it. The user-spec's scenario was corrected to +"one row per tick, except the first". + +**Rationale:** The replay loop drops the first sample of a run because a **diffed** screen has nothing +to diff it against. For a screen with `DiffIntvl{0,0}` the first sample *is* printable data and is +dropped anyway. This is not introduced here — every pass-through screen behaves this way today, +`activity` included — and it is worth recording because the reasoning does not survive contact with +this class of screen: whoever next adds a `{0,0}` screen will lose a row and should know it is +expected, not a bug in their view. + +**Alternatives considered:** skip the discard when `DiffIntvl == {0,0}` — rejected as surgery on a +shared path with golden churn across unrelated screens; if it is ever done, it is its own change with +its own review. + +--- + +## [017-feat-wal-archiver] An archive with no matching entries prints nothing at all — no rows, no header + +**Date:** 2026-08-06 +**Feature:** 017-feat-wal-archiver +**Status:** Accepted + +**Context:** The user-spec promised "header only" for a report over an archive containing no samples +of the requested screen. + +**Decision:** No data rows and no column header — only the three INFO lines every report emits at +start-up — and exit 0. The user-spec's edge case and its acceptance criterion were corrected to match +the code. + +**Rationale:** The header cannot be printed: `printStatHeader` returns early unless the view has been +aligned, and alignment happens inside the data branch, so with no samples there is nothing to align +from and nothing is printed. A "no data" notice exists for exactly one screen, `procpidstat`, and +giving a second screen one would introduce behaviour no other screen has. Recorded because the +question ("shouldn't it at least say something?") will be asked again for the next screen. + +**Alternatives considered:** an INFO line for empty reports — rejected as inconsistent with every +other screen; making it consistent across all screens is its own change. + +--- + +## [017-feat-wal-archiver] Report flag values are a closed whitelist, because `ReportType` is not a label + +**Date:** 2026-08-06 +**Feature:** 017-feat-wal-archiver +**Status:** Accepted + +**Context:** `-W` became a string flag (`w`/`a`), joining `-J`, `-D`, `-P`, `-X` in taking a +sub-selector value. The question is what an unrecognised value should do. + +**Decision:** map only the known letters, with no `default` arm; anything else falls through and the +command exits with `report type is not specified, quit`. Tests cover other flags' letters (`c`, `t`, +`g`) explicitly, not just one arbitrary unknown string. + +**Rationale:** `ReportType` is load-bearing, not an inert label — it is the tar-entry filter in +`isFilenameOK` and the key into the view map. A value leaking through would select a zero-value +`view.View` and produce a **silently empty report** rather than an error, which is the worst possible +outcome for a report tool: a clean exit that shows nothing. Failing closed keeps the flag family +consistent — `-D`/`-J` already behave this way. + +**Alternatives considered:** defaulting an unrecognised value to `wal` — rejected: it would silently +run a different report than the operator asked for. + +--- + +## [017-feat-wal-archiver] One shared test-role helper, and it must not take `*testing.T` + +**Date:** 2026-08-06 +**Feature:** 017-feat-wal-archiver +**Status:** Accepted + +**Context:** Two packages (`internal/query` and `internal/stat`) needed to prove privilege behaviour +in both directions — a query succeeds under a `pg_monitor`-only role and fails without it. That needs +`CREATE ROLE`/`GRANT`/`SET ROLE`, of which the tree contained none, and the test image is deliberately +frozen so the roles cannot be baked into the fixtures. + +**Decision:** one helper, `postgres.SetupTestRole`, in `internal/postgres/testing.go` — the existing +shared home for test helpers, imported by both packages. It creates the role idempotently at test +time, optionally grants `pg_monitor`, and the callers always `RESET ROLE` afterwards. + +**Rationale:** the wave-conflict analysis compared *files* and missed that two tasks were adding +package-level helpers to the **same Go package** — two identically-named helpers do not compile, and +two differently-named copies are duplication a reviewer would rightly reject. The non-obvious +constraint, and the reason this is worth an ADR: `internal/postgres/testing.go` has **no build tag**, +so it links into the production binary. The helper therefore returns an `error` and must never take a +`*testing.T` or import `testing` — the natural signature for a test helper is the one thing that file +cannot have. Roles are not dropped afterwards: reusability across repeated runs is the point, correct +for ephemeral CI containers and worth knowing on a long-lived cluster. + +**Alternatives considered:** per-package helpers (duplicated logic in one package); baking the roles +into the test image (needs an image bump and changes what every existing test sees); skipping the +privilege tests and relying on the stand — rejected, that is exactly the gap that let the wrong +`pg_ls_dir` privilege assumption survive into ADR [010] unnoticed. + +--- + +## [017-feat-wal-archiver] The `archive_status` walk was measured and left unthrottled + +**Date:** 2026-08-06 +**Feature:** 017-feat-wal-archiver +**Status:** Accepted + +**Context:** The `.ready` listing runs every tick, in the TUI and in `pgcenter record`, and after the +backlog function swap the verbose panel pays an `lstat` walk of the same directory on every screen. +The screen is opened exactly when that directory is largest. The outcome was agreed in advance so the +measurement could not end in a shrug: bad numbers mean throttling, acceptable numbers mean the +remainder is written down. + +**Decision:** no throttling. The numbers live here so the next person asking "how expensive is that +directory walk" has an answer without re-measuring. + +**Rationale:** measured on the stand under the conditions the design fixed — a `pg_monitor`-only role +(for a superuser the comparison would be 1 walk → 2; for `pg_monitor` it is genuinely 0 → 1, because +the old aggregate failed instantly), verbose on, a concurrent `pgcenter record`, 200 005 `.ready` +files. Backlog query: ~1108 ms mean, against 0.9 ms on an empty directory. View-switch latency: +70–240 ms, i.e. no input lag. Effective refresh with verbose on: 1.9 s/tick against 1.0 s/tick for a +`master`-built binary — at that size the feature halves the refresh rate, on every screen. It is +accepted anyway because the cost is linear at ~5.5 µs per file: 5 000 segments ≈ 28 ms, 20 000 ≈ +110 ms, both inside the noise. The doubling needs 200 000 segments — **3.1 TB of unarchived WAL** — a +state that would be noticed long before, and in which a two-second refresh is nowhere near the +operator's biggest problem. + +**Deliberately not recorded as tech debt.** The roadmap owner declined: a debt entry is a commitment +to fix, and there is no intention to fix this. If it is ever reopened, the machinery already exists — +[010]'s `verboseCollectState` + `latencyGuardThreshold`, today used only for the DB-size aggregate — +and it would need applying in two places: the verbose panel's aggregate (which is what drops the +refresh rate, since it runs on every screen) and the screen's own `.ready` sub-select. + +**Alternatives considered:** applying the latency guard now — briefly started, then stopped once the +per-file cost was put against realistic backlog sizes; reverting the panel to `pg_ls_dir` — rejected, +it restores the `n/a` this part of the feature exists to remove. diff --git a/docs/features-catalog.md b/docs/features-catalog.md index e60bb130..d8bb9433 100644 --- a/docs/features-catalog.md +++ b/docs/features-catalog.md @@ -416,3 +416,66 @@ with the timestamp of that moment, and it holds indefinitely. token (`[PAUSED]`) to the command-line composer it introduced, left of the filter indicator. [009-feat-horizontal-scroll] — its column scroll and width keys are the ones that keep working on a frozen frame. + +--- + +### [017-feat-wal-archiver] Archiver Screen, FPI on PG 19, and `report -W w|a` + +**What it does:** Adds an `archiver` screen — one row, nine columns over `pg_stat_archiver` plus a +live count of `.ready` segments waiting to be archived — so the question "has archiving stopped, when, +on which segment, and how much has piled up" is answered in one look instead of in `psql`. It is +reached by pressing `w` a second time (the hotkey now cycles `wal` ↔ `archiver`) or from the new `W` +menu, and it is recorded and replayed like any other screen. Two smaller things ship in the same pass +over the WAL area: on PostgreSQL 19 the `wal` screen gains a `fpi,KiB` column showing how much WAL the +full-page images actually cost, and the verbose panel's archiving backlog now works for a plain +`pg_monitor` role instead of showing `n/a` to everyone but a superuser. + +**Key scenarios:** +- During an incident: `w`, `w` — read `ready` (how many segments are queued), `failed` climbing tick + by tick, `last_failed` (the segment to grep for in the PostgreSQL log) and `archived_age` (how long + since the last success). Decide whether the disk gives you hours or minutes. +- After fixing `archive_command`: the same screen shows `archived` rising, `ready` draining and + `failed` no longer moving — the incident closes on evidence rather than on faith. +- On an unfamiliar cluster: counters at zero with **blank** name and age cells, and a screen caption + that says `requires archive_mode=on`, so archiving that was never configured does not read as a + quiet healthy cluster. +- Post-mortem from a recording: `pgcenter report -W a -f night.tar -s 02:00 -e 06:00` prints one row + per tick and pins the minute the backlog started growing. `pgcenter report -d -W a` describes the + columns. +- On PG 19, sort the `wal` screen's `fpi` and `fpi,KiB` side by side to decide about `wal_compression` + and checkpoint spacing. + +**Limitations:** +- **Breaking CLI change:** `report -W` is no longer a boolean — it takes `w` or `a`. Old invocations + fail rather than change meaning, but the most common legacy shape (`report -W -f dump.tar`) fails + with `report type is not specified, quit`, because pflag takes `-f` as the flag's value. Worse for + scripts than it looks: like every other report failure, it still exits 0. +- The whole screen needs `pg_monitor` (or superuser). Without it the screen shows the PostgreSQL + permission error and retries next tick — it does not fall back to eight columns. `pgcenter record` + under such a role stops recording entirely, which is the recorder's existing behaviour. +- PostgreSQL 14 and newer only. +- The report drops the first sample of a run, so an N-tick recording prints N−1 rows of a screen that + diffs nothing. Pre-existing for every pass-through screen, `activity` included. +- A recording that contains no archiver samples prints no rows and no column header — only the report's + own three information lines — and exits 0. There is no "no data" notice; exactly one screen + (`procpidstat`) has one. +- A `wal` recording made **on PG 19 by a pre-0.12 pgcenter** replays against the new 8-column layout + and fails with `diff failed: …`. Narrow (PG 19 is still beta) and documented in the release notes + rather than fixed. +- The `.ready` directory is listed on every tick, in the TUI and in `record`, with no throttling. The + cost is ~5.5 µs per file: unnoticeable at a five-thousand-segment backlog, and it halves the refresh + rate at 200 000 (≈3.1 TB of unarchived WAL). Since the verbose panel rides every screen, that cost is + paid on every screen when `v` is on. +- On a cluster whose `archive_status` directory is missing, the verbose panel now shows `0` instead of + `n/a` — the new function tolerates a missing directory where the old one errored. +- The `archive_mode=on` caption is static: it is printed whether archiving is off or working perfectly, + and reads no GUC. `Q` still does not reset `pg_stat_archiver`. + +**Touches:** [010-feat-overview-dashboard] — the verbose panel's archiving backlog is the earlier, +coarser signal (bytes, not segments) and now uses the same function as this screen, so the two numbers +cannot disagree; its ADR about privileges is superseded here. [008-feat-record-report-0-11-views] — +record/report is folded in from day one under that feature's pure-SQL rule, and the `-J c|t` flag shape +is what `-W w|a` copies. [006-feat-pg-stat-io] — the `j`/`J` cycle-plus-menu machinery is the model for +`w`/`W`. [012-feat-pg19-compatibility-baseline] — the PG 19 catch-up column deliberately left to this +feature so the WAL area was entered once. [009-feat-horizontal-scroll] — the `source` column stays +frozen while the two 24-character WAL names are reached by scrolling on a narrow terminal. diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch1.json b/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch1.json deleted file mode 100644 index 403651e6..00000000 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch1.json +++ /dev/null @@ -1,97 +0,0 @@ -{ - "validator": "dev-reality-checker", - "feature_base": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver", - "tasks_checked": ["01", "02", "03"], - "status": "changes_required", - "findings": [ - { - "severity": "critical", - "category": "feasibility", - "task": "01", - "issue": "Task 01 contradicts itself about the shared role helper required by tech-spec Decision 19. The Context Files section says '[internal/postgres/testing.go] — MODIFY: add the single shared role-creation / SET ROLE test helper that both this task's privilege tests and Task 03's use (tech-spec Decision 19)'. Every other section forbids exactly that: Description says 'this task creates exactly two new files and touches nothing that exists'; What-to-do says 'Do not touch internal/view/, top/, record/, report/ or any existing file'; the Details 'Files:' block lists only internal/query/archiver.go and internal/query/archiver_test.go; and the first Acceptance Criterion is 'internal/query/archiver.go and internal/query/archiver_test.go exist; no other file in the tree is modified.' An executor obeying the ACs never creates the helper, and Task 03 (depends_on: [\"01\"]) has nothing to reuse. The implementation hint 'Idempotent creation is a DO $$ … $$ block … run it through conn.Exec' also reads as inline-in-the-test, reinforcing the wrong reading.", - "fix": "Make the helper a first-class deliverable of Task 01: add a 'What to do' bullet naming the helper in internal/postgres/testing.go, add the file to the Details 'Files:' block, and change the first AC to 'internal/query/archiver.go and internal/query/archiver_test.go are created and internal/postgres/testing.go gains the shared role helper; no other file is modified.' Add an AC pinning the helper name so Task 03 can reference it." - }, - { - "severity": "critical", - "category": "feasibility", - "task": "03", - "issue": "Stale wave/dependency text contradicts the frontmatter and the task's own header note. Frontmatter is depends_on: [\"01\"], wave: 2, and the header block says 'Moved from Wave 1 to Wave 2 (tech-spec Decision 19) ... Task 01 now owns a single shared helper in internal/postgres/testing.go; this task reuses it and must not define its own.' But Details says: 'Dependencies: none — depends_on: []. No new Go packages. Wave 1, alongside Tasks 1, 2 and 4.' Context Files also still cites the tech-spec as '(Task 3 in Wave 1; Decisions 8, 11, 18)' — Decision 19 is not referenced anywhere in the Context Files.", - "fix": "Rewrite the Details 'Dependencies' line to 'depends_on: [\"01\"] — Wave 2; Task 01 owns the shared role helper in internal/postgres/testing.go.' Update the Context Files tech-spec line to '(Task 3, Wave 2; Decisions 8, 11, 18, 19)' and add internal/postgres/testing.go as the helper source." - }, - { - "severity": "critical", - "category": "feasibility", - "task": "03", - "issue": "The 'Coordination risk' paragraph instructs the executor to build a competing helper, which is precisely what Decision 19 exists to prevent. It reads: 'If a suitable helper already exists in the package when you start, reuse it; otherwise give yours an overview-specific name. ... internal/stat is a separate package and needs its own helper.' Decision 19 states the helper lives ONCE in internal/postgres/testing.go so that one helper serves internal/query and internal/stat alike. Following this paragraph produces two or three duplicated role-creation/SET ROLE implementations, directly contradicting the task's own header note four screens earlier.", - "fix": "Replace the whole 'Coordination risk' paragraph with: 'Task 01 added the shared role-creation / SET ROLE helper to internal/postgres/testing.go (Decision 19). Reuse it from both internal/query and internal/stat. Do NOT define a package-local helper in either package, and do not introduce a second set of role names.'" - }, - { - "severity": "major", - "category": "hallucination", - "task": "03", - "issue": "The edge case 'AS name must be dropped ... Keeping AS name renames the whole relation and the query fails with column \"name\" does not exist' is false. Verified live on PostgreSQL 17.10: `SELECT count(*) FILTER (WHERE name LIKE '%.ready') FROM pg_ls_waldir() AS name;` executes successfully and returns 0. pg_ls_waldir() has the identical OUT signature (name text, size bigint, modification timestamptz) to pg_ls_archive_statusdir(). Also confirmed with `SELECT count(*) FILTER (WHERE name LIKE 'Europe%') FROM pg_timezone_names() AS name;` (returns 64, same as without the alias) and against pg_settings. PostgreSQL resolves an unqualified identifier as a column reference before considering a whole-row reference, so a relation aliased `name` that has a column `name` still resolves `name` to the column. Dropping the alias remains the right cleanup, but the stated rationale is invented, and the second half of the same bullet ('pg_ls_dir(text) returns SETOF text with an unnamed column, so AS name was doing double duty') is correct only for the old function — it does not imply the new one breaks.", - "fix": "Restate the bullet as a cleanup, not a failure: 'pg_ls_dir(text) returns SETOF text, so AS name supplied the column name the FILTER resolved against. pg_ls_archive_statusdir() already exposes a name column, so the alias is redundant and is dropped. (Keeping it would still execute — column references win over whole-row references — but it would be misleading.)' Do not build a test or AC on the non-existent failure." - }, - { - "severity": "major", - "category": "feasibility", - "task": "03", - "issue": "Internal contradiction between the Verification Steps and the Implementation hints. Verification says: 'Grep the tree for the stale claims — no hit may remain: grep -rn \"pg_ls_dir\" internal/'. The implementation hint says the rewritten doc comment should carry '...which superuser and pg_monitor can execute (unlike pg_ls_dir, which is superuser-only — worth one clause, since that is the bug being fixed and Decision 8 supersedes ADR [010])'. Both cannot be satisfied. Current tree has 5 pg_ls_dir hits under internal/ (overview.go:95, :97, :102; stat/postgres.go:288; query/overview_test.go:124), all inside files this task owns, so the grep is achievable — but only by dropping the explanatory clause the hints ask for.", - "fix": "Narrow the grep gate to the stale claim rather than the token, e.g. `grep -rn \"pg_ls_dir requires\\|pg_ls_dir, which requires\" internal/` and `grep -rn \"has pg_monitor\" internal/`, and keep the one explanatory clause about pg_ls_dir being superuser-only." - }, - { - "severity": "major", - "category": "security", - "task": "01", - "issue": "Neither task constrains the shared helper's placement semantics, and internal/postgres/testing.go is not a test file: it carries no build tag and does not end in _test.go, so it is compiled into the shipped pgcenter binary (this is already true of NewTestConfig/NewTestConnect/NewTestConnectVersion). Consequences the task files do not address: (a) if the helper takes *testing.T it drags the `testing` package into the production build; (b) a helper that issues CREATE ROLE / GRANT pg_monitor ships in the released binary; (c) role names must be compile-time constants — identifiers cannot be parameterized in SQL, so a caller-supplied name would be raw string concatenation into DDL. The existing file's functions take no *testing.T, which is the precedent to follow but is never stated.", - "fix": "Add to Task 01's Details: the helper must not take *testing.T (return an error; callers assert with require/assert), role names are unexported package constants with a pgcenter_test_ prefix, no caller-supplied identifier is ever concatenated into the DDL, and the roles are created NOLOGIN with no password and nothing granted beyond pg_monitor." - }, - { - "severity": "major", - "category": "hallucination", - "task": "02", - "issue": "Task 02 states: 'internal/view/view_test.go belongs to Task 05, which adds the wal assertions to TestViews_Configure.' Task 05 states the opposite — it describes TestViews_Configure as asserting 'only on progress/replication/activity screens and has no wal or archiver assertions', and its What-to-do/TDD Anchor add only TestNew_ArchiverView, the TestNew count 27->28 and the TestView_VersionOK rows. Verified against the code: `grep -n 'wal' internal/view/view_test.go` returns nothing. Net effect — the PG 19 wal layout is pinned only inside internal/query; no task pins that view.Configure actually propagates Ncols=8/DiffIntvl{2,6} into the registered wal view.", - "fix": "Either delete the false forward-reference from Task 02, or assign the assertion explicitly: add a TestViews_Configure row at version 190000 asserting views[\"wal\"].Ncols == 8 and DiffIntvl == [2]int{2,6}, and name the owning task (Task 05, since it owns view_test.go)." - }, - { - "severity": "minor", - "category": "hallucination", - "task": "01", - "issue": "Edge-cases section claims 'every printCmdline call in the tree passes an explicit \"%s\" verb'. False — many call sites pass a bare literal format string with no arguments: top/extra.go:44, :64, :71; top/dialog.go:144, :282; top/pglog.go:16, :22; top/ui.go:304. The conclusion the claim supports (the SQL literal '%.ready' is safe) is nevertheless correct for an unrelated reason: query strings are never passed through printf-style formatting, and query.Format is text/template which reacts only to {{ }} (internal/query/query.go:91-104, verified).", - "fix": "Replace with the accurate argument: query constants are never printf-formatted; query.Format is text/template and only {{ }} is meaningful, so % in the SQL literal is inert. The identical literal already lives in OverviewArchivingBacklog." - }, - { - "severity": "minor", - "category": "hallucination", - "task": "01", - "issue": "Citation 'testing/prepare-test-environment.sh:17-33' is offered as evidence for 'archive_mode=off, no archive_command'. Lines 17-33 are the postgresql.auto.conf heredoc and mention neither setting; `grep -n 'archive_mode\\|archive_command' testing/prepare-test-environment.sh` returns nothing at all. The claim is true (both are left at their defaults, archive_mode=off) but rests on absence, not on that block.", - "fix": "Reword to 'the fixture setup script sets neither archive_mode nor archive_command, so both keep their defaults (archive_mode=off)' and drop the misleading line range." - }, - { - "severity": "minor", - "category": "hallucination", - "task": "02", - "issue": "Details says 'internal/query/wal.go (33 lines today)'. The file is 32 lines (wc -l). Every other line reference in Task 02 checks out exactly: PgStatWALPG14 at :5-11, PgStatWALDefault at :15-21, SelectStatWALQuery at :25-32, wal_test.go 56 lines, Test_SelectStatWALQuery at :10-31 with the 190000 row at :21, Test_StatWALQueries at :34-56 with the version list at :35, SelectStatBgwriterQuery at bgwriter.go:41-52, PostgresV19 at query.go:22.", - "fix": "Change to '32 lines today', or drop the line count." - }, - { - "severity": "minor", - "category": "hallucination", - "task": "01", - "issue": "Three line ranges overshoot by one to three lines. internal/postgres/postgres.go Exec/Query/QueryRow cited as ':119-133' — the three wrappers span 118-131. internal/stat/postgres.go calculateDelta short-circuit cited as ':589-597' — the `if interval != [2]int{0,0}` guard is at :590 (:589 is its comment). Task 03 cites ADR [010] at docs/decisions-log.md:654-666 — the heading is at :653. All point at the right code; purely cosmetic drift.", - "fix": "Optional: tighten the ranges. No functional impact." - }, - { - "severity": "minor", - "category": "tdd", - "task": "03", - "issue": "The Acceptance Criterion 'A cluster whose archive_status directory is absent now reports 0 B instead of n/a (pg_ls_archive_statusdir() is missing_ok=true)' is not checkable by anything this task builds — the fixture clusters always have the directory and the task is explicitly forbidden from touching the fixtures. It sits in a list whose every other item names a mutation and the test it reddens, so it reads as a gate when it is a documentation item (its user-facing half is Task 09).", - "fix": "Move it out of Acceptance Criteria into the Details 'Edge cases' block (where the same fact already appears), or mark it explicitly as 'documented, not test-gated — carried by Task 09'." - } - ], - "stats": { - "tasks_checked": 3, - "claims_verified": 58, - "issues_found": 12 - } -} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch2.json b/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch2.json deleted file mode 100644 index f54806c6..00000000 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch2.json +++ /dev/null @@ -1,90 +0,0 @@ -{ - "validator": "dev-reality-checker", - "feature_base": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver", - "tasks_checked": ["04", "05", "06"], - "status": "changes_required", - "findings": [ - { - "severity": "critical", - "category": "feasibility", - "task": "04", - "issue": "Acceptance Criterion 'Exactly the four known showWAL sites changed; `grep -rn showWAL --include=*.go .` returns four hits and no more' cannot be satisfied after the task is implemented. Four hits is the CURRENT state (verified: cmd/report/report.go:25,70,151 and cmd/report/report_test.go:46). After the change production grows to four lines in report.go alone (field, flag, `case opts.showWAL != \"\":`, `switch opts.showWAL`), and the mandated TDD Anchor adds ~12 more test references (2 rows in Test_selectReport, 8 rows in Test_selectReport_WALWhitelistIsClosed, 1 in Test_selectReport_WALPrecedence, 1 in Test_options_validate). The grep will return ~16. The Verification Steps hedge this ('count *sites*'), but the AC checkbox is unconditional — an executor checking it literally is pushed toward deleting the very whitelist rows Decision 17 exists to protect.", - "fix": "Reword the AC to what is actually invariant: 'no production file outside cmd/report/report.go mentions showWAL, and cmd/report/report.go has exactly the three edited sites (field, flag registration, selectReport arm)'. Drop the numeric grep count, or scope it to `grep -rn showWAL --include=*.go . | grep -v _test.go`." - }, - { - "severity": "critical", - "category": "feasibility", - "task": "06", - "issue": "The `verify:` frontmatter ('bash — go test ./top/... (runs without PostgreSQL)'), Acceptance Criterion '`go test ./top/...` passes' and Verification Step 1 all assert that the top package tests pass on a host without PostgreSQL. They do not. Verified by running it on a clean tree: Test_getQueryReport (top/report_test.go:12-14) fails on a refused connection and then PANICS with a nil-pointer dereference in internal/postgres.(*DB).QueryRow (internal/postgres/postgres.go:125), which aborts the whole test binary and exits FAIL. The tests this task cares about (Test_switchViewTo, Test_statioNextView, Test_selectMenuStyle, the three help tests) do all run and pass BEFORE the panic, but the prescribed command can never report success — an automated wave gate running the `verify:` command will read the task as failed forever.", - "fix": "Change the verify command and the AC to a filtered run that genuinely passes on the host, e.g. `go test ./top/ -run 'Test_walNextView|Test_switchViewTo|Test_statioNextView|Test_selectMenuStyle|Test_helpTemplate'` (confirmed green), and state explicitly that the unfiltered `go test ./top/...` requires the CI image lesovsky/pgcenter-testing:0.0.11 — the same caveat Task 05 already carries for `make test`." - }, - { - "severity": "critical", - "category": "tdd", - "task": "06", - "issue": "The claim 'menuSelect cannot be unit-tested. Every branch ends in an unconditional menuClose(g, v), which calls g.DeleteView/g.SetCurrentView on a *gocui.Gui that only gocui.NewGui can produce' is FALSE — verified empirically against the existing menuStatIO branch. A zero-value &gocui.Gui{} is enough: gocui@v0.5.0 Gui.SetView (gui.go:130) builds a real *View with real dimensions without touching any terminal backend, so v.SetCursor(0,1) works and both cursor branches are reachable; DeleteView (gui.go:219) and SetCurrentView just scan a slice and return ErrUnknownView, they do not panic. A throwaway probe drove menuSelect(app)(app.ui, mv) for cy=0 and cy=1 and observed app.config.view.Name become 'stat_io' / 'stat_io_time', the correct view arriving on viewCh, and app.config.menu reset to menuNone — menuClose merely returns a benign 'unknown view' error the test asserts on. The pause_test.go:552-577 comment the task cites says only that a NIL Gui panics; it never considered the zero value, which top/ui_test.go:417 already uses (`layout(&app{config: c, ui: &gocui.Gui{}})`). Consequence: the W-menu path — one of the two user-facing halves of this task — is left with zero automated coverage on a false premise, and the task's own hint ('Before assuming anything in top/ is untested, grep top/*_test.go') is exactly the discipline it skips.", - "fix": "Delete the 'menuSelect cannot be unit-tested' edge case and add a real test to the TDD Anchor: top/menu_test.go::Test_menuSelect_WAL — build app{config: newConfig(), ui: &gocui.Gui{}}, set app.config.menu = selectMenuStyle(menuWAL), create the menu view with app.ui.SetView(\"menu\", 0, 5, 72, 6+len(items)), SetCursor(0, cy) for cy in {0,1}, read viewCh in a goroutine, call menuSelect(app)(app.ui, mv) and assert app.config.view.Name is 'wal' / 'archiver'. Add the corresponding mutation (swap the two cursor arms) to the Acceptance Criteria. Do not restore the forbidden pattern from pause_test.go — this test observes the real config mutation, so it can fail." - }, - { - "severity": "critical", - "category": "tdd", - "task": "06", - "issue": "The companion claim '`keybindings()` cannot be unit-tested either, for the same reason (it needs a live *gocui.Gui). The W-is-free claim is a static one: grep the table for W before and after' is also FALSE. Verified: keybindings(app) with app.ui = &gocui.Gui{} returns nil and sets InputEsc — gocui.SetKeybinding (gui.go:249) only appends to a slice. Registration is assertable through the exported DeleteKeybinding (gui.go:262), which returns nil for a registered binding and 'keybinding not found' otherwise. A probe confirmed today's state: 'J' deletes once then reports not found (registered exactly once), and 'W' reports not found (free). This turns the task's central 'W is free and claimed exactly once' assertion from a grep-and-trust into a regression guard that survives future edits to the keys table.", - "fix": "Replace the 'cannot be unit-tested' edge case with top/keybindings_test.go::Test_keybindings_WAL: assert.NoError(keybindings(&app{config: newConfig(), ui: &gocui.Gui{}})), then assert.NoError on the first DeleteKeybinding(\"sysstat\", 'W', gocui.ModNone) and assert.Error on the second (registered exactly once). Add the mutation 'remove the W row -> the test goes red' to the Acceptance Criteria." - }, - { - "severity": "major", - "category": "feasibility", - "task": "05", - "issue": "Step 6, the Acceptance Criterion '`go test ./internal/view/... ./record/...` passes on the host (no PostgreSQL needed for the count tests)' and the Verification Step that repeats it are wrong. Verified on a clean tree: internal/view passes, but the record package panics in Test_tarRecorder (record/recorder_test.go:37 -> internal/stat.GetPostgresProperties -> internal/postgres.(*DB).QueryRow, postgres.go:125) and the binary exits FAIL. The task's own guidance compounds this — it says 'the rest of the record package skips on a missing fixture', while the real behaviour is a panic that aborts the run. (Test_filterViews itself does run and pass before the panic, so the number can still be observed; only the prescribed pass/fail gate is unusable. patterns.md:154 is more accurate: 'skips/fails on a missing PG fixture'.)", - "fix": "Change the AC and Verification Step to `go test ./internal/view/...` plus `go test ./record/ -run 'Test_filterViews|TestFilterViews'` — both confirmed green on the host — and replace 'skips on a missing fixture' with 'panics and aborts the binary on a missing fixture, so read the Test_filterViews result before the panic, not the package exit code'." - }, - { - "severity": "major", - "category": "tdd", - "task": "05", - "issue": "The mutation expectation 'Set `NotRecordable: true` -> Test_filterViews must go red on every row (the view is dropped before the version gate)' is arithmetically wrong. On the four <= PG13 rows the archiver view is dropped either way — by NotRecordable instead of by the version gate — so wantN and wantV are identical under the mutation and those rows stay GREEN. The mutation reddens only the three >= PG14 rows ({190000,\"public\"}, {140000,\"\"}, {140000,\"public\"}), where wantV drops back from 28/19/25 to 27/18/24. Stating 'every row' invites the executor to conclude the mutation did not take effect and go hunting, or to 'strengthen' the <= PG13 rows to compensate.", - "fix": "Reword to: 'Set NotRecordable: true -> Test_filterViews goes red on the three >= PG14 rows only; the four <= PG13 rows stay green because the view is dropped there either way. That asymmetry is the point — it is what distinguishes the NotRecordable gate from the version gate.'" - }, - { - "severity": "major", - "category": "missing_file", - "task": "06", - "issue": "Acceptance Criterion and Verification Step 3 prescribe `go build ./cmd/pgcenter`. There is no such directory — `go build ./cmd/pgcenter` fails with 'stat .../cmd/pgcenter: directory not found'. The main package lives in ./cmd (cmd/pgcenter.go, cmd/help.go); the Makefile builds it as `go build ... -o bin/pgcenter ./cmd` (Makefile:40).", - "fix": "Replace `go build ./cmd/pgcenter` with `go build ./cmd` (or `make build`) in both the Acceptance Criteria and Verification Step 3." - }, - { - "severity": "major", - "category": "missing_file", - "task": "04", - "issue": "The 'Sanity by hand' verification step prescribes `go run . report -W 2>&1` and `go run . report -W -f /nonexistent.tar 2>&1`. Both fail before reaching the flag parser: the repository root has no Go files ('no Go files in /home/lesovsky/Git/github.com/lesovsky/pgcenter'). The main package is ./cmd — `go run ./cmd report -W` works (verified today it reaches report.RunMain and reports 'open pgcenter.stat.tar: no such file or directory', which is the pre-change bool behaviour).", - "fix": "Change both invocations to `go run ./cmd report -W` and `go run ./cmd report -W -f /nonexistent.tar`." - }, - { - "severity": "minor", - "category": "hints", - "task": "04", - "issue": "Details/Files item 1 claims 'The field currently sits in the bool column block of the struct; after the change gofmt re-aligns the comment column across the whole struct and the diff will carry whitespace-only lines. Let it — do not hand-align.' This will not happen. The options struct (cmd/report/report.go:15-41) already contains string fields (showDatabases, showStatIO, showStatements, showProgress), so gofmt's type and comment columns are already sized for 'string'; changing `showWAL bool` to `showWAL string` shifts nothing. Telling the executor that whitespace churn is 'expected' pre-authorises a diff that should not appear.", - "fix": "Drop the paragraph, or replace it with 'the struct already aligns to `string`, so the diff is exactly the two changed tokens and the comment text — any whitespace-only lines in the diff mean something went wrong'." - }, - { - "severity": "minor", - "category": "hints", - "task": "06", - "issue": "Edge case 'The new help line is 88 characters. Its j,J neighbour is already 85' — both figures are off by one. Measured on the tree: top/help.go:19 (the j,J line) is 86 characters, and the proposed line ` w,W 'w' pg_stat_wal / pg_stat_archiver switch, 'W' WAL statistics menu.` at the same description column (22) is 89. The conclusion the numbers support — that the new line is consistent with the block and that the <= 80 rule at top/help_test.go:78-79 applies only to the Space entry — is correct and unaffected.", - "fix": "Correct the two numbers to 86 and 89, or drop the exact figures and keep only the qualitative point (the j,J neighbour is already well past 80; the <= 80 assertion is scoped to the Space entry)." - }, - { - "severity": "minor", - "category": "tdd", - "task": "04", - "issue": "The TDD Anchor asks to extend the existing Test_options_validate table with a row `options{showWAL: \"-f\"}` AND to 'assert the message is exactly `report type is not specified, quit`'. The existing table (cmd/report/report_test.go:10-32) carries only `valid bool`, `opts options` and an unused `want report.Config` — it has no field to express an expected error message, and its loop only calls assert.Error for invalid rows. The instruction cannot be followed as written without reshaping the table or splitting the message assertion into a separate test.", - "fix": "Say which shape is intended: either add a `wantErr string` field to the table and assert.EqualError when non-empty, or keep the table untouched and pin the literal in a separate small test (e.g. Test_options_validate_WALUnmappedMessage) — the second keeps the existing rows byte-identical." - } - ], - "stats": { - "tasks_checked": 3, - "claims_verified": 56, - "issues_found": 11 - } -} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch1.json b/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch1.json deleted file mode 100644 index 1ac9fca4..00000000 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch1.json +++ /dev/null @@ -1,184 +0,0 @@ -{ - "validator": "dev-task-validator", - "feature_base": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver", - "iteration": 1, - "tasks_checked": ["01", "02", "03", "04", "05"], - "status": "changes_required", - "findings": [ - { - "severity": "critical", - "category": "frontmatter", - "task": "01", - "section": "frontmatter", - "issue": "The frontmatter is not parseable YAML. `verify:` is an unquoted plain scalar containing a colon+space (\"... in the CI image: 9 columns on PG 14-19 ...\"), which yields `mapping values are not allowed here` (confirmed with yaml.safe_load). Any tool that reads status/depends_on/wave/skills/reviewers from this file fails on the whole block, not just on `verify`.", - "fix": "Quote the value (`verify: \"bash — go test ./internal/query/...; 9 columns on PG 14-19, succeeds under pg_monitor, fails without it\"`) or reduce it to `verify: bash` with the detail moved to a `#` comment, as tasks 02/05 do. Task 03 already shows the quoted form." - }, - { - "severity": "critical", - "category": "carry-forward", - "task": "01", - "section": "Description / What to do / Acceptance Criteria", - "issue": "Task 01 contradicts tech-spec Decision 19 and the tech-spec's own `Files to modify` for Task 1 (`internal/query/archiver.go`, `internal/query/archiver_test.go`, `internal/postgres/testing.go`). The Description says the task \"creates exactly two new files and touches nothing that exists\", `What to do` ends with \"Do not touch ... any existing file\", and the first acceptance criterion is \"no other file in the tree is modified\" — while Context Files simultaneously lists `internal/postgres/testing.go` as **MODIFY** for the shared role helper. The shared helper therefore has no step in `What to do`, no TDD anchor and no acceptance criterion, so nothing obliges Task 01 to produce it. Task 03 was moved to Wave 2 and given `depends_on: [\"01\"]` for exactly that helper (Decision 19), so an executor following Task 01 literally blocks Task 03.", - "fix": "Make the helper a first-class part of Task 01: add a `What to do` step (\"add the idempotent role-creation / SET ROLE / RESET ROLE helper to internal/postgres/testing.go, named and shaped so internal/query and internal/stat both use it — Decision 19\"), an acceptance criterion pinning its name/signature and its use by both privilege tests, and correct the two 'touches nothing that exists' claims plus the 'no other file is modified' criterion to name the three files from the tech-spec." - }, - { - "severity": "critical", - "category": "consistency", - "task": "03", - "section": "Details → Dependencies", - "issue": "Body contradicts frontmatter and the task's own header note. Frontmatter is `depends_on: [\"01\"]`, `wave: 2`, and the blockquote at the top states the Decision 19 move to Wave 2; but `Details → Dependencies` reads \"none — `depends_on: []`. No new Go packages. Wave 1, alongside Tasks 1, 2 and 4.\" An executor reading Details will treat this as a Wave-1 task with no dependency on Task 01.", - "fix": "Rewrite the Dependencies subsection: depends on Task 01 (the shared role helper in `internal/postgres/testing.go`, Decision 19), Wave 2, no new Go packages. Also fix the stale Context Files annotation \"tech-spec (Task 3 in Wave 1; ...)\"." - }, - { - "severity": "critical", - "category": "consistency", - "task": "03", - "section": "Details → Coordination risk", - "issue": "The 'Coordination risk' paragraph is the pre-Decision-19 text and instructs the opposite of the decision: \"If a suitable helper already exists in the package when you start, reuse it; otherwise give yours an overview-specific name\" and \"`internal/stat` is a separate package and needs its own helper.\" Decision 19 mandates ONE helper, in `internal/postgres/testing.go`, created by Task 01 and imported by both `internal/query` and `internal/stat`; two differently-named copies are exactly what the decision rejected. This paragraph also re-introduces the compile-collision race the wave move was made to remove.", - "fix": "Replace the paragraph with: Task 01 owns the single helper in `internal/postgres/testing.go`; this task imports and reuses it for both the `internal/query` and the `internal/stat` tests and must not define a second one. If the helper is missing when this task starts, that is a Task 01 defect to report, not a reason to fork it." - }, - { - "severity": "major", - "category": "content", - "task": "01", - "section": "Acceptance Criteria", - "issue": "Mutation hazard (the 016 class). The criterion \"replace `pg_ls_archive_statusdir()` with `pg_ls_dir('pg_wal/archive_status')`: `Test_StatArchiverQuery_PgMonitorRoleSucceeds` turns red with a permission error. This is the mutation that proves the positive privilege test is exercising privileges and not the superuser fixture connection\" names a mutation that will not fail for the stated reason. `pg_ls_dir(text)` returns `SETOF text` with an unnamed column, so without the `AS name` relation alias the query fails at parse/plan time with `column \"name\" does not exist` (42703) — Task 03's own edge-case note documents this. The test goes red for a syntax reason under any role, including a leaked superuser session, so the anti-vacuity proof the criterion claims is not delivered.", - "fix": "Write the mutation as `FROM pg_ls_dir('pg_wal/archive_status') AS name` and require the observed failure to be SQLSTATE 42501 naming `pg_ls_dir` — anything else means the test is not running under the restricted role. Optionally add the direct guard Task 03 uses (assert `current_user` and `rolsuper = false` before running the query)." - }, - { - "severity": "major", - "category": "content", - "task": "03", - "section": "Acceptance Criteria", - "issue": "Two role-setup mutations are not reversible by reverting code, so they can produce a false green and a poisoned cluster. Roles and grants are cluster-global and creation is idempotent with no REVOKE. (a) \"remove the `GRANT pg_monitor` from the role setup → `Test_ArchivingBacklogQuery_PgMonitorRole` must go RED\" stays GREEN if the role already holds `pg_monitor` from an earlier run in the same container — and Verification Steps explicitly run mutations inside a live container after a green run. (b) \"add `GRANT pg_monitor` to the deny role\" leaves that grant in place after the code is reverted, so `Test_ArchivingBacklogQuery_NoPrivilegeRole` fails spuriously on every later run against that cluster.", - "fix": "State that each role-setup mutation must be run in a freshly started CI container, or make role setup deterministic (drop-and-recreate, or an explicit REVOKE of everything before the intended GRANT) so the grant set is a function of the test code alone. Add the revert step for (b): `REVOKE pg_monitor FROM `." - }, - { - "severity": "major", - "category": "decomposition", - "task": "03", - "section": "Details / TDD Anchor", - "issue": "Tasks 01 and 03 both create test roles on the same six fixture clusters, in the same `internal/query` package, with opposite grant sets — and neither task fixes the actual role names or the grants. Because creation is idempotent and never revokes, the first creator wins: if Task 01's deny role and Task 03's `pg_monitor` role end up sharing a name (or vice versa), a privilege test silently inverts and still reports green. Task 03 makes this worse by asserting \"a shared name is functionally safe\".", - "fix": "Pin the exact role names and grant sets once, in the shared helper Decision 19 puts in `internal/postgres/testing.go` (e.g. `pgcenter_test_monitor` = pg_monitor only, `pgcenter_test_noprivs` = no grants), and have both tasks call it rather than issuing their own DDL. Remove the 'shared name is functionally safe' claim." - }, - { - "severity": "major", - "category": "content", - "task": "04", - "section": "Acceptance Criteria", - "issue": "The criterion \"Exactly the four known `showWAL` sites changed; `grep -rn showWAL --include=*.go .` returns four hits and no more\" cannot be satisfied by the task as specified: the task itself adds `Test_selectReport_WALWhitelistIsClosed` (8+ rows referencing `showWAL`), `Test_selectReport_WALPrecedence` and a `Test_options_validate` row. The task's own Verification Steps already contradict it (\"the test file will have more rows referencing it — count *sites*\"). An unsatisfiable acceptance criterion either gets quietly dropped or forces the executor to weaken the tests.", - "fix": "Restate as a production-site criterion: the three production sites in `cmd/report/report.go` (struct field, flag registration, `selectReport` arm) are the only production references to `showWAL`, and no file outside `cmd/report/` mentions it — verified by `grep -rn showWAL --include=*.go . | grep -v _test.go`." - }, - { - "severity": "major", - "category": "content", - "task": "05", - "section": "TDD Anchor", - "issue": "Mutation hazard (the 016 class). \"Set `NotRecordable: true` → `Test_filterViews` must go red on every row (the view is dropped before the version gate)\" overstates what the test can observe. In `record/record.go:199-233` a `NotRecordable` drop and a version-gate drop increment the same `filtered` counter, so on the four ≤PG13 rows both `wantN` and `wantV` are unchanged and those rows stay GREEN under the mutation; only the three ≥PG14 rows (`wantV` 28→27) redden. An executor checking 'every row' will see the ≤PG13 rows green and conclude the gate is broken.", - "fix": "Scope the claim: setting `NotRecordable: true` reddens the three ≥PG14 rows of `Test_filterViews` (wantV) and leaves the ≤PG13 rows green, because there the view is already dropped by the version gate and counted in the same `wantN`." - }, - { - "severity": "major", - "category": "consistency", - "task": "02", - "section": "Description / scope boundaries", - "issue": "Cross-task contradiction about `internal/view/view_test.go`. Task 02 states \"`internal/view/view_test.go` belongs to Task 05, which adds the `wal` assertions to `TestViews_Configure`\"; Task 05 states \"`TestViews_Configure` ... **has no `wal` or `archiver` assertion** — do not claim to update it\" and its acceptance criteria never mention it. As written, no task pins the PG 19 `wal` layout as it emerges from `Configure()` — the coverage Task 02 hands off is never received.", - "fix": "Settle ownership in one place. Either Task 05 adds a `TestViews_Configure` (or `TestNew_WalView`-style) assertion that at 190000 the configured `wal` view carries `Ncols: 8` / `DiffIntvl{2,6}` — and Task 02 keeps its hand-off sentence — or Task 02 drops the claim and states that the PG 19 layout is pinned by the selector tests alone." - }, - { - "severity": "minor", - "category": "content", - "task": "01", - "section": "TDD Anchor / Acceptance Criteria", - "issue": "Task 01's privilege tests have no direct 'the SET ROLE actually took effect' guard, while Task 03 makes exactly that guard load-bearing (assert `current_user` and `rolsuper = false` before running the aggregate, with a dedicated mutation for deleting the `SET ROLE`). The asymmetry leaves Task 01's positive privilege test resting entirely on the `pg_ls_dir` mutation — which, per the major finding above, currently reddens for the wrong reason.", - "fix": "Add the same pre-assertion to `Test_StatArchiverQuery_PgMonitorRoleSucceeds` and `Test_StatArchiverQuery_WithoutPgMonitorFails`, plus the mutation 'delete the SET ROLE → the test must go red on its own current_user/rolsuper guard'." - }, - { - "severity": "minor", - "category": "content", - "task": "01", - "section": "Acceptance Criteria", - "issue": "The mutation \"replace the `ready` sub-select with the literal `0` (no privileged call at all) → `Test_StatArchiverQuery_WithoutPgMonitorFails` turns red\" silently assumes `pg_stat_archiver` itself is selectable by a role holding no grants at all. If any fixture version restricts it, the mutated query still errors and the test stays green, proving nothing.", - "fix": "Have the task confirm on the fixtures that the mutated (unprivileged) query succeeds under the deny role — i.e. that the only thing making the real query fail is `pg_ls_archive_statusdir()` — and note the observation in the decisions-log entry." - }, - { - "severity": "minor", - "category": "content", - "task": "02", - "section": "Acceptance Criteria", - "issue": "Mutation M1 (`DiffIntvl` → `{2,7}`) is claimed to redden \"the `DiffIntvl` boundary assertion in `Test_StatWALQueries`\". With 8 live columns (indices 0-7), the assertion on `DiffIntvl[1]+1` indexes element 8 and the test fails by panic rather than by assertion. It is red either way, but a panic in a table test also aborts the surrounding subtests and obscures the signal.", - "fix": "Bound the boundary assertion (assert `DiffIntvl[1]+1 < len(fields)` first, or compare header names by lookup) so M1 produces a readable failure." - }, - { - "severity": "minor", - "category": "content", - "task": "04", - "section": "Acceptance Criteria", - "issue": "`Test_selectReport_WALPrecedence` guards an invariant (the new arm did not move up the first-match-wins chain) but no acceptance criterion names a mutation for it; AC \"Flag precedence is unchanged: the `showWAL` arm sits between `showFunctions` and `showBgwriter`, and `-A` still beats `-W`\" states the property without a red-proof, which is the shape `patterns.md` rules out for invariant guards.", - "fix": "Add 'Mutation 4 — move the `case opts.showWAL != \"\":` arm above `case opts.showActivity:` → `Test_selectReport_WALPrecedence` turns red. Run it, observe, revert.'" - }, - { - "severity": "minor", - "category": "content", - "task": "03", - "section": "Acceptance Criteria", - "issue": "The user-spec promises \"для суперпользователя видимых изменений нет\" (superuser sees the same backlog value as before), but no criterion or test pins output equivalence between the old and new function — and it cannot be pinned on the fixtures, where the backlog is always 0. The criterion as written only requires \"one non-NULL bigint in bytes\".", - "fix": "Record explicitly that superuser-path value equivalence is verified by the stand run (Task 10) and add that item to Task 10's checklist, so the user-spec criterion is not silently dropped between Task 03 and QA." - }, - { - "severity": "minor", - "category": "consistency", - "task": "05", - "section": "frontmatter", - "issue": "`depends_on: [\"01\", \"02\"]` includes Task 02, but the task neither edits nor compiles against anything Task 02 produces — its own Details say \"This task does not edit anything wal-related, but it shares the Wave 1 → Wave 2 gate\". A wave gate is already expressed by `wave: 2`; the extra edge over-constrains the dependency graph.", - "fix": "Either drop \"02\" from `depends_on` (keeping `wave: 2`), or state the real reason for the edge in the Dependencies subsection." - }, - { - "severity": "minor", - "category": "carry-forward", - "task": "05", - "section": "What to do", - "issue": "`OrderDesc: true` is pinned for the archiver entry and asserted by `TestNew_ArchiverView`, but the tech-spec's Data Models block specifies only `Ncols: 9`, `DiffIntvl{0,0}`, `OrderKey: 0`, `UniqueKey: 0`, `NotRecordable: false`. It matches the `wal`/`bgwriter` precedent so it is a reasonable addition, but it is an addition to the spec, not a carry-forward.", - "fix": "Note the field in the decisions-log entry for the task (and, if the tech-spec is being touched anyway, add `OrderDesc` to the Data Models block)." - }, - { - "severity": "minor", - "category": "structure", - "task": "01", - "section": "Context Files", - "issue": "The template requires `project.md` among Project knowledge; this repo's project-knowledge directory has no `project.md` (it has overview.md / architecture.md / patterns.md / deployment.md). Tasks 02 and 05 document the substitution inline; tasks 01, 03 and 04 list `overview.md` without saying why `project.md` is absent.", - "fix": "Add the same one-line note used in tasks 02 and 05 (\"there is no `project.md` in this repo's PK; `overview.md` plays that role\") to tasks 01, 03 and 04." - }, - { - "severity": "minor", - "category": "structure", - "task": "05", - "section": "Context Files", - "issue": "Link-path convention is inconsistent across the batch: tasks 01-04 use repo-root-relative paths (`docs/features/...`, `internal/query/...`, `.claude/skills/...`), task 05 uses file-relative paths (`../../../internal/view/view.go`, bare `017-feat-wal-archiver.md`). Executors resolving links one way or the other will hit dead paths in one of the two styles.", - "fix": "Normalise task 05 to the repo-root-relative style used by tasks 01-04." - }, - { - "severity": "minor", - "category": "structure", - "task": "01", - "section": "TDD Anchor / Verification Steps", - "issue": "Template scaffolding left in place: the HTML comments `` and `` and the Russian boilerplate line \"Тесты, которые нужно написать ДО реализации ...\" survive in tasks 01 and 03 (task 03 keeps the boilerplate line and the Details comment; tasks 02, 04, 05 already removed most of them).", - "fix": "Delete the leftover template comments and the generic boilerplate sentence — the sections carry real content and the instructions are already in the acceptance criteria." - }, - { - "severity": "minor", - "category": "content", - "task": "02", - "section": "Details → Files", - "issue": "`internal/query/wal.go` is described as \"33 lines today\"; the file is 32 lines. The line anchors for the constants and the selector are correct, and the wal_test.go anchors (Test_SelectStatWALQuery at :10-31, the 190000 row at :21) check out against the tree.", - "fix": "Correct the line count, or drop it — the structural anchors are what matter." - } - ], - "stats": { - "tasks_checked": 5, - "issues_found": 21, - "critical": 4, - "major": 6, - "minor": 11 - } -} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch2.json b/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch2.json deleted file mode 100644 index 7ce1eb5c..00000000 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch2.json +++ /dev/null @@ -1,111 +0,0 @@ -{ - "validator": "dev-task-validator", - "feature_base": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver", - "tasks_checked": ["06", "07", "08", "09", "10"], - "iteration": 1, - "status": "changes_required", - "findings": [ - { - "severity": "critical", - "category": "content", - "task": "08", - "section": "Acceptance Criteria", - "issue": "Mutation A5 cannot redden any subcase — it is vacuous, and it is the criterion the task offers as 'the guard on the dependency that makes this task Wave 3'. A5 says: remove the archiver entry from view.New(), 'Test_app_doReport_Archiver turns red on all three subcases'. Verified against the code: (a) subcase no_archiver_entries — readTar never sets statOK for a tar with only meta.*/sysinfo.* entries, so nothing is sent on dataCh, processData never reaches the data branch, doReport returns nil and the buffer stays empty with or without the registration; (b) subcases populated and never_archived — newApp takes views[config.ReportType] (report/report.go:83-85), but processData then rebuilds the view through views.Configure (report/report.go:281-291), and Configure's 'archiver' case (Task 5) re-assigns QueryTmpl/Ncols/DiffIntvl from SelectStatArchiverQuery. Everything the replay actually renders comes from either the recorded PGresult (diff.Cols = curr.Cols; align.SetAlign(*d,...) computes ColsWidth/Cols) or from DiffIntvl/OrderKey/OrderDesc/UniqueKey (countDiff → stat.Compare). For the archiver screen those are {0,0}/0/0 — identical to a zero-value view.View. Even if Configure's case were removed too, DiffIntvl stays the zero value {0,0}, so the output is byte-identical. The mutation therefore cannot be 'run, observed red, reverted' as the task demands; the executor will either stall or record evidence that did not happen.", - "fix": "Delete A5 or replace it with a mutation that can actually fail. Options: (1) drop the claim and state plainly, next to the existing honesty note, that the view registration is NOT observable in the replay path for a {0,0}/OrderKey 0/UniqueKey 0 screen — it is pinned by Task 5's registration test; (2) if a dependency guard is wanted here, mutate something the replay does read, e.g. change the archiver view's OrderKey/UniqueKey (only detectable with a multi-row fixture, which this screen does not have) or keep A1, which already covers the selector-driven DiffIntvl. Also correct the Dependencies note, which justifies Wave 3 by 'newApp resolves the view by report type … an unregistered name yields a zero-value view.View' — true, but with no observable consequence in this test." - }, - { - "severity": "major", - "category": "content", - "task": "06", - "section": "Acceptance Criteria", - "issue": "The mutation 'reverting the help text to the old `r,w 'r' replication, 'w' WAL,` line … turns Test_helpTemplate_walEntry red' does not hold as written. Test_helpTemplate_walEntry keys entirely off the marker `'w' pg_stat_wal`, which lives on the NEW w,W line: restoring the `'w' WAL,` clause to the r line leaves the marker unique, the line prefix ` w,W`, the exact description, the j,J adjacency and the descColumn equality all intact — the test stays green. Only the parenthesised alternative ('or rewording the new entry's description') actually reddens it. Consequently the neighbouring criterion 'the `r,w` line has become `r` with only the `'r' replication,` clause' has no guarding assertion at all, in a task whose own premise is that the help screen is pinned by test rather than by review.", - "fix": "Add an assertion to Test_helpTemplate_walEntry (or a second test) that the `'r' replication` entry line's description equals exactly `'r' replication,` — using helpEntryLine + descColumn, as the pause test does — or assert that helpTemplate no longer contains the substring `'w' WAL`. Then restate the mutation as: 'restoring `'w' WAL,` to the r line turns red'." - }, - { - "severity": "major", - "category": "content", - "task": "08", - "section": "Acceptance Criteria", - "issue": "Two of the three archiver subcases have no mutation gate. B1–B3 all perturb the populated archiver fixture or the wal pg19 fixture; A1–A4 touch the query selectors, which the never_archived and no_archiver_entries subcases do not exercise (both are pass-through/empty). With A5 vacuous (see the critical finding), nothing in the acceptance list demonstrates that the two subcases carrying the feature's user-visible promises — 'blank cells, not 0/-/n/a' (user-spec criterion 2) and 'no rows and no header' (Decision 15, user-spec criterion 15) — are capable of failing. The task states the field-count assertion 'fails if the blank cells render 0, - or n/a', but that is an argument, not a run mutation, and the task's own rule is that an argued guard counts only after it has been seen red.", - "fix": "Add two Class-B mutations: (B4) in the never-archived fixture set the four NULL cells to sql.NullString{String: \"0\", Valid: true} and confirm the field-count assertion goes red (5 → 9 fields); (B5) add a real archiver..000.json entry to the empty-archive tar and confirm the 'buffer is empty' assertion goes red. Both are cheap, both run in the same suite, and both make the two subcases falsifiable." - }, - { - "severity": "minor", - "category": "content", - "task": "08", - "section": "Context Files", - "issue": "Every markdown link in this task is written repo-root-relative (docs/features/017-feat-wal-archiver/017-feat-wal-archiver.md, .claude/skills/project-knowledge/patterns.md, report/report.go, docs/decisions-log.md), but the task file itself lives in docs/features/017-feat-wal-archiver/, so none of the links resolve from it. Tasks 06, 07, 09 and 10 use the correct forms (017-feat-wal-archiver.md for siblings, ../../../ for repo paths). The Reviewers and Post-completion paths are affected as well.", - "fix": "Rewrite the links in Context Files, Reviewers and Post-completion to match the sibling tasks: bare filenames for feature artifacts, ../../../ prefix for repo files (../../../report/report.go, ../../../.claude/skills/project-knowledge/patterns.md, ../../../docs/decisions-log.md)." - }, - { - "severity": "minor", - "category": "consistency", - "task": "10", - "section": "Details", - "issue": "Edge case 'Three accepted behaviours must not be filed as defects' repeats the spec's wording: 'the verbose panel now shows `0 B` instead of `n/a`'. Task 09 establishes — correctly, verified in the code — that pretty.Size(0) returns the bare string \"0\" (internal/pretty/pretty.go:11-12) and the row renders as '… 0 archiving backlog …' (top/stat.go:711-731); `0 B` is a spec-prose artefact that the program never prints. As written, the QA gate could look for a literal that cannot appear and file a false FAIL, or accept the wrong evidence.", - "fix": "Change the Decision 11 edge case in Task 10 to 'shows a backlog of zero (rendered `0`, not `0 B`) instead of `n/a`', and cross-reference Task 09's Post-completion note, which records the same correction." - }, - { - "severity": "minor", - "category": "decomposition", - "task": "07", - "section": "frontmatter", - "issue": "depends_on: [\"05\"] does not cover the dependencies the body names. Details → Dependencies lists Task 1 (authoritative column names/order), Task 2 (the fpi,KiB column), Task 4 (the string -W flag) and Task 5. Task 1 and 2 are reachable transitively (Task 05 depends_on [\"01\",\"02\"]); Task 4 is not in the chain at all, yet the task's own Verification Steps and one acceptance criterion run `./bin/pgcenter report -d -W a`, which requires it. Execution order happens to be safe because Task 4 is in Wave 1, so this is a declaration/body mismatch rather than an ordering violation.", - "fix": "Set depends_on: [\"04\", \"05\"] (Task 4 is Wave 1, so the wave assignment is unchanged), or state explicitly in Dependencies that Task 4 is relied on by wave ordering only and is deliberately not declared." - }, - { - "severity": "minor", - "category": "consistency", - "task": "08", - "section": "Description", - "issue": "The legacy shared fixture is described as 'a 2021 PG14beta1 recording' in the Description, but Verification Step 7 calls the same fixture 'the legacy PG13-era tar'. One of the two is wrong; the recording's samples are dated 2021-06-14 (report/report_test.go:34+).", - "fix": "Pick one description and use it in both places — read the meta entry's version_num from report/testdata/pgcenter.stat.golden.tar if the exact version matters to the claim." - }, - { - "severity": "minor", - "category": "content", - "task": "06", - "section": "What to do", - "issue": "Step 7 asserts 'the leftover `r` keeps its own line (the tech-spec settles this cosmetic question — do not fold it into the `a,b,f,o` line above)'. The tech-spec does not settle it: its Architecture section says only that 'the `w` entry moves out of the plain-switch line into its own `w,W` line, and the `Q`-does-not-reset caveat gains `archiver`', and Task 6's Implementation Tasks entry says 'the three help-screen lines'. Nothing there addresses the leftover `r`. The instruction itself is fine as a decision; the attribution is not.", - "fix": "Either drop the attribution ('keep the leftover `r` on its own line — folding it into a,b,f,o would reflow a block that three tests key off'), or add the ruling to the tech-spec so the citation becomes true." - }, - { - "severity": "minor", - "category": "content", - "task": "07", - "section": "Details", - "issue": "Two line references are off by one against the current tree, in a task whose other references are exact. pgStatWALDescription is cited as `:140-157`; the constant actually spans report/describe.go:141-156 (its doc comment is :140). The trailing-whitespace rows are cited as `:146, :151`; the rows that actually end with a trailing space are `- waldir_size` (:145) and `- write,ms` (:151) — :146 is `- wal,KiB`, which has none. The fpi row reference `:148` is correct.", - "fix": "Correct the two references, or drop the line numbers from the trailing-whitespace note and keep the instruction ('two existing wal rows end with a trailing space — do not clean them up')." - }, - { - "severity": "minor", - "category": "structure", - "task": "08", - "section": "TDD Anchor", - "issue": "Template residue left in filled sections: task 08 keeps `` inside a fully written TDD Anchor, plus the boilerplate line 'Тесты, которые нужно написать ДО реализации…' and the `` / `` comments; tasks 09 and 10 keep the Verification Steps / Details comments as well. Not placeholders in the blocking sense (no [Task Name], {PK path}, TODO/TBD anywhere in the batch), but Task 06 shows the intended end state — every comment replaced by real prose.", - "fix": "Strip the instructional HTML comments from sections that are filled, and replace the generic Russian TDD preamble in task 08 with the task-specific one it already has two paragraphs later." - } - ], - "verified_ok": [ - "Frontmatter: all five tasks carry status/depends_on/wave/skills/verify/reviewers/teammate_name and nothing beyond the template; status is 'planned' everywhere; depends_on is an array of ID strings everywhere; skills and reviewers are arrays.", - "Waves and dependencies: 06→05, 07→05, 08→05 (wave 3 over wave 2), 09→04 (wave 3 over wave 1), 10→01..09 (wave 4). No cycles; every referenced ID exists.", - "Skill↔reviewer mapping matches ~/.claude/skills/tech-spec-planning/references/skills-and-reviewers.md: code-writing→3 reviewers (06,07,08), documentation-writing→dev-code-reviewer (09), pre-deploy-qa→[] (10). Frontmatter skills/reviewers match the Required Skills and Reviewers sections in all five tasks.", - "TDD Anchor: present and substantive for the code tasks (06,07,08); correctly ABSENT from task 09 (release note) and task 10 (QA) — the template prescribes deleting it for non-code work, and both tasks carry Verification Steps instead.", - "Section presence and order match the template in all five files; Details carries Files / Dependencies / Edge cases / Implementation hints everywhere.", - "Carry-forward from tech-spec: every 'Files to modify' entry of tech-spec Tasks 6-9 appears in the corresponding task, with no file dropped and no unlisted file added. The tech-spec Testing Strategy items land where expected (walNextView cycle + menu/switchViewTo count updates → 06; describe map entry → 07; archiver + wal PG18/PG19 goldens → 08).", - "Task 10 counts the acceptance criteria correctly: the user-spec «Критерии приёмки» holds exactly 23 checkboxes and the tech-spec «Acceptance Criteria» exactly 11.", - "Code-anchored claims spot-checked and correct: top/keybindings.go 'w' at :38 and 'J' at :49 with no existing 'W' binding; top/menu.go iota block :14-26, menuStatIO style :87-95, menuSelect branch :194-203; top/menu_test.go table :8-25 with the quoted counts; Test_switchViewTo table :592-624 with the sizes→wal row at :604 and wal→replication at :605; Test_statioNextView at :670-683; top/pause_test.go:552-577 does document that menuSelect is unreachable from a unit test; report/describe.go fpi row at :148; report/report.go describeReport map with 'wal' at :674 and Test_describeReport's 'wal' row at :1184; report/report_test.go update flag at :24 and the legacy wal case at :73-77; report/report_record_bgwriter_test.go meta/mkRow/invariants/-update at the cited lines; ADR [008] at docs/decisions-log.md:462; cmd/report/report.go -W flag at :70 and 'report type is not specified, quit' at :95; internal/pretty/pretty.go Size(0)==\"0\" at :11-12.", - "Task 08's honesty boundary is CORRECT as stated: replay renders column names from the recorded PGresult (diff.Cols = curr.Cols; formatStatSample→align.SetAlign; printStatHeader prints v.Cols), never from the SQL, so reordering aliases in internal/query/*.go genuinely cannot redden these goldens.", - "Task 08 mutations A1-A4 and B1-B3 all check out against the code: interval bounds in internal/stat/postgres.go diff() are inclusive (l > interval[1]) and the loop runs over curr.Ncols, so {2,6}→{2,5} moves buffers_full to pass-through; SelectStatWALQuery's current returns really are (PgStatWALDefault, 7, {2,5}) for PG18 and (PgStatWALPG14, 11, {2,9}) for PG14, so A2/A4 land as described (A4 additionally reddens by driving diffPair into ParseInt on '02:00:00'); calculateDelta's {0,0} short-circuit makes A1 flip 100003 to 3; report-time Configure runs off the recording's meta version, which is what makes the version switch testable at all.", - "Task 07 mutations 1-3 all check out: describeReport returns nil for an unknown type (pinned today by the 'invalid' row asserting NoError), so a missing map entry is invisible to an exit-code check and only the named test catches it; the tab-anchored markers do keep fpi off fpi,KiB and write off write,ms, matching the idiom already used by Test_describeActivityColumnOrder.", - "Task 06 mutations on walNextView, switchViewTo's dispatch arm, the menuWAL item count and the Q-caveat line all check out, including the claim that the existing {sizes → wal → wal} and {wal → replication} rows survive unchanged.", - "Task 09's code claims are accurate, including the two it corrects against the specs: `diff failed` is a wrapped error (internal/stat/postgres.go:593) and must be quoted as a prefix, and the verbose backlog renders `0`, not `0 B`." - ], - "stats": { - "tasks_checked": 5, - "issues_found": 10, - "critical": 1, - "major": 2, - "minor": 7 - } -} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-adequacy-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-adequacy-review.json similarity index 100% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-adequacy-review.json rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-adequacy-review.json diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-arch-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-arch-review.json similarity index 100% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-arch-review.json rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-arch-review.json diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-code-research.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-code-research.md similarity index 100% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-code-research.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-code-research.md diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-completeness.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-completeness.json similarity index 100% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-completeness.json rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-completeness.json diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-decisions.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-decisions.md new file mode 100644 index 00000000..a5fed6e0 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-decisions.md @@ -0,0 +1,570 @@ +# Decisions Log: WAL / Archiver + +Отчёты агентов о выполнении задач. Каждая запись создаётся агентом, выполнившим задачу. + +--- + +## Task 05: Регистрация view `archiver` и обновление счётных тестов + +**Status:** Done +**Commit:** 13c33f2 +**Agent:** основной агент + +**Summary:** В `view.New()` добавлена запись `archiver` (`MinRequiredVersion: query.PostgresV14`, `QueryTmpl: query.PgStatArchiverDefault`, `Ncols: 9`, `DiffIntvl{0,0}`, `OrderKey 0` / `OrderDesc true`, непустые `ColsWidth`/`Filters`, `NotRecordable` оставлен нулевым), в `Configure()` — соответствующий `case "archiver":`. Ключевое решение — `MinRequiredVersion` здесь несущий, а не косметический: общего нижнего порога в проекте нет (реестр обслуживает вплоть до PG 9.4, `TestViews_Configure` гоняет `90400`), и при нулевом значении экран предлагался бы на PG ≤ 11, где `pg_ls_archive_statusdir()` не существует, а `pgcenter record` обрывает **всю** запись на первой же ошибке запроса. Попутно закрыт долг задачи 02: в `TestViews_Configure` добавлены `wal`-ассерты в ветки `case 190000:` и `case 140000:` — до этого ничто не пиннило, что `Configure()` реально доносит PG 19 layout до зарегистрированного view (селекторный табличный тест проверяет селектор, а не проводку). + +**Deviations:** Отклонений от спека нет — все AC выполнены. Три уточнения по факту исполнения: + +1. `TestNew_ArchiverView` пиннит больше полей, чем перечисляет шаг 4 задачи: добавлены `Name`, `QueryTmpl`-seed и непустота `ColsWidth`/`Filters`. Задача внутренне противоречива — шаг 4 даёт узкий список, а AC №1 требует «pins every field listed above», включая non-nil карты. Разошлись в пользу AC: каждое из трёх полей провалило litmus-тест (мутация оставляла все пять тестов зелёными), причём потеря карт — не неверное число, а паника в gocui-обработчике при первом расширении колонки или установке фильтра. +2. `Msg` закреплён полным равенством, а не подстрокой (`assert.Contains` → `assert.Equal`). AC №1 требует строку verbatim; равенство — строгое надмножество подстроки, поэтому мутация «убрать `archive_mode=on`» по-прежнему краснеет, но теперь ловится и опечатка в префиксе, которую `Contains` пропускал. +3. Ассерт принадлежности `archiver` в `Test_filterViews` сделан через lookup в карте + `assert.Equal`, а не через предложенный ревьюером `assert.Contains`: падение `Contains` на `view.Views` печатает все 28 структур `View` (141 КБ вывода) и хоронит единственный нужный бит. + +**Tech debt:** + +1. Инварианты реестра (`key == v.Name`, непустые `ColsWidth`/`Filters`) не проверяются ни для одного view кроме `archiver`. Гарды `TestNew_BgwriterView`, `TestNew_ReplslotsView`, `TestNew_StatIOView`, `TestNew_StatIOTimeView`, `TestNew_StatementsJITView` пропускают их все. Реестро-широкий `TestNew_ViewMapInvariants` предложен обоими ревьюерами и сознательно не добавлен: он охраняет view, принадлежащие другим задачам и фичам, то есть делает эту задачу владельцем падений, которые она не может вызвать. Оба ревьюера в round 2 согласились с отсрочкой; dev-test-reviewer прогнал предложенный инвариант против текущего реестра — все 28 view его удовлетворяют, дефекта за отсрочкой не прячется. Правильное место записи — раздел `patterns.md` «Adding a New View», куда заглянет следующая регистрация. +2. `record/recorder.go` роняет всю запись при ошибке любого одного view (`record/record.go:172-175`). Для `archiver` это Decision 4 by design (роль без `pg_monitor` теряет экран целиком), и `wal` с тем же порогом PG14 уже вызывает `pg_ls_waldir()` в том же классе привилегий — то есть экспозиция не новая. Долговременное решение — пропускать сбойный view с INFO-строкой, как это уже делает ветка `pg_stat_statements not found`, в `tarRecorder.collect()`, а не в реестре. Найдено dev-security-auditor. +3. Имена WAL-файлов (колонки 3 и 6) рендерятся без escape-санитизации. Decision 16 корректен для настоящего сервера (`VALID_XFN_CHARS` не пропускает ESC), но не для враждебного эндпоинта, говорящего по протоколу; это уже зафиксированный техдолг [029] (`docs/tech-debt.md:47`), покрывающий все текстовые колонки всех экранов. Здесь не расширен; чинить один раз в `printDataCell`. +4. Комментарий-обоснование над таблицей `Test_filterViews` вырос до 25 строк на 7 строк данных и накапливает археологию четырёх фич. Предложение dev-code-reviewer свернуть историю в один инвариант отклонено (задача явно предписывает комментарий **расширить**, а слои документируют, почему держатся числа в остальных строках), и в round 2 сам ревьюер его снял. Оставлено как housekeeping для следующего владельца файла. +5. Раздел Edge cases самого task-файла (и первая редакция комментария в тесте) приписывает запись в `ColsWidth` функции `align.SetAlign`. Фактически `internal/align/align.go:18` строит новую карту, а `top/stat.go:788` присваивает её целиком — этот путь nil-карту **чинит**, а не роняет. Реальные незащищённые in-place writer'ы — `top/config_view.go:100`, `:124` и `:166`. В комментарии теста исправлено; в task-файле — нет. + +**Reviews:** + +*Round 1:* +- dev-code-reviewer: approved_with_suggestions, 1 major + 3 minor → [017-feat-wal-archiver-task-05-dev-code-reviewer-review.json](017-feat-wal-archiver-task-05-dev-code-reviewer-review.json) +- dev-security-auditor: approved, 0 critical / 0 major, 2 minor (оба вне скоупа, вынесены в Tech debt) → [017-feat-wal-archiver-task-05-dev-security-auditor-review.json](017-feat-wal-archiver-task-05-dev-security-auditor-review.json) +- dev-test-reviewer: needs_improvement, 2 major + 5 minor → [017-feat-wal-archiver-task-05-dev-test-reviewer-review.json](017-feat-wal-archiver-task-05-dev-test-reviewer-review.json) + +*Round 2 (после исправлений):* +- dev-code-reviewer: approved_with_suggestions, 1 minor (фактическая неточность комментария про `align.SetAlign` — исправлена) → [017-feat-wal-archiver-task-05-dev-code-reviewer-review-round2.json](017-feat-wal-archiver-task-05-dev-code-reviewer-review-round2.json) +- dev-test-reviewer: passed, 2 minor (отсроченный реестровый инвариант + опечатка в комментарии — исправлена) → [017-feat-wal-archiver-task-05-dev-test-reviewer-review-round2.json](017-feat-wal-archiver-task-05-dev-test-reviewer-review-round2.json) + +Приняты и применены все major-находки round 1 (`ColsWidth`/`Filters`, `Name`) и все minor, кроме двух: сворачивание исторического комментария в `record_test.go` (отклонено, ревьюер снял в round 2) и реестро-широкий `TestNew_ViewMapInvariants` (отсрочено, оба ревьюера согласились — см. Tech debt 1). Оба ревьюера round 2 независимо воспроизвели весь мутационный набор в песочнице и подтвердили результаты построчно. + +**Verification:** +- Счётчики before → after: `TestNew` 27 → 28; `TestView_VersionOK` 190000 27 → 28, 160000 27 → 28, 140000 24 → 25, строки 130000/120000/110000/100000 **не тронуты**; `Test_filterViews` `wantV` 27/18/24 → 28/19/25 на трёх строках ≥ PG14 и `wantN` 8/11/13/13 → 9/12/14/14 на четырёх строках ≤ PG13, блок-комментарий расширен. +- Хост: `go test ./internal/view/...` → ok; `go test ./record/... -run Test_filterViews` → ok. Полный пакет `record` в образе `lesovsky/pgcenter-testing:0.0.11`: `go test -race -p 1 -timeout 300s ./internal/view/... ./record/...` → оба ok (в т.ч. `Test_app_record`, который считает recordable views динамически). Дополнительно в образе прогнаны `./report/...` и `./top/...` → ok, чтобы исключить незамеченный счётный пин вне двух известных мест. +- `gofmt -l` пусто, `go vet` чисто, `golangci-lint run ./internal/view/... ./record/...` → 0 issues, `gosec` → 0 issues, `go build -o /dev/null ./cmd` → ok. `git diff record/record.go` пуст; `case "wal":` в `view.go` — только контекстная строка диффа. +- Мутационный контроль — каждая мутация применена к продакшн-коду, падение **наблюдалось**, мутация откачена: + - **M1** удаление `MinRequiredVersion` → `TestView_VersionOK` красный ровно на четырёх строках ≤ PG13 (130000 `19/20`, 120000 `16/17`, 110000 `14/15`, 100000 `14/15`); `Test_filterViews` красный на тех же четырёх строках, и после доработки — с поимённым `archiver kept? version=…` на каждой. + - **M2** `Ncols` 9 → 8 и `DiffIntvl{0,0}` → `{0,1}` → `TestNew_ArchiverView` (`TestViews_Configure` при этом остаётся зелёным: селектор переприсваивает 9 — это и есть честная граница того, что пиннят archiver-ассерты). + - **M3** `Msg` без `archive_mode=on` → `TestNew_ArchiverView`. + - **M4** `NotRecordable: true` → `Test_filterViews` красный **ровно на трёх строках ≥ PG14** (`{190000,"public"}` `0/1` и `28/27`, `{140000,""}` `9/10` и `19/18`, `{140000,"public"}` `3/4` и `25/24`); четыре строки ≤ PG13 остаются **зелёными** — `filterViews` удаляет view и делает `filtered++` в обеих ветках, так что подмена *причины* отбрасывания там ничего не сдвигает. Ожидать красноты на каждой строке арифметически неверно. + - **M5** откат ветки PG 19 из `SelectStatWALQuery` (задача 02) → `TestViews_Configure` красный в ветке `case 190000:` по тексту запроса, `Ncols` `8/7` и `DiffIntvl` `{2,6}/{2,5}`. Это и есть настоящий гейт проводки. + - **M6** удаление `ColsWidth` и `Filters` из записи → `TestNew_ArchiverView`, два падения `NotNil`. + - **M7** `Name` → `"archive"` → `TestNew_ArchiverView` (все остальные тесты ищут view по ключу карты и остаются зелёными). + - **M8** `QueryTmpl`-seed → `query.PgStatWALPG14` → `TestNew_ArchiverView`. + - **M9** опечатка только в префиксе `Msg` (`"Show archiver stats (requires archive_mode=on)"`) → `TestNew_ArchiverView`; под прежним `assert.Contains` эта мутация оставалась зелёной. + - **Отрицательный контроль (задокументирован, не дефект):** удаление `case "archiver":` из `Configure()` оставляет пакет зелёным — `New()` уже выставляет те же `QueryTmpl`/`Ncols`/`DiffIntvl`. Archiver-ассерты в `TestViews_Configure` — регрессионный страж дрейфа между `SelectStatArchiverQuery` и статической записью, а не доказательство проводки; оговорка вынесена в комментарий рядом с самими ассертами. + +--- + +## Task 02: PG 19 FPI column on the wal screen + +**Status:** Done +**Commit:** 7a523a1 +**Agent:** основной агент +**Summary:** В `internal/query/wal.go` добавлена константа `PgStatWALPG19` — это `PgStatWALDefault` плюс ровно одно выражение `round(wal_fpi_bytes / 1024, 2) AS "fpi,KiB"`, вставленное сразу после счётчика `wal_fpi`, чтобы количество full page images и объём, который они стоят, стояли рядом и оба попадали внутрь диффуемого диапазона. `SelectStatWALQuery` получил третью ветку `version >= PostgresV19` → `(PgStatWALPG19, 8, [2]int{2, 6})`; `stats_age` сместился на колонку 7 и остался вне интервала — это текстовое значение `date_trunc`, попадание которого внутрь интервала роняет весь сэмпл на `strconv.ParseInt`, а не одну ячейку. Ветки PG 14–17 и PG 18 не тронуты: diff `wal.go` состоит только из добавлений. `internal/view/view.go` править не потребовалось — его `case "wal":` уже делегирует селектору. + +**Deviations:** Отклонений от спека по существу нет — все AC выполнены. Четыре уточнения по факту исполнения: +1. Тестов добавлено больше, чем в TDD Anchor. Сверх двух заявленных no-Postgres гардов добавлен деривационный ассерт: `PgStatWALPG19` приравнивается к `PgStatWALDefault` с ровно одной вставленной подстрокой. Причина — находка dev-test-reviewer (major), подтверждённая эмпирически: единственное новое выражение задачи содержит конверсию `/1024`, и ни один из четырёх ассертов TDD Anchor её не пиннил — замена на голый `wal_fpi_bytes` оставляла всё зелёным, а на экране колонка с заголовком KiB показывала бы байты (ошибка в 1024 раза). Деривация закрывает заодно и второй minor: `Ncols=8` и сохранность остальных семи колонок теперь гарантированы структурно, без фикстуры. +2. `assert.NoError` после `conn.Query` и `Format` в `Test_StatWALQueries` заменён на `require.NoError` — то есть отступление от прецедента `io_test.go`, на который ссылалась задача. Причина: ровно тот failure mode, который предсказывает caveat задачи (переименование `wal_fpi_bytes` на beta3/RC), при нефатальном ассерте тонет в каскаде из трёх производных падений вместо одного сообщения по существу. Обоими ревьюерами предложено независимо. +3. Полнопакетный гейт в CI-образе с первого раза не запустился: параллельный агент в этот момент держал `internal/query/archiver_test.go` в TDD-красном состоянии без реализации, и пакет не компилировался. WAL-скоуп прогнан на зеркале репозитория в scratchpad без этого файла; после того как соседний агент влил `archiver.go`, полный `go test -race -p 1 ./internal/query/...` прогнан в образе и зелёный. Файлы соседнего агента не редактировались, коммит сделан с явным pathspec на два файла. +4. Метрики в `017-feat-wal-archiver-metrics.json` не писались: по `metrics-protocol.md` метрики пишет только lead-агент, субагенты задач — нет. + +**Tech debt:** +1. Doc-комментарий `PgStatWALDefault` по-прежнему гласит «PG 18+», хотя PG 19+ теперь уходит в свою ветку — константа покрывает только PG 18. Там же расходится имя: с тремя ветками «Default» больше не означает «самый новый layout», в отличие от `bgwriter.go`, где каждая константа версионно ограничена. Не исправлено осознанно: AC требует, чтобы diff `wal.go` состоял только из добавлений. Просится переименование `PgStatWALDefault` → `PgStatWALPG18` в задаче, которая следующей владеет этим файлом (ссылок вне `internal/query` нет). +2. В соседних ветках селектора смешаны именованная константа (`version >= PostgresV19`) и литерал (`version >= 180000`). Задача явно выносит нормализацию литерала из скоупа — забирать вместе с пунктом 1. +3. Сильнейшие утверждения задачи (живой счёт колонок и упорядоченный PG 19 header) стоят за `t.Skipf`, а скип зелёный. В этом прогоне подтверждено, что сабтест `pg_stat_wal/190000` реально выполнился (PASS, не SKIP), но ничто в тесте этого не обеспечивает: CI-образ, в котором фикстура PG 19 не поднялась, отрапортует успех. Предложенный ревьюером env-гейт (`PGCENTER_FIXTURES_REQUIRED`) не применён — он требует правки общей для пакета idiom `t.Skipf` и инвокации CI, а задача явно предписывает `t.Skipf` сохранить. Экспозиция зафиксирована здесь, чтобы не потерялась. +4. `wal_fpi_bytes` проверен против **PG 19beta2**. Если имя уедет на beta3/RC — радиус поражения ровно одна изменённая строка запроса, и живой тест упадёт с undefined column, что и есть верный сигнал. + +**Reviews:** + +*Round 1:* +- dev-code-reviewer: approved_with_suggestions, 4 minor (все опциональные; два конфликтуют с AC и вынесены в Tech debt) → [017-feat-wal-archiver-task-02-dev-code-reviewer-review.json](017-feat-wal-archiver-task-02-dev-code-reviewer-review.json) +- dev-security-auditor: approved, 0 findings → [017-feat-wal-archiver-task-02-dev-security-auditor-review.json](017-feat-wal-archiver-task-02-dev-security-auditor-review.json) +- dev-test-reviewer: needs_improvement, 1 major + 3 minor → [017-feat-wal-archiver-task-02-dev-test-reviewer-review.json](017-feat-wal-archiver-task-02-dev-test-reviewer-review.json) + +Major и один minor от dev-test-reviewer закрыты деривационным ассертом, ещё один minor — переходом на `require.NoError` (см. Deviations 1–2). Оставшийся minor (env-гейт против зелёного скипа) и два minor от dev-code-reviewer (комментарий `PgStatWALDefault`, литерал `180000`) отклонены как конфликтующие с AC либо явно вынесенные задачей из скоупа — вынесены в Tech debt. Третий minor dev-code-reviewer — отсутствие этого файла — закрыт данной записью. + +**Verification:** +- Полный `go test -race -p 1 -timeout 300s ./internal/query/...` в образе `lesovsky/pgcenter-testing:0.0.11` с фикстурами PG 14–19 → `ok ... 3.488s`. Сабтест `Test_StatWALQueries/pg_stat_wal/190000` → **PASS**, не SKIP: живой `FieldDescriptions()` вернул ровно 8 колонок в порядке `source, waldir_size, wal,KiB, records, fpi, fpi,KiB, buffers_full, stats_age`. +- `gofmt -l internal/query` — пусто; `golangci-lint run ./internal/query/...` → 0 issues; `go build -o /dev/null ./cmd` — ok; diff `wal.go` — только добавления (17 строк, 0 удалений). +- Мутационный контроль — каждая мутация применена к продакшн-коду, падение наблюдалось, мутация откачена: + - **M1** `[2]int{2, 7}` (stats_age втянут в диффуемый диапазон) → красный дважды: `Test_SelectStatWALQuery` строки 190000 и 200000 (`expected [2]int{2,6}, actual [2]int{2,7}`) и на живой фикстуре `Test_StatWALQueries/pg_stat_wal/190000` — `"8" is not greater than "8"`, «DiffIntvl upper bound must leave at least one column (stats_age) outside». + - **M2** выражение `"fpi,KiB"` перенесено в конец select-листа, после `stats_age` → красный `Test_SelectStatWALQuery_PG19ColumnOrder` (`"fpi,KiB" must precede wal_buffers_full` + нарушен суффикс `AS stats_age FROM pg_stat_wal`) и на фикстуре — упорядоченный список заголовков разошёлся именно перестановкой `fpi,KiB` в хвост, плюс `the last diffed column must be buffers_full: expected buffers_full, actual stats_age`. + - **M3** `version >= PostgresV19` → `version == PostgresV19` → красной стала ровно строка 200000 в `Test_SelectStatWALQuery`; строка 190000 осталась зелёной, что и подтверждает адресность форвард-строки. + - **M4** `wal_fpi_bytes` добавлен в `PgStatWALDefault` вместо отдельной константы, ветка PG 19 убрана (константа `PgStatWALPG19` оставлена определённой, иначе пакет не компилируется и красного теста не будет, а будет ошибка сборки) → красный `Test_SelectStatWALQuery_LegacyBranchesUntouched`: «should not contain "wal_fpi_bytes" — wal_fpi_bytes does not exist before PG 19». + - **M5** (сверх AC, для проверки исправления по ревью) `round(wal_fpi_bytes / 1024, 2)` → `wal_fpi_bytes` → до исправления все четыре гарда оставались зелёными; после добавления деривационного ассерта → красный `Test_SelectStatWALQuery_PG19ColumnOrder`, «PG 19 query must be the PG 18 query plus exactly the fpi,KiB expression». +- Мутации M1–M4 независимо воспроизведены dev-code-reviewer в отдельном worktree с тем же результатом. + +--- + +## Task 04: report CLI — `-W` becomes a string flag + +**Status:** Done +**Commit:** 9ec944b +**Agent:** основной агент +**Summary:** `options.showWAL` переведён с `bool` на `string`, флаг `-W` — с `BoolVarP` на `StringVarP`, в `selectReport` добавлен закрытый whitelist `w` → `wal`, `a` → `archiver` без `default`-ветки. Ключевое решение — Decision 17: отсутствие `default` во внутреннем switch несущее, а не упущение: неотображённое значение выпадает из обоих switch к финальному `return ""`, и `validate()` отклоняет его до конструирования `report.Config`, поэтому в `ReportType` (фильтр tar-записей в `isFilenameOK` и ключ карты view в `newApp`) пользовательские байты попасть не могут. Ломающее изменение принято осознанно: обе задокументированные failure shapes (`flag needs an argument: 'W' in -W` и `report type is not specified, quit`) закреплены тестами с точными литералами. + +**Deviations:** Нет отклонений от спека. Три уточнения по факту: +1. Спек утверждает, что описание флага видно в `pgcenter report --help`. Фактически `--help` для `report` полностью перекрыт рукописным текстом `printReportHelp()` в `cmd/help.go:170` (`SetHelpTemplate`/`SetUsageTemplate` в `cmd/pgcenter.go:56-57`), поэтому строка `Usage` у cobra-флага пользователю не показывается. Строка задана verbatim как требует AC и закреплена `Test_walFlagDefinition`, но `cmd/help.go` вне двухфайлового скоупа задачи и ни одной задачей фичи не покрыт — см. Tech debt. +2. Тестов добавлено больше, чем в TDD Anchor: `Test_selectReport_WALPrecedence` вырос из одного ассерта в таблицу из 4 строк, в whitelist-таблицу добавлены строки с ведущим и хвостовым пробелом, добавлены сквозные проверки `validate() → Config.ReportType`. Все — по результатам ревью, каждая закрывает мутацию, остававшуюся зелёной. +3. `Test_walFlagDefinition` закрепляет `NoOptDefVal == ""` — поле, не названное в TDD Anchor. Без него cobra-шим, явно отвергнутый Decision 7, проходил весь suite незамеченным. + +**Tech debt:** +1. `cmd/help.go:170` по-прежнему печатает `-W, --wal show pg_stat_wal statistics` — текст описывает булев флаг и не упоминает селекторы `w`/`a`. Это единственная справка, которую реально видит пользователь `pgcenter report --help`. Правка однострочная (по образцу строк `-D`/`-X`/`-P` с `SELECTOR`), но файл вне скоупа задачи 04 и не назначен ни одной задаче фичи — нужно назначить до релиза, иначе задача 09 сошлётся в release notes на текст, которого пользователь не увидит. +2. Ошибка `report type is not specified, quit` печатается, но процесс завершается с кодом **0** (`main()` в `cmd/pgcenter.go:67` печатает ошибку без `os.Exit(1)`). Дефект pre-existing и общерепозиторный (`-J q` ведёт себя так же), но именно это изменение делает его значимым: обёртка вида `pgcenter report -W -f dump.tar > out.txt || alert` теперь пишет пустой файл и рапортует успех. Details задачи утверждают «the command exits non-zero» — фактически это не так. +3. `github.com/spf13/pflag` теперь импортируется тестом напрямую, но в `go.mod:27` помечен `// indirect`. Сборка, тесты и линт проходят, `go mod tidy -diff` в CI нет; нужен `go mod tidy` в момент, когда в ветке не работают параллельные агенты. +4. Ожидаемо и намеренно не компенсировано: `-W a` отображается корректно, но view `archiver` (задача 05) и запись в `describeReport` (задача 07) появляются в следующих волнах, поэтому до их слияния `-W a` даёт пустой отчёт. Merge-гейт фичи, а не дефект задачи. + +**Reviews:** + +*Round 1:* +- dev-code-reviewer: approved_with_suggestions, 2 minor → [017-feat-wal-archiver-task-04-dev-code-reviewer-review.json](017-feat-wal-archiver-task-04-dev-code-reviewer-review.json) +- dev-security-auditor: approved, 1 major + 3 minor (major и два minor — вне скоупа, вынесены в Tech debt) → [017-feat-wal-archiver-task-04-dev-security-auditor-review.json](017-feat-wal-archiver-task-04-dev-security-auditor-review.json) +- dev-test-reviewer: needs_improvement, 2 major + 4 minor → [017-feat-wal-archiver-task-04-dev-test-reviewer-review.json](017-feat-wal-archiver-task-04-dev-test-reviewer-review.json) + +*Round 2 (после исправлений):* +- dev-code-reviewer: approved, 0 findings → [017-feat-wal-archiver-task-04-dev-code-reviewer-review-round2.json](017-feat-wal-archiver-task-04-dev-code-reviewer-review-round2.json) +- dev-test-reviewer: passed, 1 minor (закрыт после ревью) → [017-feat-wal-archiver-task-04-dev-test-reviewer-review-round2.json](017-feat-wal-archiver-task-04-dev-test-reviewer-review-round2.json) + +Одна рекомендация round 1 отклонена: строить зеркальный `pflag.FlagSet` из полей `Lookup("wal")` вместо литералов. Причина — производное зеркало всегда несёт пустой `NoOptDefVal` независимо от реального флага, то есть при установленном шиме оно продолжало бы рапортовать `flag needs an argument`, пока реальный CLI молча принимает голый `-W`. Оба ревьюера в round 2 согласились и сняли предложение. + +**Verification:** +- `go test ./cmd/report/... -v` → 10 passed, все новые имена тестов присутствуют; зелено под `-race` и `-shuffle=on` +- `go build ./...`, `go vet ./cmd/...`, `gofmt -l cmd/report` (пусто), `golangci-lint run ./cmd/report/...` → 0 issues +- Мутационный контроль (каждая применена, наблюдалась красной, откачена): `default: return "wal"` → `Test_selectReport_WALWhitelistIsClosed`; `case "a"` возвращает `"wal"` → `Test_selectReport`; правка одного символа в help-тексте → `Test_walFlagDefinition`; `NoOptDefVal = "w"` → `Test_walFlagDefinition`; перенос ветки выше `showDatabases`/`showFunctions` и ниже `showBgwriter` → `Test_selectReport_WALPrecedence`; `strings.TrimSpace`/`TrimLeft`/`ToLower` → `Test_selectReport_WALWhitelistIsClosed`; `ReportType` захардкожен в `validate()` → `Test_selectReport_WALWhitelistIsClosed` +- Ручная проверка: `go run ./cmd report -W` → `flag needs an argument: 'W' in -W`; `go run ./cmd report -W -f /nonexistent.tar` → `report type is not specified, quit` (файл не открывается); `-W x` → та же ошибка; `-W w` / `-W a` / `-A -W a` доходят до открытия файла + +--- + +## Task 01: Archiver query, selector and the shared test-role helper + +**Status:** Done +**Commit:** ae50197 (round-1 содержимое тех же трёх файлов попало в 23a7b1f — см. Deviations) +**Agent:** основной агент + +**Summary:** Добавлены `internal/query/archiver.go` с константой `PgStatArchiverDefault` (9 колонок в зафиксированном порядке: `source, ready, archived, last_archived, archived_age, failed, last_failed, failed_age, stats_age`) и селектором `SelectStatArchiverQuery(_ int) (string, int, [2]int)` → `(PgStatArchiverDefault, 9, [2]int{0,0})`; версионной ветки нет, потому что `pg_stat_archiver` схемно идентичен на PG 14–19, а `pg_ls_archive_statusdir()` есть на всех (форма `SelectStatIOTimeQuery`, `io.go:99`). В `internal/postgres/testing.go` добавлен общий хелпер `SetupTestRole(db *DB, name string, pgMonitor bool) error` — идемпотентное создание роли через `DO`-блок, опциональный `GRANT pg_monitor`, `SET ROLE`; он возвращает `error` и не тянет пакет `testing`, потому что файл не имеет build-тега и попадает в релизный бинарь. Привилегии Decision 4 доказаны тестами в обе стороны: роль с `pg_monitor` выполняет запрос, роль без него получает SQLSTATE `42501` с именем `pg_ls_archive_statusdir` в сообщении. + +**Deviations:** + +1. **TDD Anchor противоречит сам себе по тесту 3.** Преамбула утверждает, что тесты 1 и 3 идут без PostgreSQL, а собственный буллет теста 3 требует сканирования строк фикстуры в `sql.NullString` и проверки `Valid` — это невозможно без сервера. Разрешено расщеплением: проверка «в запросе нет `coalesce`» вынесена в бессерверный `Test_StatArchiverQuery_Structure`, живая проверка NULL/значений осталась в `Test_StatArchiverQuery_NullsStayNull`. Оба ревьюера round 2 подтвердили, что это правильный выбор. +2. **Тестов шесть, а не пять.** Сверх пяти из TDD Anchor добавлены `Test_StatArchiverQuery_Structure` (бессерверная фиксация предиката `.ready` и порядка алиасов — по major-находке test-ревьюера: на фикстурах пустой каталог статусов, поэтому `count(*) FILTER (WHERE name LIKE '%.ready')` и голый `count(*)` живьём неразличимы) и `Test_SetupTestRole_RejectsUnsafeName` (по сходящейся находке code-ревьюера и test-ревьюера round 2 — новая проверка имени роли иначе не покрыта ничем). +3. **Мутация M7 краснеет иначе, чем обещает AC.** AC требует, чтобы `pgMonitor: false` в позитивном тесте дал красный «с SQLSTATE 42501». После усиления гварда (проверка `pg_has_role` и точного состава членства) тест краснеет раньше — на самом гварде, до запроса, — и 42501 в этой мутации больше не достигается. В round 1, до усиления, 42501 наблюдался. Зависимость от `42501` теперь утверждается не разовой мутацией, а постоянно — тестом `Test_StatArchiverQuery_WithoutPgMonitorFails` на каждом прогоне. Формулировку чек-бокса в task-файле стоит поправить. +4. **Коммит не тот, который планировался.** Round-1 содержимое всех трёх файлов было заметено параллельным агентом задачи 04 в чужой коммит `23a7b1f` (он сделал `git add`/commit поверх моего staged-состояния). `internal/query/archiver.go` с тех пор не менялся, поэтому в коммите ae50197 его нет — там только правки по итогам ревью в двух оставшихся файлах. Содержимое дерева корректное; пострадала только атрибуция. + +**Tech debt:** + +1. Идемпотентность `SetupTestRole` (ветка «роль уже существует») проверяется только процедурой Verification Step 3 — два прогона в одном контейнере, — но не автотестом. Тест на это должен лежать в `internal/postgres/testing_test.go`, а это четвёртый файл, запрещённый критерием приёмки №1. Вынесено в follow-up; оба ревьюера round 2 согласились, что отложить правильно. +2. Роли `pgcenter_test_archiver_monitor` / `pgcenter_test_archiver_norole` остаются на кластере навсегда (teardown запрещён — на нём держится критерий переиспользуемости). Для эфемерных CI-контейнеров это верный размен; на долгоживущем кластере роли надо снимать вручную. Роли `NOLOGIN`, без членов, `SET ROLE` в них требует суперюзера — практического доступа не дают. +3. Регексп `^[a-z_][a-z0-9_]*$` ограничивает синтаксис, но не семантику: `none`, `default`, `public` его проходят. Три из них падают громко, `SET ROLE NONE` — тихо (сброс к session user). Сегодня недостижимо (оба вызова передают константы) и ловится ниже `assertRestrictedSession`. Записано, чтобы регексп позже не читали как более сильный контракт. +4. Рекомендация security-аудитора заменить якоря `^`/`$` на `\A`/`\z` не применена: в Go `regexp.Perl` включает `OneLine`, поэтому обхода через завершающий `\n` нет — оба ревьюера проверили это эмпирически. Ценность правки только в переносимости паттерна в другой язык. + +**Reviews:** + +*Round 1:* +- dev-code-reviewer: approved_with_suggestions, 3 minor → [017-feat-wal-archiver-task-01-dev-code-reviewer-review.json](017-feat-wal-archiver-task-01-dev-code-reviewer-review.json) +- dev-security-auditor: approved, 3 minor → [017-feat-wal-archiver-task-01-dev-security-auditor-review.json](017-feat-wal-archiver-task-01-dev-security-auditor-review.json) +- dev-test-reviewer: needs_improvement, 2 major + 7 minor → [017-feat-wal-archiver-task-01-dev-test-reviewer-review.json](017-feat-wal-archiver-task-01-dev-test-reviewer-review.json) + +*Round 2 (после исправлений):* +- dev-code-reviewer: approved_with_suggestions, 4 minor → [017-feat-wal-archiver-task-01-dev-code-reviewer-review-round2.json](017-feat-wal-archiver-task-01-dev-code-reviewer-review-round2.json) +- dev-security-auditor: approved, 3 minor (round-1 находка 1 закрыта) → [017-feat-wal-archiver-task-01-dev-security-auditor-review-round2.json](017-feat-wal-archiver-task-01-dev-security-auditor-review-round2.json) +- dev-test-reviewer: passed, 5 minor → [017-feat-wal-archiver-task-01-dev-test-reviewer-review-round2.json](017-feat-wal-archiver-task-01-dev-test-reviewer-review-round2.json) + +Одна рекомендация round 1 отклонена по существу: живая проверка предиката `.ready` через `VALUES`-литерал. Она проверяла бы семантику `LIKE` самого PostgreSQL, а не код репозитория, и пропускалась бы на хосте; подстрочная фиксация в бессерверном тесте краснеет на всех тех же мутациях (снятый FILTER, `%.done`, `%ready%`). Test-ревьюер в round 2 согласился и снял рекомендацию, назвав бессерверную фиксацию не более дешёвым, а более удачным инструментом. + +**Verification:** +- `go test -race -p 1 -timeout 300s ./internal/query/... ./internal/postgres/...` в образе `lesovsky/pgcenter-testing:0.0.11` → зелено; `-v`-прогон: **0 пропущенных** подтестов Archiver, все PG 14–19 реально отработали +- Переиспользуемость: два прогона подряд в одной сессии контейнера → оба зелёные (создание роли идемпотентно, второй прогон встречает существующие роли) +- `make lint` (golangci-lint + gosec) → 0 issues; `make vuln` → чисто; `gofmt -l` пусто; `go build -o /dev/null ./cmd` → ок; `grep -n '"testing"' internal/postgres/testing.go` → пусто +- Мутационный контроль — 11 мутаций, каждая применена в контейнере, наблюдалась **красной**, откачена; итоговое дерево побайтово совпадает с исходным: + - M1 `Ncols` 9→8 → `Test_SelectStatArchiverQuery` + - M2 `DiffIntvl {0,0}`→`{2,5}` → `Test_SelectStatArchiverQuery` (и только он) + - M3 удаление колонки `ready` → `Test_StatArchiverQueries` и по счётчику, и по списку имён + - M4 перестановка `ready`/`archived` → `Test_StatArchiverQueries` по **порядку имён** (счётчик не сработал) + `Test_StatArchiverQuery_Structure` + - M5 `coalesce(last_archived_wal,'-')` → `Test_StatArchiverQuery_NullsStayNull` + `Test_StatArchiverQuery_Structure` + - M6 пропуск `SetupTestRole`/`SET ROLE` в позитивном тесте → `Test_StatArchiverQuery_PgMonitorRoleSucceeds` ровно на своём гварде (`current_user` + `rolsuper`), до запроса + - M7 `pgMonitor: false` после `DROP ROLE` на всех шести кластерах → `Test_StatArchiverQuery_PgMonitorRoleSucceeds` (в round 1 — с SQLSTATE 42501, после усиления гварда — на членстве; см. Deviations 3); `Test_StatArchiverQuery_WithoutPgMonitorFails` в том же прогоне остался зелёным → `RESET ROLE` не утекает + - M8 подстановка литерала `0` вместо привилегированного вызова → `Test_StatArchiverQuery_WithoutPgMonitorFails` + - M9 предикат `'%.ready'` → `'%.done'` → `Test_StatArchiverQuery_Structure` + - M10 выдача `pg_monitor` deny-роли после `DROP ROLE` → `Test_StatArchiverQuery_WithoutPgMonitorFails` на «Should be empty, but was [pg_monitor]» + - M11 расширение регекспа имени роли до верхнего регистра → `Test_SetupTestRole_RejectsUnsafeName` + +--- + +## Task 03: Verbose panel backlog on a pg_monitor-accessible function + +**Status:** Done +**Commit:** e3ac49d +**Agent:** основной агент + +**Summary:** `OverviewArchivingBacklog` переведён с `pg_ls_dir('pg_wal/archive_status') AS name` на `pg_ls_archive_statusdir()` — контракт вывода посимвольно тот же (один `bigint`, байты, `count(.ready) × wal_segment_size`), алиас `AS name` снят как мёртвый синтаксис (функция сама отдаёт OUT-колонку `name`). Причина ровно одна: у `pg_ls_dir` ACL `{postgres}` — только суперюзер, поэтому роль с одним `pg_monitor` получала 42501 на каждом тике и панель показывала `n/a` вместо первого сигнала об остановке архивации (Decision 8 отменяет ADR [010]). Механизм деградации в `collectOverviewStat` (собственный `QueryRow`, проглоченная ошибка, `ArchivingBacklogValid`) не тронут — изменился только комментарий над ним. Роли под тесты создаются общим хелпером `postgres.SetupTestRole` из задачи 01; собственного хелпера и inline `CREATE ROLE`/`GRANT` не добавлено, `internal/postgres/testing.go` не изменялся. + +**Deviations:** + +1. **Тестов четыре, а не три.** Сверх TDD Anchor добавлен бессерверный `Test_ArchivingBacklogQuery_Structure` — по major-находке test-ревьюера и по прецеденту задачи 01: на фикстурах `archive_mode=off` и пустой каталог статусов, поэтому любая живая проверка бэклога сводится к `0 >= 0`, и снятие FILTER `.ready` либо множителя `pg_size_bytes(...)` осталось бы зелёным во всех живых тестах. Обе мутации наблюдались красными только на нём. +2. **`assert.True(t, got.Valid)` в collect-тесте заменён на сравнение с суперюзерским baseline'ом.** TDD Anchor называет `Valid`, но `collectOverviewStat` выставляет `s.Valid = true` безусловно (`postgres.go:202`) — ассерция не могла бы покраснеть никогда. Вместо неё снимается образец под суперюзером до `SET ROLE` и сравниваются `TotalSizeValid`/`DatabasesCount`: утверждение «остальная выборка не пострадала» стало фальсифицируемым. +3. **Поправлен пятый комментарий сверх четырёх названных** — `internal/stat/postgres.go:84`, комментарий поля `ArchivingBacklogValid`: он утверждал, что поле становится `n/a` при `archive_mode=off`. Это неверно (каталог статусов создаётся initdb, агрегат возвращает настоящий `0`) и противоречило сразу Decision 11, переписанному комментарию потребителя и `Test_collectOverviewStat_Degradation`. Major-находка code-ревьюера; правка в одну строку внутри файла, уже входящего в скоуп. +4. **Роль для `internal/stat` отдельная** (`pgcenter_test_backlog_collect`), а не общая с `internal/query` (`pgcenter_test_backlog_monitor` / `pgcenter_test_backlog_norole`). DO-блок в `SetupTestRole` не атомарен, и общий объект между пакетами дал бы гонку на `pg_authid` вне `-p 1`; сходящаяся minor-находка code- и test-ревьюеров. +5. **Рекомендация перевести новые тесты на `connectArchiverFixture` отклонена** — форма `NewTestConnectVersion` + `t.Skipf` предписана Implementation Hints самой задачи и совпадает с остальными тестами файла, а аккуратное переиспользование требовало бы переименования хелпера в `archiver_test.go` (файл задачи 01). Test-ревьюер снял находку, записав переименование в follow-up. + +**Tech debt:** + +1. `assertRestrictedSession`/`resetRole` продублированы: в `internal/query` они пакетные (задача 01), в `internal/stat` guard написан инлайном. Объединение требует переноса рядом с `SetupTestRole` в `internal/postgres/testing.go` — файл вне скоупа обеих задач. На финализацию. +2. Ни один тест не наблюдает **ненулевой** бэклог: фикстуры работают с `archive_mode=off`. Поведенческая проверка отнесена к стендовому прогону (задача 10); на этом слое её заменяет структурный тест. +3. Ролей на кластере стало на три больше (`pgcenter_test_backlog_*`), teardown'а нет по Decision 18 — для эфемерных CI-контейнеров это верный размен, на долгоживущем кластере роли снимаются вручную. +4. ADR [010] в `docs/decisions-log.md` по-прежнему называет `pg_monitor` достаточным для `pg_ls_dir` — не трогается в этой задаче намеренно, правка на финализации фичи. + +**Reviews:** + +*Round 1:* +- dev-code-reviewer: approved_with_suggestions, 1 major + 4 minor → [017-feat-wal-archiver-task-03-dev-code-reviewer-review.json](017-feat-wal-archiver-task-03-dev-code-reviewer-review.json) +- dev-security-auditor: approved, 3 minor → [017-feat-wal-archiver-task-03-dev-security-auditor-review.json](017-feat-wal-archiver-task-03-dev-security-auditor-review.json) +- dev-test-reviewer: needs_improvement, 1 major + 4 minor → [017-feat-wal-archiver-task-03-dev-test-reviewer-review.json](017-feat-wal-archiver-task-03-dev-test-reviewer-review.json) + +*Round 2 (после исправлений):* +- dev-code-reviewer: approved_with_suggestions, 0 critical/major → [017-feat-wal-archiver-task-03-dev-code-reviewer-review-round2.json](017-feat-wal-archiver-task-03-dev-code-reviewer-review-round2.json) +- dev-security-auditor: approved, 2 minor (обе вне скоупа задачи) → [017-feat-wal-archiver-task-03-dev-security-auditor-review-round2.json](017-feat-wal-archiver-task-03-dev-security-auditor-review-round2.json) +- dev-test-reviewer: passed, 0 находок → [017-feat-wal-archiver-task-03-dev-test-reviewer-review-round2.json](017-feat-wal-archiver-task-03-dev-test-reviewer-review-round2.json) + +**Verification:** +- `go test -race -p 1 -count=1 -timeout 300s ./internal/query/... ./internal/stat/...` в образе `lesovsky/pgcenter-testing:0.0.11` (PG 14–19) → зелено; два подряд некэшированных прогона в одном контейнере → оба зелёные (создание ролей идемпотентно) +- `go build ./cmd`, `go vet`, `gofmt` по четырём файлам → чисто; `make lint` (golangci-lint + gosec) → 0 issues; `make vuln` → чисто +- Грепы из Verification Steps: `grep -rn "pg_ls_dir" internal/` → только объясняющие комментарии и `NotContains`-ассерция; `grep -rn "has pg_monitor" internal/` → пусто +- Мутационный контроль (каждая применена, наблюдалась **красной**, откачена): + - M1 `FROM` откачен на `pg_ls_dir('pg_wal/archive_status') AS name` → `Test_ArchivingBacklogQuery_PgMonitorRole` и `Test_collectOverviewStat_PgMonitorRole` с `permission denied for function pg_ls_dir (SQLSTATE 42501)`, плюс `Test_ArchivingBacklogQuery_Structure` + - M2 пропуск вызова `SetupTestRole` в позитивном тесте → красный на собственном гварде: `current_user` = `postgres`, `rolsuper` = true, членство пустое — до запроса + - M3 `SetupTestRole(conn, backlogRoleMonitor, false)` на **свежем контейнере** → красный на гварде членства; с дополнительно перевёрнутым ожиданием гварда — красный на самом запросе с `permission denied for function pg_ls_archive_statusdir (SQLSTATE 42501)` + - M4 `SetupTestRole(conn, backlogRoleNoRole, true)` на **свежем контейнере** (отравляет кластер, контейнер выброшен) → `…_NoPrivilegeRole` красный на «Should be empty, but was [pg_monitor]»; с перевёрнутым ожиданием гварда — красный на «An error is expected but got nil» + - M5 снят FILTER `.ready` → `Test_ArchivingBacklogQuery_Structure` + - M6 снят множитель `pg_size_bytes(current_setting('wal_segment_size'))` → `Test_ArchivingBacklogQuery_Structure` +- Мутации M3/M4 прогонялись на копии дерева внутри свежего контейнера, поэтому рабочее дерево ими не затрагивалось; M1/M2/M5/M6 применялись к дереву и откатывались + +--- + +## Task 09: User-facing documentation + +**Status:** Done +**Commit:** a35ada7 +**Agent:** основной агент + +**Summary:** Создан `doc/release-notes/v0.12.0.md` в прозаическом стиле `v0.9.0.md` (`## Release` → `Release date: TBD` → `### Overview` → секции): ломающее изменение `-W` (строковый флаг, `-W w` / `-W a`) с дословной цитатой `report type is not specified, quit` и объяснением, почему pflag съедает `-f` как значение флага; известное ограничение — запись `wal`, снятая **на PG 19** pgcenter'ом старше 0.12, падает при воспроизведении с ошибкой, начинающейся с `diff failed` (записи PG 14–18 не затронуты); заметка про archiving backlog, поданная с выигрыша (`pg_monitor` наконец видит значение), а не с регрессии. Дополнительно, по расширению скоупа Wave 1, обновлена строка `cmd/help.go:170` — реальный help `pgcenter report --help` печатается через `printReportHelp()`/`SetHelpTemplate`, а не cobra, поэтому описание флага из `StringVarP` (задача 04) до пользователя не доходило; строка приведена к паттерну `SELECTOR` соседних `-D`, `-X`, `-P` с сохранением табуляции. + +**Deviations:** + +1. **`0 B` не цитируется как экранный литерал** (известно заранее, п. Post-completion). Спеки пишут новое значение бэклога как `0 B`, но `pretty.Size` в нулевом случае возвращает голую строку `"0"` (`internal/pretty/pretty.go:11-12`). В release notes формулировка прозой — «reports a backlog of zero instead of `n/a`»; `n/a` остаётся цитатой, потому что это настоящий литерал (`naReserve`). Это исправление формулировки спека, а не отход от его смысла. +2. **Секции названы `### Breaking changes` и `### Known limitations`** вместо `### New features`/`### Fixes` из v0.9.0. Скелет (заголовок, `Release date:`, обрамляющее предложение, `### Overview` с одностроечниками, затем разворачивающая проза, `### Other` как список) сохранён дословно, но в этой фиче нет ни новых фич для пользователя, ни фиксов — есть ломающее изменение и ограничение. Задача прямо разрешает «reuse that skeleton, minus the sections you have nothing to put in». +3. **`Release date: TBD`** — строка оставлена явно незаполненной, дата не выдумана (0.12.0 не выпущен). +4. **Добавлен второй файл сверх исходного AC** (`git diff --name-only` в AC требовал ровно один файл). Это не отход, а расширение скоупа, записанное в самой задаче в разделе «Scope added during Wave 1»: `cmd/help.go:170`. Соответственно AC про «ровно один файл» больше не действует; всё остальное из списка исключений (`cmd/report/report.go`, `README.md`, `doc/pgcenter-report-readme.md`, `docs/roadmap-0.12.0.md`) не тронуто. +5. **Про exit code сказано прямо и дважды.** Замер Wave 1 подтверждён локально: `main()` (`cmd/pgcenter.go:66-70`) печатает ошибку и возвращается без `os.Exit(1)`. В notes это отдельный абзац с примером `|| alert` и пустым выходным файлом. Заодно замерена и вторая форма (`-W` последним токеном): `flag needs an argument: 'W' in -W`, **тоже exit 0** — в тексте это оговорено, чтобы читатель не решил, что вторая форма скриптово безопасна. + +**Tech debt:** + +1. Флаги `-W`, `-J`, `-B`, `-L` не документированы нигде, кроме `cmd/help.go` и `--help`: ни README, ни `doc/pgcenter-report-readme.md` не содержат справочника флагов (Decision 13, намеренно вне скоупа). Написание справочника — отдельная неоценённая работа. +2. Расхождение «cobra-описание флага vs `printReportHelp()`» системное: `cmd/help.go` — рукописный шаблон, полностью перекрывающий usage cobra, поэтому любое изменение флага требует правки в двух местах, и ничто не проверяет их согласованность. В этой фиче расхождение поймали случайно, в Wave 1. Кандидат на тест-сверку «каждый флаг из `CommandDefinition.Flags()` встречается в `printReportHelp()`». +3. `doc/Changelog` не обновлён — вне скоупа задачи; актуализация на финализации релиза. + +**Reviews:** + +*Round 1:* +- dev-code-reviewer: не запускался на момент записи (задача документационная, ревью назначается оркестратором) → [017-feat-wal-archiver-task-09-dev-code-reviewer-review.json](017-feat-wal-archiver-task-09-dev-code-reviewer-review.json) + +**Verification:** +- `grep -n 'report type is not specified, quit' doc/release-notes/v0.12.0.md` → 1 попадание (строка 35); `grep -n 'diff failed' …` → 1 попадание (строка 62) — оба половины `verify`-гейта зелёные +- `grep -n '\-W w\|\-W a' …` → 4 попадания; `grep -n 'n/a' …` → 2 попадания; `grep -n '0 B' …` → **пусто** (литерал не просочился) +- Сверка каждой цитаты с источником: `cmd/report/report.go:95` → `report type is not specified, quit`; `internal/stat/postgres.go:594` → `fmt.Errorf("diff failed: %w", err)` (обёрнутая — в тексте «beginning `diff failed`»); `internal/pretty/pretty.go:11` → нулевой случай возвращает `"0"` +- Живой прогон собранного бинарника: `pgcenter report -W -f dump.tar` → `report type is not specified, quit`, `EXIT=0`; `pgcenter report -f dump.tar -W` → `flag needs an argument: 'W' in -W`, `EXIT=0` +- `go build -o /dev/null ./cmd` → ок; `gofmt -l cmd/help.go` → пусто +- `go run ./cmd report --help` → новая строка `-W, --wal SELECTOR` с продолжением `'w' - wal; 'a' - archiver` печатается и выровнена ровно по колонке соседних `-D`/`-X`/`-P` +- `git show --stat HEAD` → ровно два файла: `cmd/help.go`, `doc/release-notes/v0.12.0.md`; коммит сделан явным pathspec'ом, чужие правки в `report/` и `top/` в рабочем дереве не захвачены + +--- + +## Task 08: Golden replay tests for archiver and wal + +**Status:** Done +**Commit:** 91f4cf8 +**Agent:** основной агент + +**Summary:** Добавлены два replay-теста по форме ADR [008] (синтетический in-memory tar + golden, без живого PostgreSQL): `report/report_record_archiver_test.go` с тремя подкейсами и одним golden (экран версионно-независим) и `report/report_record_wal_test.go` в табличной форме bgwriter-теста с golden'ами PG 18 и PG 19 — у экрана `wal` до этой задачи не было replay-покрытия вообще. Ключевое, что пинится у archiver, — **pass-through**: `DiffIntvl{0,0}` короткозамыкает `calculateDelta` до `diff()`, поэтому ассерции держат абсолютные текущие значения (`100003`, `000000010000000000000024`), а не дельты; у `wal` — версионный переключатель, потому что ветки PG 18 и PG 19 расходятся и по `Ncols`, и по `DiffIntvl` (новая `fpi,KiB` встала внутрь диффуемого диапазона, сдвинув его конец с 5 на 6). Подкейс «никогда не архивировали» пинится не golden'ом, а счётом полей ANSI-очищенной строки данных (ровно 5 вместо 9) — это прямое машинное прочтение критерия user-spec «колонки пустые (не `0` и не прочерк)». + +**Deviations:** + +1. **`make lint` не запускается на хосте** — цель падает с `make: golangci-lint: Нет такого файла или каталога` (бинарь лежит в `~/go/bin`, которого нет в `PATH` у `make`). Запущен напрямую: `golangci-lint run --timeout 5m` → **0 issues** по всему дереву (включая незакоммиченные правки соседнего агента), `gofmt -l` по двум новым файлам → пусто, `go vet ./report/...` → чисто. Содержательно AC выполнен, отклонение — в способе запуска. +2. **Мутация B3 краснит обе версии, а не только pg19.** Метки времени тиков заданы один раз в общей табличной обвязке (как в bgwriter-тесте), поэтому разведение тиков на две секунды красит и `pg18`, и `pg19`. Названный критерием `pg19` покраснел (дельты ровно вдвое: `500` → `250`, `240.25` → `120.12`), что и требовалось; покраснение `pg18` — побочный эффект общей обвязки, а не пропуск. Дробить фикстуру ради «только pg19» не стал: это ухудшило бы читаемость таблицы ради ритуала. +3. **A5 не существует и не выдуман** — как и предписано AC. Удаление `archiver` из `view.New()` не может покраснить ни один подкейс: `processData` конфигурирует через собственную одноэлементную карту `view.Views{config.ReportType: v}`, а `Views.Configure` переключается по **ключу карты**, поэтому zero-value `view.View` всё равно получит `Ncols`/`DiffIntvl` из `SelectStatArchiverQuery`; вывод байт-в-байт тот же. Реестр пинится юнит-тестами задачи 05 (`internal/view/view_test.go`), это правильный слой. Замену «эквивалентной» мутацией тоже не изобретал. +4. **Четвёртого golden'а (`report_record_archiver_null.golden`) нет** — намеренно, по тех-спеку: для критерия «пустые ячейки» счёт полей строго сильнее golden'а (golden краснеет по любой причине и молчит о том, по какой; `len(strings.Fields(line)) == 5` краснеет ровно по той, о которой критерий), и мутация B4 это показывает. +5. **Ревьюеры (`dev-code-reviewer`, `dev-security-auditor`, `dev-test-reviewer`) мной не запускались** — цикл ревью ведёт оркестратор фичи, как и по задаче 07 этой же волны. + +**Tech debt:** Нет. Гармонизацию трёх почти одинаковых replay-обвязок (bgwriter / statio / новые archiver+wal) в общий хелпер сознательно не делал — это правка в чужих файлах и отдельное изменение; задача прямо запрещает её здесь. + +**Reviews:** + +*Round 1:* +- Назначаются оркестратором фичи; на момент записи не запускались. + +**Verification:** +- **Red first:** оба файла написаны с ассерциями и **без** golden'ов; `go test ./report/ -run 'Test_app_doReport_(Archiver|WAL)' -v` → все три golden-подкейса красные на `no such file or directory`, все value-сентинелы при этом присутствуют в выводе. Только после этого `-update`. +- Golden'ы прочитаны глазами (`cat -v`): у archiver порядок колонок совпадает с задачей 01 и строка данных абсолютная (`14`, `100003`, `…024`, `8` — не `4`/`3`/`3`); у pg18 семь колонок без `fpi,KiB`; у pg19 восемь, `fpi,KiB` стоит между `fpi` и `buffers_full` и показывает дельту `240.25`. +- `go test ./report/...` на хосте без PostgreSQL → зелено; `-v` подтверждает, что отработали все пять подкейсов (`populated`, `never_archived`, `no_archiver_entries`, `pg18`, `pg19`), не «ok» из кэша. +- Легаси `Test_app_doReport` (старый tar PG 14beta1) → PASS, ни один существующий golden не изменён (`git status` показывает только новые untracked-файлы). +- `go test -race -p 1 ./report/...` на хосте → зелено; тот же прогон в образе `lesovsky/pgcenter-testing:0.0.11` → `ok … 11.975s`; прогон под `TZ=Pacific/Auckland LC_ALL=C` → зелено (фикстуры используют фиксированные метки времени именно для этого). +- `go build -o /dev/null ./cmd` → ок (`go build ./cmd` неприменим: Go отказывается писать исполняемый файл `cmd` рядом с каталогом `cmd/`). +- **Мутационный контроль (каждая применена, наблюдалась красной, откачена).** Продакшн-код: + - **A1** `SelectStatArchiverQuery` → `DiffIntvl{2,2}`: красный **только** `Archiver/populated` — `archived` отрендерился как `3`, сентинел `"100003"` не найден, golden разошёлся. Это и есть гвардия pass-through. + - **A2** ветка PG 19 в `SelectStatWALQuery` обойдена (порог поднят): красный **только** `WAL/pg19` — вывод собран по раскладке PG 18, golden разошёлся; `pg18` зелёный. + - **A3** PG 19 `DiffIntvl{2,6}` → `{2,5}`: красный `WAL/pg19` — `buffers_full` напечатался абсолютным `19` вместо дельты `7`. Пинит, что новая колонка встала **внутрь** диапазона и не вытолкнула `buffers_full`. + - **A4** ветка PG 18 возвращает значения PG 14 (`PgStatWALPG14, 11, {2,9}`): красный `WAL/pg18` — диапазон дотянулся до `stats_age`, `strconv.ParseInt("02:00:00")` уронил семпл, буфер **пустой**, покраснели все сентинелы разом. +- **Мутационный контроль, фикстуры:** + - **B1** переставлены соседние `failed` и `last_failed` в `cols` и в обоих тиках: golden archiver'а разошёлся (в шапке и в строке колонки поменялись местами) — golden действительно пинит порядок колонок, а не факт наличия строки. + - **B2** одна цифра в `curr.archived` (`100003` → `100004`): красный и golden, и сентинел `"100003"`. + - **B3** тики wal разведены на две секунды: красный `pg19` (и `pg18`, см. Deviation 2) — `itv` стал 2 и все дельты ровно уполовинились. Секундный шаг подтверждён как несущий. + - **B4** четыре `sql.NullString{Valid: false}` заменены на `{String: "0", Valid: true}`: счёт полей красный — `expected: 5, actual: 9`, строка данных `Archiver 0 0 0 0 0 0 0 02:00:00`. Дополнительный прогон с `"-"` — тоже красный (`Archiver 0 0 - - 0 - - 02:00:00`). Это и есть гейт пользовательского обещания «колонки пустые». + - **B5** в tar пустого архива добавлены обратно две записи `archiver.*`, больше ничего: `assert.Empty` красный («Should be empty, but was …»). Это отличает «ничего не совпало» от «ничего и не могло совпасть» (сломанная фикстура даёт тот же пустой буфер). + - Побочная находка по ходу B4: первая версия мутации оставляла переменную `null` неиспользованной, и пакет **не компилировался** — тест не запускался, а грепом это читалось как «зелено». Мутация переделана через саму декларацию `null :=`; вывод общий — «нет FAIL» и «тест прошёл» надо различать явно. +- Коммит сделан явным pathspec'ом по пяти своим файлам; параллельные правки соседнего агента в `report/report.go`, `report/describe.go`, `report/report_test.go` и в `top/` в коммит не попали (`git show --stat` → ровно 5 файлов). + +--- + +## Task 07: describe-текст экрана archiver и строка FPI на экране wal + +**Status:** Done +**Commit:** e7192fd +**Agent:** основной агент + +**Summary:** Добавлена константа `pgStatArchiverDescription` (9 колонок в том порядке, в котором их отдаёт `query.PgStatArchiverDefault`, литеральные табы, ссылка на `PG-STAT-ARCHIVER-VIEW`) и запись `"archiver"` в карте `describeReport`; в `pgStatWALDescription` вставлена строка `fpi,KiB` с origin `wal_fpi_bytes` сразу после `fpi` и пометкой `(PG 19+)`. Отсутствие записи в карте не ловится ни одним exit-кодом (`describeReport` печатает `unknown description requested` и возвращает `nil`), поэтому единственный детектор — тест, называющий `"archiver"`; он и был написан первым. Версионная осведомлённость намеренно не вводится: `describeReport` сохраняет сигнатуру `(w io.Writer, report string)`, константы не собираются условно, и строка `fpi,KiB` печатается в том числе при описании архива PG 14–18 — это действующий контракт этой области (константа уже документирует `write`/`sync`, удалённые из `pg_stat_wal` в PG 18), а не регрессия. + +**Deviations:** + +1. **Тестов шесть, а не три.** TDD Anchor называет `Test_describeReport` + два order-теста; по major-находке test-ревьюера добавлены `Test_describeArchiverDetailsURL`, `Test_describeArchiverBlankCells` и `Test_describeWALFPIVersionNote`, а order-тесты переведены с плоского списка имён на таблицу `{name, origin}`. Причина: маркер `"\n- "+name+"\t"` заканчивается на имени колонки, поэтому всё правее первого таба не проверялось — подмена `wal_fpi_bytes`, подмена `pg_ls_archive_statusdir` на `pg_ls_waldir`, снятие `(PG 19+)`, порча URL и удаление формулировки про пустую ячейку оставляли набор зелёным, хотя три из них — дословные acceptance criteria задачи. +2. **Появились хелперы `describeRow` и `assertDescribeColumns`** вместо третьей копии inline-цикла. Задача предписывала копировать форму `Test_describeActivityColumnOrder`; форма (маркер с хвостовым табом, `require.NotEqual(-1, pos)` до сравнения порядка) сохранена дословно, вынесена только механика. Существующий `Test_describeActivityColumnOrder` не трогался: `pgStatActivityDescription` содержит блок caveats, и ассерция точного числа строк на нём не работает. +3. **Проверка набора колонок ужесточена сверх спека** — `assert.Equal(len(columns), strings.Count(text, "\n- "))`: цикл ограничивал набор только снизу, лишняя и продублированная строка оставались невидимыми (`strings.Index` видит только первое вхождение). +4. **Находка code-ревьюера про оговорку о привилегиях в строке `ready` не применена.** Задача прямо назначает называние `pg_ls_archive_statusdir` в колонке origin тем механизмом, который сообщает пользователю о привилегии; в round 2 ревьюер согласился и снял находку сам, приведя решающий довод: строка `waldir_size` на экране wal вызывает `pg_ls_waldir()` того же класса привилегий с тем же экранным отказом и оговорки не несёт — добавление её только в `ready` сделало бы файл несогласованным в другую сторону. +5. **`make lint` и `make vuln` не прогонялись:** ни `golangci-lint`, ни `govulncheck` на хосте не установлены (`Ошибка 127`). Заменены на `go vet ./report/...` и `gofmt` — чисто. Для изменения из одной записи в карте и двух raw-строковых литералов пробел считаю несущественным, но фиксирую, чтобы гейт не считался пройденным. + +**Tech debt:** + +1. Прозаическая (четвёртая) колонка строк по-прежнему не закреплена там, где её не называет отдельный тест. Проверено: переписывание текста описания любой строки, кроме четырёх оговорок про пустую ячейку и `(PG 19+)`, остаётся зелёным. Живого дефекта нет — текст виден в любом `report -d`. +2. Выравнивающие пробелы/табы правее origin не проверяются: `strings.Fields` схлопывает пробельные пробеги, поэтому закреплён ровно один таб — тот, что требует маркер. Замена выравнивающих табов на пробелы оставит набор зелёным при разъехавшейся таблице. Реальный триггер — настройка редактора; `gofmt` содержимое raw-строк не трогает. +3. `Test_describeActivityColumnOrder` остаётся единственной inline-копией цикла — свести её на общий хелпер дороже, чем оставить (см. Deviations 2). +4. `pgcenter report -d` не имеет версионной осведомлённости; строка `fpi,KiB` и строки `write`/`sync`/`write,ms`/`sync,ms` печатаются для версий, где соответствующих колонок нет. Принятый контракт, вне скоупа задачи. + +**Reviews:** + +*Round 1:* +- dev-code-reviewer: approved_with_suggestions, 0 critical/major, 3 minor (все optional) → [017-feat-wal-archiver-task-07-dev-code-reviewer-review.json](017-feat-wal-archiver-task-07-dev-code-reviewer-review.json) +- dev-security-auditor: approved, 0 находок → [017-feat-wal-archiver-task-07-dev-security-auditor-review.json](017-feat-wal-archiver-task-07-dev-security-auditor-review.json) +- dev-test-reviewer: needs_improvement, 1 major + 4 minor → [017-feat-wal-archiver-task-07-dev-test-reviewer-review.json](017-feat-wal-archiver-task-07-dev-test-reviewer-review.json) + +*Round 2 (после исправлений):* +- dev-code-reviewer: approved_with_suggestions, 0 critical/major → [017-feat-wal-archiver-task-07-dev-code-reviewer-review-round2.json](017-feat-wal-archiver-task-07-dev-code-reviewer-review-round2.json) +- dev-test-reviewer: passed, 1 minor (информационный) → [017-feat-wal-archiver-task-07-dev-test-reviewer-review-round2.json](017-feat-wal-archiver-task-07-dev-test-reviewer-review-round2.json) + +**Verification:** +- `go test ./report/... -count=1` → зелено (пакету не нужны PG-фикстуры); `go test ./report/ -run Test_describe -v` → все шесть describe-тестов реально исполняются, не пропущены по `-run` +- `make build` → успешно; `./bin/pgcenter report -d -W a` и `-d -W w` без `-f` и без архива → exit 0, печатают новый текст; `-d -W a | cat -A` → разделители реальные табы (`^I`), origin на колонке 16, description на колонке 40, как в соседних константах +- `go vet ./report/...`, `gofmt` по `describe.go`/`report_test.go` → чисто (`report/report.go:304` gofmt-грязный и в HEAD — не трогался); `make lint`/`make vuln` не прогонялись, см. Deviations 5 +- Мутационный контроль (каждая применена, наблюдалась **красной**, откачена; целостность `describe.go` после отката сверена по md5): + - M1 удаление `"archiver": pgStatArchiverDescription` из карты → `Test_describeReport` с `actual: "unknown description requested"` (команда при этом по-прежнему выходит с нулём — красный тест здесь единственный детектор) + - M2 удаление строки `- last_archived` → `Test_describeArchiverColumnOrder` на ассерции присутствия, не на порядке + - M3a удаление строки `- fpi,KiB` → `Test_describeWALColumnOrder` на присутствии + - M3b перенос `fpi,KiB` перед `fpi` → `Test_describeWALColumnOrder` на порядке («272 is not greater than 358») + - M4 `wal_fpi_bytes` → `wal_fpi_BOGUS` → `Test_describeWALColumnOrder`, «row "fpi,KiB" documents the wrong origin» + - M5 `pg_ls_archive_statusdir` → `pg_ls_waldir` → `Test_describeArchiverColumnOrder`, «row "ready" documents the wrong origin» + - M6 снятие `(PG 19+)` → `Test_describeWALFPIVersionNote` + - M7 добавление лишней строки и M8 дублирование строки `- failed` → ассерция точного числа строк (`expected 9, actual 10`) + - M9 подмена якоря URL на `#PG-STAT-BGWRITER-VIEW` → `Test_describeArchiverDetailsURL` + - M10 удаление оговорки про пустую ячейку только из строки `last_archived` → красным становится ровно подтест `Test_describeArchiverBlankCells/last_archived` + - M11 обрезание строки `- failed` до имени и origin → `require.Greater(len(fields), 3)`, «3 is not greater than 3» + - M12 подмена литерала `'Archiver'` на `'WAL'` в строке `source` → `Test_describeArchiverColumnOrder` + - Дополнительно оба ревьюера round 2 независимо прогнали свои мутации на изолированных копиях дерева (перевод карты на `pgStatWALDescription`, замена табов пробелами, перестановки строк и origin'ов, строка после URL) — все красные на названных тестах +- Коммит сделан явным pathspec'ом по трём своим файлам; параллельные правки соседних агентов в `top/`, `report/report_record_*_test.go` и `report/testdata/*.golden` в коммит не попали (`git show --stat` → ровно 3 файла) + +--- + +## Task 06: TUI navigation — `w` cycle, `W` menu, help + +**Status:** Done +**Commit:** 18fb822 +**Agent:** основной агент + +**Summary:** Добавлен навигационный слой к экрану `archiver` по образцу пятичастного прецедента `j`/`J`: `walNextView` рядом с `statioNextView`, ветка `case "wal":` в диспетче `switchViewTo`, константа `menuWAL` внутри menu-группы iota-блока, её двухпунктовая ветка в `selectMenuStyle`, ветка в `menuSelect` (курсор 0 → `wal`, 1 → `archiver`, default → `wal`, через `viewSwitchHandler` напрямую, ровно один `printCmdline` на путь), биндинг `{"sysstat", 'W', menuOpen(menuWAL, app.config, "")}` и три строки `helpTemplate`. Ключевое решение — не изобретать отдельное имя группы: строка `"wal"` одновременно имя вью и имя цикла (Decision 6), зафиксировано комментарием в единственной точке диспетчеризации. Главное методическое: утверждение прошлой ревизии таск-файла, что `menuSelect` и `keybindings()` невозможно покрыть юнит-тестом, оказалось ложным — нулевой `&gocui.Gui{}` доходит до каждой ветки, поэтому обе половины пути `W` покрыты реальными тестами, а не грепом. + +**Deviations:** + +1. **`keybindings()` разделён на `keybindings()` + `keybindingsList(app *app) []key`** — сверх буквы задачи, которая предполагала только вставку строки в таблицу. Причина: dev-test-reviewer (round 1, major) показал мутацией, что `{"sysstat", 'W', menuOpen(menuStatIO, ...)}` оставляет **весь** фильтрованный прогон зелёным — критерий user-spec «`W` открывает WAL-меню» не был защищён ничем. gocui держит зарегистрированные биндинги неэкспортируемыми и не даёт ни достать, ни выполнить обработчик, поэтому единственный способ проверить, *какой* обработчик несёт клавиша, — вернуть таблицу наружу и вызвать строку. Это ровно паттерн patterns.md «Extract the decision out of the unreachable closure», на который ссылается сама задача; ни один параметр никуда не протянут, `keybindings()` по-прежнему владеет `InputEsc` и циклом регистрации. Мутация воспроизведена мной независимо до правки. +2. **Три теста сверх списка TDD Anchor:** `Test_keybindingsWALOpensMenu` (гоняет обработчик `'W'`, пиннит тип меню, заголовок, пункты и содержимое окна `menuDraw`), `Test_keybindingsWALCycles` (гоняет обработчик `'w'` с экрана `wal`, ждёт `archiver`) и подтест «lower-case 'w' still registered». Закрывают major round 1 и по одному minor каждого ревьюера: заголовок `menuWAL` и строка `'w'` до этого держались только на диффревью. Побочно снято утверждение задачи, что `menuOpen` «не задействован» — он задействован и запиннен. +3. **`wg.Wait()` в `Test_switchViewTo` перенесён внутрь замыкания `t.Run`** (minor dev-code-reviewer). Преднаходящийся дефект харнесса, но нагруженный именно этой задачей: ассерт живёт в горутине, и на красной строке testify звал `t.Errorf` на завершённом сабтесте — падение приходило паникой `Fail in goroutine after Test_switchViewTo/26_ has completed`, без имени строки и с обрывом остального прогона. После правки мутация «удалить `case "wal":`» рапортует `--- FAIL: Test_switchViewTo/26_`. +4. **Ничего из этого не касается запретов задачи:** удалённый `Test_menuConfPathDoesNotLift` не восстановлен, в `editPgConfig` шов не заведён, комментарий в `top/pause_test.go` исправлен ровно в двух названных предложениях (проверено: `top/pgconfig.go:70-74` действительно уходит по раннему возврату на `!db.Local`). +5. **Строка `{current: "activity", to: "wal", want: "wal"}`** оставлена, хотя dev-test-reviewer назвал её дублем `sizes → wal`: она предписана TDD Anchor дословно. + +**Tech debt:** + +1. `menuSelect` (`top/menu.go`) перевалил за 100 строк — новая ветка `menuWAL` его и перевела. Это шесть почти одинаковых вложенных switch, где содержательна только пара «индекс курсора → имя вью»; естественная форма — таблица `map[menuType][]string` с `menuConf`/`menuNone` как единственными явными ветками. Не тронуто сознательно: задача предписывает «Copy, do not improvise», повторяемость преднаходящаяся, а переделка шести веток вне мандата. Кандидат на отдельную уборку (major, optional, dev-code-reviewer round 1 — он же сам пометил это как «NOT a request to change this task»). +2. `app.ui.InputEsc = true` не покрыт ничем: его удаление не валит ни один тест. Преднаходящийся пробел; строка переехала в этом диффе, но перенос доказуемо нейтрален (та же функция, перед тем же циклом, между ними ничего не читает). +3. Идиома «читатель `viewCh` в горутине с таймаутом» продублирована между `top/keybindings_test.go` и `top/menu_test.go`. При двух копиях выносить хелпер преждевременно. + +**Reviews:** + +*Round 1:* +- dev-code-reviewer: approved_with_suggestions, 1 major (optional) + 3 minor → [017-feat-wal-archiver-task-06-dev-code-reviewer-review.json](017-feat-wal-archiver-task-06-dev-code-reviewer-review.json) +- dev-security-auditor: approved, 0 critical/major, 1 minor (информационный — намеренная утечка горутины в тесте) → [017-feat-wal-archiver-task-06-dev-security-auditor-review.json](017-feat-wal-archiver-task-06-dev-security-auditor-review.json) +- dev-test-reviewer: needs_improvement, 1 major + 3 minor → [017-feat-wal-archiver-task-06-dev-test-reviewer-review.json](017-feat-wal-archiver-task-06-dev-test-reviewer-review.json) + +*Round 2 (после исправлений):* +- dev-code-reviewer: approved_with_suggestions, 0 critical/major, 3 minor → [017-feat-wal-archiver-task-06-dev-code-reviewer-review-round2.json](017-feat-wal-archiver-task-06-dev-code-reviewer-review-round2.json) +- dev-test-reviewer: needs_improvement, 0 critical/major, 3 minor → [017-feat-wal-archiver-task-06-dev-test-reviewer-review-round2.json](017-feat-wal-archiver-task-06-dev-test-reviewer-review-round2.json) + +*Round 3 (после исправлений):* +- dev-test-reviewer: passed → [017-feat-wal-archiver-task-06-dev-test-reviewer-review-round3.json](017-feat-wal-archiver-task-06-dev-test-reviewer-review-round3.json) + +**Verification:** +- CI-образ `lesovsky/pgcenter-testing:0.0.11`, `go test -race -p 1 -count=1 -timeout 300s ./top/... ./internal/view/...` → `ok top 9.560s`, `ok internal/view 1.065s` (первый прогон включал и `./record/...` → `ok 7.174s`) +- Хостовой фильтрованный прогон `go test ./top/ -run 'Test_walNextView|Test_switchViewTo|Test_selectMenuStyle|Test_menuSelectWAL|Test_keybindingsWAL|Test_helpTemplate'` → зелено; под `-v` видно, что фильтр действительно захватывает все новые тесты (`-run` в Go регистрозависим — из-за этого `Test_keybindingsWalCycles` был переименован в `…WALCycles`, иначе он молча не запускался). `Test_helpTemplate_pauseEntry`, `_pauseLiftingActions`, `_formatVerbs` проходят без изменений +- `go build -o /dev/null ./cmd`, `go vet ./top/...`, `gofmt -l top/` → чисто; `golangci-lint run ./top/...` → 0 issues +- Мутационный контроль (каждая применена, наблюдалась **красной**, откачена): + - M1 `walNextView` в ветке `case "wal":` возвращает `"wal"` → `Test_walNextView` («expected archiver, actual wal») **и** строка 26 `Test_switchViewTo` + - M2 удалена ветка `case "wal":` из `switchViewTo` → красной стала **только** строка 26 (`wal → archiver`); строки 6 (`sizes → wal`), 27 (`archiver → wal`) и 28 (`activity → wal`) остались зелёными — как и предсказывала задача, диспетч доказывает именно первая строка + - M3 удалён один пункт из стиля `menuWAL` → `Test_selectMenuStyle` («expected 2, actual 1») + - M4 переставлены цели `case 0:` и `case 1:` ветки `menuWAL` → `Test_menuSelectWAL` красный на обеих позициях; `out_of_range` остался зелёным + - M5 удалён сброс `app.config.menu = selectMenuStyle(menuNone)` → `Test_menuSelectWAL` красный на ассерции `menuNone` во всех трёх подтестах + - M6a вторая строка `'W'` в таблице → `Test_keybindingsWAL` на «удаляется ровно один раз»; M6b строка `'W'` удалена → на первом удалении + - M7 возвращён клаузул `'w' WAL,` в строку `r` → `Test_helpTemplate_walEntry` («marker "'w' " must appear on exactly one line») **и** `Test_helpTemplate_replicationEntry` + - M8 возвращён ключевой токен `r,w` → `Test_helpTemplate_replicationEntry` на префиксе ` r ` + - M9 переформулировано описание новой строки → `Test_helpTemplate_walEntry` + - M10 убран `archiver` из оговорки про `Q` → `Test_helpTemplate_resetCaveat` + - M11 `'W'` привязан к `menuOpen(menuStatIO, …)` → `Test_keybindingsWALOpensMenu` (три ассерции сразу) + - M12 удалена строка `'w'` → `Test_keybindingsWALCycles` («no binding for key 119 on view "sysstat"») и подтест «lower-case 'w' still registered» + - M13 переформулирован заголовок `menuWAL` → `Test_keybindingsWALOpensMenu` + - M14 цикл регистрации пропускает строку `'w'` → подтест «lower-case 'w' still registered» (ребро «таблица → gocui», которого `boundHandler` не видит) + - M15 `menuDraw` превращён в no-op → `Test_keybindingsWALOpensMenu` («"" does not contain " pg_stat_wal"») + - M16 строка `'w'` продублирована → подтест «lower-case 'w' still registered» на втором удалении (`execKeybindings` в gocui зовёт **все** совпавшие обработчики, так что дубль означал бы двойной цикл за нажатие) + - M11–M16 прогонялись под тем же фильтром из frontmatter, что и M1–M10; ревьюеры round 2 и round 3 независимо перепрогнали свои мутации на копиях/рабочем дереве и подтвердили красноту +- Визуальная сверка блока `general actions:`: `a,b,f,o` / `r` / `s,t,i` / `d,D` / `x,X` / `p,P` / `j,J` / `w,W` / `S` … — описания в одной колонке (проверено ассерциями `descColumn` между записями, без магических чисел) +- Коммит сделан явным pathspec'ом; параллельные правки соседних агентов в `report/` в него не попали (`git show --stat` → ровно 15 файлов: 9 своих в `top/` и 6 JSON-отчётов) + +--- + +## Task 10: Pre-deploy QA + +**Status:** Done (вердикт — NEEDS WORK) +**Commit:** — +**Agent:** основной агент +**Summary:** Прогнаны обе половины приёмки: автоматическая — полностью зелёная и без единого дефекта фичи, ручная — **не выполнена**, стенд `pgpro@10.128.28.194` жив на уровне сети (ping 32 ms, порт 22 открыт, баннер SSH отдаётся), но отвергает все доступные ключи и всех пользователей. Пройдено 24 критерия из 34 (23 user-spec + 11 tech-spec), 0 провалено, 10 не проверяемы без стенда; полный отчёт — [logs/working/qa-report.json](../../../logs/working/qa-report.json), копия в [017-feat-wal-archiver-qa-report.json](017-feat-wal-archiver-qa-report.json). +**Deviations:** Ручной гейт не выполнен из-за недоступности стенда — не пропущен молча, а зафиксирован как критический блокер с перечнем потерявших доказательство критериев. Замер стоимости (Decision 9) не сделан, поэтому его заранее согласованный исход **не применён**: выбирать между «троттлинг» и «приемлемо» без чисел — это и есть тот shrug, который Decision 9 запрещает. +**Tech debt:** Ничего нового этой задачей не заведено. К регистру на финализации: (1) `report` завершается кодом 0 на всех путях отказа (зафиксировано в release notes, чинить вне фичи); (2) `go.mod` помечает `github.com/spf13/pflag` как `// indirect`, хотя `cmd/report/report_test.go` импортирует его напрямую — любой сборкой с `-mod=mod` дерево пачкается (владелец — задача 04, сборка в readonly-режиме не ломается, проверено на чистом worktree HEAD); (3) `report/report.go` и `internal/stat/procpidstat_test.go` не отформатированы gofmt — ровно так же в `master`, к фиче отношения не имеют, `make lint` их не ловит (в конфиге golangci-lint v2 секция `formatters` пуста). + +**Reviews:** + +Нет — у QA-задачи по каталогу нет ревьюеров, отчёт сам является результатом. + +**Verification:** +- `make build` → `bin/pgcenter` из HEAD `b47ff76`; отдельно собран бинарь из `master` (`25f9754`) для A/B +- Полный `go test -race -p 1 -timeout 300s ./...` в `lesovsky/pgcenter-testing:0.0.11` с фикстурами PG 14–19 → exit 0, все пакеты `ok`, `grep -c 'DATA RACE'` = 0. Повторный прогон под `-v` для учёта скипов: **1085 PASS, 86 SKIP, 0 FAIL** (1171 RUN). Все 86 скипов — подтесты EOL-кластеров (9.4–9.6, 10–13), которых нет в образе (долг [019]); ни одного скипа в диапазоне PG 14–19 и ни одного в тестах фичи. `profile.Test_profileLoop` (флака [030]) прошёл в обоих прогонах +- `export PATH="$PATH:$(go env GOPATH)/bin"` — все четыре инструмента установлены (прежний отчёт «не установлены» был неверен). `make lint` → golangci-lint `0 issues.`, gosec тихо, exit 0. `make vuln` → `No vulnerabilities found`, exit 0 +- Ослабления тестов нет — прочитаны диффы, не саммари: `TestNew` 27→28; `TestView_VersionOK` 27→28 / 24→25, строки ≤PG13 не тронуты; `Test_filterViews` пересчитан в обе стороны **и усилен** новым полем `wantArchiver` (счётчики в одиночку прошли бы и при выпавшем `archiver`); `Test_selectMenuStyle` +строка `menuWAL`; `Test_switchViewTo` +3 строки и `wg.Wait()` перенесён внутрь подтеста +- CLI и report проверены на **живой записи**, не выводом юнит-тестов: `-W a` (3 строки на 4 тика — Decision 12), `-W w`, `-d -W a`, `-d -W w` (строка `fpi,KiB`), `-W x` и `-W -f dump.tar` → `report type is not specified, quit` (exit 0), `-W` последним токеном → `flag needs an argument: 'W' in -W` (exit 0); архив PG 18, записанный бинарём с `master`, воспроизводится новым бинарём **без изменений раскладки**; тот же архив с `-W a` → ни строк, ни заголовка, exit 0 (Decision 15); архив PG 19 от старого бинаря → `diff failed: …` (известное ограничение); запись под ролью без `pg_monitor` обрывается с 42501. Лог — `logs/working/cli-checks.log` +- Ручной прогон на стенде — **не выполнен**. Попытки: ssh по умолчанию, четыре локальных ключа с `IdentitiesOnly`, четыре имени пользователя — везде `Permission denied (publickey,password)`; в `~/.ssh/config*`, в памяти проекта и в документах фичи записи для `10.128.28.194` нет. Без доказательства остались: работающая и сломанная архивация, цикл `w` и меню `W` на живом терминале, **подпись экрана ровно один раз на каждом из двух путей** (главный смысл ручного гейта), узкий терминал с замороженной колонкой `source`, поведение экрана под ролью без `pg_monitor` и весь замер стоимости. Стенд оставлен нетронутым — сессия не открывалась + +--- + +## Task 10 (продолжение): ручная половина приёмки на стенде + +**Status:** Done (вердикт — **GO** с обязательным follow-up по Decision 9) +**Commit:** явным pathspec'ом, только документы +**Agent:** основной агент +**Summary:** Стенд `pgpro@10.128.28.194` ожил и был доступен весь прогон. Ручная половина выполнена +полностью: все семь сценариев пройдены, включая главный — подпись экрана **ровно один раз на каждом +из двух путей входа, проверенных по отдельности**. Итог по фиче: **34 из 34** критериев пройдены +(24 автоматических из прошлого прогона + 10 ручных), 0 провалено, 0 непроверяемых. Замер стоимости +(Decision 9) сделан под предписанными условиями, и его заранее согласованный исход **применён**: +числа плохие → троттлинг возвращается отдельным решением. Отчёт — +[017-feat-wal-archiver-qa-report.json](017-feat-wal-archiver-qa-report.json), сырые захваты — +`logs/working/qa-017-stand/manual-run-2026-08-06.txt`. + +**Инвентаризация стенда:** Debian 12, passwordless sudo, PostgresPro ent 18.4 +(`postgrespro-ent-18.service`), PGDATA `/var/lib/pgpro/ent-18/data`, сокет `/tmp`, `pg_hba` первой +строкой `local all all trust`. `tmux` отсутствовал — поставлен через apt. Инодов свободно 682 882, +поэтому 200 000 файлов-пустышек созданы **полным числом**, без урезания. Важное: собственная БД PPEM +(`ppem`, 48 МБ) живёт **в этом же кластере**, поэтому каждый рестарт ради `archive_mode` прерывал +`ppem.service` — сделано осознанно, всего три рестарта, после каждого все четыре сервиса проверены +`systemctl is-active`. + +**Результаты сценариев:** + +| Сценарий | Итог | Ключевое доказательство | +|---|---|---| +| Работающая архивация (`/bin/true`, 3× switch) | PASS | `archived` 0→3, `archived_age` 00:00:04, `last_archived=000000010000000000000023`, `ready` вернулся к 0, `failed` не изменился | +| Сломанная архивация (`/bin/false`, 3× switch) | PASS | `failed` 6,6,9,9,12 по 25-секундным пробам; `ready`=3 и не убывает; `last_failed` заполнен; `failed_age` 00:00:01 на первом захвате; `archived` неизменен | +| `archive_mode=off` | PASS | четыре текст/age-ячейки — сплошные пробелы (не `0`, не `-`, не `n/a`), проверено нарезкой строки по офсетам заголовка | +| Навигация `w` / `W` / help | PASS | `wal→archiver→wal`, `w` с третьего экрана → `wal`, меню из двух рабочих пунктов, строки `w,W` и `archiver` в оговорке про `Q` | +| **Подпись ровно один раз на каждом пути** | PASS | хоткей `w` → count=1; меню `W` → count=1; дубля нет ни на одном call site | +| Узкий терминал 60 колонок | PASS | `source` заморожена на всех позициях и несёт `ESC[1m`, которого нет у остального заголовка (`capture-pane -e`); `[`/`]` достают все колонки | +| Деградация по правам | PASS | роль без `pg_monitor` → `permission denied for function pg_ls_archive_statusdir` **в области таблицы**, часы идут, ретрай на следующем тике, краха нет; `wal` идентичен на `master` | + +**Decision 9 — замер и применённый исход.** Условия соблюдены буквально: роль `qa_monitor` **только** +с `pg_monitor` (не суперпользователь), verbose включён, параллельно шёл `pgcenter record`, +`archive_mode=off`, 200 005 файлов `.ready`. Числа: + +1. **Время запроса** `count(*) FILTER (WHERE name LIKE '%.ready') FROM pg_ls_archive_statusdir()` — + 895.7 / 1189.8 / 1260.6 / 1054.4 / 1140.6 мс, **среднее ~1108 мс** (на пустом каталоге ~0.9 мс). + Интервал обновления по умолчанию — 1 с, то есть запрос не просто «сопоставим» с интервалом, а на + большинстве проб его превышает. +2. **Задержка смены экрана** (`archiver` открыт, нажатие `w`) — 70 / 240 / 230 мс. Вязкости нет — + единственное число, вышедшее приемлемым. +3. **Цена verbose-панели, A/B** — эффективная частота обновления: фича verbose=off 17 тиков/15 с + (~1.0 с), фича verbose=**on** 8 тиков/15 с (~1.9 с), `master` verbose=on 17 тиков/15 с (~1.0 с). + Само поле: у фичи `3.1T archiving backlog`, у `master` — `n/a` (его `pg_ls_dir()` только для + суперпользователя и падает за 4–15 мс). Задержка нажатия 8–17 мс на обоих: цена платится на + стороне коллектора и проявляется как **вдвое реже обновляющийся экран**, а не как лаг ввода. + +**Исход применён: троттлинг возвращается отдельным решением.** Триггер Decision 9 («время запроса +сопоставимо с интервалом обновления ИЛИ смена экрана заметно вязкая») сработал по первой половине. +A/B показывает следствие напрямую: для роли с `pg_monitor` и включённым verbose фича **вдвое снижает +частоту обновления на всех экранах**, тогда как `master` не затронут. Это и есть предсказанный +переход «ноль → полная стоимость». Троттлинг строится на уже существующей машинерии +`verboseCollectState` + `latencyGuardThreshold` (ADR [010]) и нужен в двух точках: (1) обход +`OverviewArchivingBacklog` в verbose-панели — именно он роняет частоту, потому что выполняется на +каждом экране и каждый тик; (2) под-запрос `.ready` самого экрана `archiver`. Эта задача **решает, но +не реализует** — ни одна задача 1–9 не должна кода. Поскольку исход «троттлинг», а не «приемлемо», +остаток **не** уходит просто в регистр техдолга. + +**Deviations:** +- «Роль вообще без прав» недостижима как сценарий: с пустой ролью pgcenter падает **до** TUI на + стартовой пробе `shared_preload_libraries` (42501) — одинаково на фиче и на `master`. Поэтому + терминальная половина US-12 проверена ролью с `pg_read_all_settings`, но **без** `pg_monitor`: она + запускает TUI и честно упирается в `pg_ls_archive_statusdir()`. Зафиксировано как info, не дефект. +- Плато в росте `failed` — это собственный backoff архиватора PostgreSQL (~60 с после подряд идущих + отказов), а не залипание pgcenter; поэтому «рост от тика к тику» показан на окне ~100 с. + +**Tech debt (только запись, регистр правит `/done`):** +1. **Троттлинг** двух обходов каталога — см. исход Decision 9 выше. Отдельным решением, с числами. +2. `last_archived` / `last_failed` обрезаются до ширины заголовка (`000000010000~`, `0000000100~`), + если экран `archiver` открыт на кластере, который ещё ничего не архивировал: ширины замерзают на + первом батче, а `top` никогда не сбрасывает `view.Aligned`. Имена сегментов различаются только + хвостом, поэтому обрезка убивает ровно информативную часть. **Предсуществующее**: `git diff + develop...HEAD` не трогает ни `align.SetAlign`, ни `printDataCell`, ни `alignViewToResult` + (единственная правка рядом — +19 строк регистрации view). Если войти на экран, когда значения уже + есть, имена видны целиком (проверено: `000000010000000000000023`). +3. Прежние пункты в силе: `report` возвращает 0 на всех путях отказа; `go.mod` и `// indirect` у + `pflag`; два неотформатированных gofmt файла. + +**Verification:** +- Обе сборки сделаны локально и скопированы на стенд под разными именами, вызывались по абсолютному + пути; md5 сверены на обоих концах: фича `9fe71095…` (27d9a8f), `master` `3ee820f5…` (25f9754) +- `tmux new-session -d -s cap -x 190 -y 52`, узкий проход — отдельная сессия `-x 60`; захваты `-p` + для раскладки, `-p -e` там, где проверялся атрибут +- Кода задача не меняла: правки только в `docs/features/017-feat-wal-archiver/` и `logs/working/` +- **Стенд возвращён в исходное состояние и это проверено.** Порядок соблюдён: 200 000 пустышек + удалены **до** восстановления `archive_mode`, затем остальные `.ready` (до базовых 0), + `ALTER SYSTEM RESET archive_mode, archive_command`, один финальный рестарт. Итог: `SHOW + archive_mode` = `off` (source=default, pending_restart=false), `SHOW archive_command` = + `(disabled)` (source=default), `wal_level` = `replica` (source=default), в + `postgresql.auto.conf` ноль строк `archive_`, в `archive_status` 0 файлов. Каталог + `archive_status` был пересоздан на остановленном кластере, потому что инод директории вырос до + 11 579 392 байт от 200k записей (ext4 не ужимает каталоги) — вернулся к 4096. Роли + `qa_monitor`/`qa_plain`/`qa_settings` удалены (после revoke их грантов), таблица `qa_junk1` + удалена, оба бинаря и все скрипты стёрты из `/tmp`, tmux-сервер погашен. `systemctl is-active`: + `postgrespro-ent-18`, `ppem`, `ppem-agent`, `pgpro-otel-collector` — **все active**, БД `ppem` + доступна (49 МБ). Единственный остаток: `stats_reset` у `pg_stat_archiver` новее исходного + 08:50:25, потому что сценарий 3 требовал сброса счётчиков; сами счётчики совпадают с базой (0/0). diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-execution-plan.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-execution-plan.md new file mode 100644 index 00000000..6bb43d9d --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-execution-plan.md @@ -0,0 +1,29 @@ +# Execution Plan: 017-feat-wal-archiver + +**Branch:** `feature/017-feat-wal-archiver` (from `develop`) +**Auto-approved** under autopilot. + +## Waves + +| Wave | Tasks | Parallel? | Notes | +|------|-------|-----------|-------| +| 1 | 01 archiver query, 02 PG 19 FPI, 04 report -W | yes | disjoint files; 01 also adds the shared test-role helper | +| 2 | 03 verbose backlog, 05 view registration | yes | 03 depends on 01's helper; 05 depends on 01+02 | +| 3 | 06 TUI w/W, 07 describe, 08 goldens, 09 release notes | yes | all depend on 05 (09 on 04) | +| 4 | 10 pre-deploy QA | — | full suite in the CI image + manual stand run | + +## Verification environment + +- Full suite: CI image `lesovsky/pgcenter-testing:0.0.11` with PG 14–19 fixtures. +- Host runs must be `-run` scoped: `./top/...` and `./record/...` panic without PostgreSQL. +- Build: `make build` (note: `go build ./cmd` fails — Go refuses to write an executable named `cmd` next to the `cmd/` directory; use `make build` or `go build -o /dev/null ./cmd` as a compile check). Lint needs `export PATH="$PATH:$(go env GOPATH)/bin"`. + +## Review rule + +Every task names concrete mutations of production code and the test that must redden on each. +Implementers run them, observe red, revert. "Looks correct" is not accepted. + +## User checks (Wave 4) + +Manual stand run on `pgpro@10.128.28.194`: three archiving states, navigation on both entry paths, +60-column terminal, and the cost measurement under a pg_monitor-only role with verbose on. diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-interview.yml b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-interview.yml similarity index 100% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-interview.yml rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-interview.yml diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-metrics-summary.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-metrics-summary.md new file mode 100644 index 00000000..677fe28e --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-metrics-summary.md @@ -0,0 +1,85 @@ +# Metrics Summary: 017-feat-wal-archiver + +## Context + +| Dimension | Value | +|-----------|-------| +| Model | Opus 5 (1M context) | +| Feature size | M | +| Started | 2026-08-05 | +| Completed | 2026-08-06 | + +**Caveat:** phase timestamps were recorded manually by the orchestrator and are approximate to ~5 +minutes. Human wait was instrumented only in the `user_spec` phase (the interview); the approval +gates of the later phases are counted as touch time, so flow efficiency below is an upper bound. + +## Timeline + +| Phase | Duration (min) | Touch (min) | Human wait (min) | +|-------|---------------|-------------|------------------| +| User Spec | 70 | 48 | 22 | +| Tech Spec | 50 | 50 | 0 | +| Task Decomposition | 295 | 295 | 0 | +| Feature Execution | 450 | 450 | 0 | +| Done | 17 | 17 | 0 | +| **Sum of phases** | **882** | **860** | **22** | +| Lead time (first start → last end) | 1130 | | | +| Idle between phases | 248 | | | + +**Flow efficiency: 76.1%** (860 touch / 1130 lead). The 248 idle minutes are two overnight gaps +between decomposition, execution and finalization, not a queue. + +Task decomposition took as long as it did for a reason worth keeping: two validation rounds found +tasks whose mutations targeted files those same tasks were forbidden to touch, which is a defect that +only shows up when a validator actually tries to run the plan. + +## Quality + +| Metric | Value | +|--------|-------| +| Validation rounds | user_spec: 2, tech_spec: 3, task_decomposition: 2 | +| Validation findings (crit/major/minor) | 1 / 18 / 74 (12 reports) | +| Review rounds by task | 01:2, 02:1, 03:2, 04:2, 05:2, 06:3, 07:2, 08:—, 09:—, 10:— | +| Review findings (crit/major/minor) | 0 / 15 / 96 (36 reports) | +| First pass rate | 0% (0 of 7 reviewed tasks cleared round 1 without a major) | + +**A 0% first-pass rate here is a signal about the reviewers, not about broken code.** Every task +reached round 1 with a green suite; what the majors found was almost uniformly the same class — a +test that passes and cannot fail. Task 02: none of the four TDD-anchor asserts pinned the `/1024` +conversion, so replacing `round(wal_fpi_bytes / 1024, 2)` with the bare column left the suite green +while the screen would have shown bytes under a KiB header. Task 06: binding `W` to the wrong menu +left the entire filtered run green, because gocui keeps registered handlers unexported — which is +what forced `keybindings()` to be split so the table row itself became callable. Tasks 01 and 03: the +fixtures run with an empty `archive_status` directory, so a live check could not tell +`count(*) FILTER (WHERE name LIKE '%.ready')` from a bare `count(*)`, and the predicate had to be +pinned by a server-free structural test instead. Three tasks (08, 09, 10) had no reviewer cycle — +golden tests, documentation and QA — so the rate is computed over seven. + +Tasks 06 needed a third round; every other reviewed task closed in two. + +## Volume + +| Metric | Value | +|--------|-------| +| Interview questions | 12 | +| Tasks | 10 (in 4 waves) | +| Agents spawned | ~90 (implementers, reviewers, validators) | +| Commits | 29 (on the feature branch) | + +## Verification + +| Gate | Result | +|------|--------| +| `make test` (`-race -p 1`, PG 14–19 fixtures in the CI image) | pass — 1085 PASS / 86 SKIP / 0 FAIL, 0 data races | +| `make lint` (golangci-lint + gosec) | pass, 0 issues | +| `make vuln` (govulncheck) | pass | +| Acceptance criteria, automated half | 24 of 34 | +| Acceptance criteria, stand run | 10 of 10, 0 FAIL | + +All 86 skips are EOL-cluster subtests (PG 9.4–13) absent from the test image — debt [019] — with no +skip anywhere in the PG 14–19 range and none in the feature's own tests. + +The manual gate ran twice: the first attempt found the stand unreachable and was reported as a +blocker rather than waved through, and the run was repeated on 2026-08-06 once it came back. That is +also where the `archive_status` cost measurement was taken (200 005 `.ready` files) and where the +truncation defect now registered as debt [035] was found by A/B against a `master`-built binary. diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-metrics.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-metrics.json similarity index 51% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-metrics.json rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-metrics.json index b3675919..7af09f48 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-metrics.json +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-metrics.json @@ -8,7 +8,7 @@ "feature_size": "M", "model": "Opus 5 (1M context)", "date_started": "2026-08-05T16:59:47Z", - "date_completed": "" + "date_completed": "2026-08-06T11:50:00Z" }, "phases": { "user_spec": { @@ -35,33 +35,64 @@ "human_wait_time_min": 0, "steps": {} }, - "feature_execution": null, - "done": null + "feature_execution": { + "start_time": "2026-08-06T02:00:00Z", + "end_time": "2026-08-06T09:30:00Z", + "duration_min": 450, + "touch_time_min": 450, + "human_wait_time_min": 0, + "steps": {} + }, + "done": { + "start_time": "2026-08-06T11:33:00Z", + "end_time": "2026-08-06T11:50:00Z", + "duration_min": 17, + "touch_time_min": 17, + "human_wait_time_min": 0, + "steps": {} + } }, "quality": { "validation_rounds": { "user_spec": 2, "tech_spec": 3, - "task_decomposition": 1 + "task_decomposition": 2 }, "validation_findings": { - "critical": 0, - "major": 0, - "minor": 0 + "critical": 1, + "major": 18, + "minor": 74 + }, + "review_rounds": { + "task_01": 2, + "task_02": 1, + "task_03": 2, + "task_04": 2, + "task_05": 2, + "task_06": 3, + "task_07": 2 }, - "review_rounds": {}, "review_findings": { "critical": 0, - "major": 0, - "minor": 0 + "major": 15, + "minor": 96 }, - "first_pass_rate_pct": null + "first_pass_rate_pct": 0 }, "volume": { "interview_questions": 12, "tasks_count": 10, "waves_count": 4, - "agents_spawned": 36, - "commits_count": 13 + "agents_spawned": 90, + "commits_count": 29 + }, + "summary": { + "total_lead_time_min": 1130, + "sum_of_phase_durations_min": 882, + "idle_between_phases_min": 248, + "total_touch_time_min": 860, + "total_human_wait_time_min": 22, + "flow_efficiency_pct": 76.1, + "note": "Phase timestamps recorded manually by the orchestrator; approximate to ~5 min. Only the user_spec phase had human wait instrumented." } } diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-qa-report.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-qa-report.json new file mode 100644 index 00000000..0260a006 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-qa-report.json @@ -0,0 +1,372 @@ +{ + "feature": "017-feat-wal-archiver", + "task": "10 — Pre-deploy QA", + "date": "2026-08-06", + "branch": "feature/017-feat-wal-archiver", + "headCommit": "b47ff76", + "status": "passed", + "statusReason": "The manual gate was run on 2026-08-06 against pgpro@10.128.28.194 (Debian 12, PostgresPro ent 18.4) with tmux at fixed geometry and both the feature (27d9a8f) and master (25f9754) binaries shipped and invoked by absolute path. All seven stand scenarios passed, including the once-per-path subtitle check on both entry paths separately. The Decision 9 cost measurement was taken under the conditions Decision 9 fixes and its outcome applied: the numbers are bad, so throttling is reopened as its own decision. One MAJOR finding (feature-introduced, A/B-confirmed) and one MINOR pre-existing rendering observation were recorded.", + "verdict": "GO — all 34 acceptance criteria pass, 24 in the automated half and the remaining 10 on a live stand. One mandatory follow-up: Decision 9's measurement came out on the BAD side of its own pre-agreed threshold, so throttling returns as its own decision (see decision9 and findings[0]). No task of this feature owes a code fix; the throttle is a new decision, exactly as Decision 9 specified. The project owner decides whether it lands before or after the merge.", + "environment": { + "status": "healthy", + "rebuildAttempts": 1, + "note": "pgcenter is a CLI: Phase 1 collapses to a fresh build plus the CI image fixtures.", + "services": [ + "bin/pgcenter: rebuilt from HEAD 27d9a8f, 18233101 bytes, md5 9fe71095fd812d80341970be37518410", + "master A/B binary: built from master 25f9754, 18215768 bytes, md5 3ee820f5e4d31de16e37826d4ee1c634", + "lesovsky/pgcenter-testing:0.0.11 with PG 14-19 fixtures on 21914-21919: up (automated half)", + "stand: Debian 12, PostgresPro ent 18.4, unit postgrespro-ent-18.service, tmux 3.3a installed via apt", + "fixture archive_mode=off, archive_command=(disabled) — confirmed live, which is why the stand run is mandatory" + ], + "standAccess": "OK — ssh key auth worked on 2026-08-06; connectivity was stable for the whole run.", + "fixCommits": [] + }, + "summary": { + "totalChecks": 34, + "passed": 34, + "failed": 0, + "notVerifiable": 0, + "criticals": 0, + "majors": 1, + "minors": 3 + }, + "testSuite": { + "status": "passed", + "details": "go test -race -p 1 -timeout 300s ./... inside lesovsky/pgcenter-testing:0.0.11 with PG 14-19 fixtures up: exit 0, every package ok, 0 WARNING: DATA RACE. Verbose re-run for skip accounting: 1085 PASS, 86 SKIP, 0 FAIL (1171 RUN). All 86 skips are EOL-cluster subtests (PG 9.4-9.6, 10-13) absent from the image — debt [019]; none falls in the PG 14-19 range and none belongs to feature 017. Every archiver and wal subtest ran on all six live versions. profile.Test_profileLoop (flaky, debt [030]) PASSED on both full runs.", + "logs": [ + "logs/working/full-test-run.log", + "logs/working/full-test-verbose.log" + ] + }, + "lint": { + "status": "passed", + "details": "PATH extended with $(go env GOPATH)/bin — golangci-lint, gosec, govulncheck and gofmt are all installed there (an earlier task's 'not installed' report was wrong). make lint: golangci-lint '0 issues.', gosec quiet, exit 0. make vuln: 'No vulnerabilities found', 0 vulnerabilities called, exit 0." + }, + "testWeakeningAudit": { + "status": "passed", + "details": "Diffs read line by line, not summaries. TestNew 27->28. TestView_VersionOK 190000/160000 27->28, 140000 24->25, PG13 and below unchanged (archiver is MinRequiredVersion PG14) plus a new failure message naming the version. Test_filterViews: wantV 27->28 / 18->19 / 24->25 on PG14+, wantN 8->9 / 11->12 / 13->14 on PG13 and below, plus a NEW wantArchiver membership field so the counts cannot be satisfied by dropping archiver while another gate loosens. Test_selectMenuStyle gained a menuWAL row (want 2). Test_switchViewTo gained three rows and moved wg.Wait() inside the subtest closure so a failing row names itself instead of panicking. Every change is a new correct number or a strengthening; nothing deleted, loosened or removed." + }, + "acceptanceCriteria": [ + { + "id": "US-1", + "criterion": "archiver shows 9 columns in the order source, ready, archived, last_archived, archived_age, failed, last_failed, failed_age, stats_age", + "status": "passed", + "evidence": "Test_StatArchiverQueries asserts the live pg_wire column names against the locked list (by NAME, not length) on PG 14/15/16/17/18/19 — all six PASS. Real replay confirms the same header: logs/working/cli-checks.log:52. ALSO STAND 2026-08-06: live header captured as 'source ready archived last_archived archived_age failed last_failed failed_age stats_age' — 9 columns in the specified order." + }, + { + "id": "US-2", + "criterion": "archive_mode=off: counters are zero and last_archived / archived_age / last_failed / failed_age are BLANK (not 0, not a dash)", + "status": "passed", + "evidence": "The fixtures run archive_mode=off (verified live: 'off' / '(disabled)'). Test_StatArchiverQuery_NullsStayNull pins the four columns as SQL NULL and the two counters as '0' on all six versions. Rendered proof through the shared print path: cli-checks.log:54 — 'Archiver 0 0 0 11 days 23:04:05'. The TUI screen itself was not captured (stand); the report and TUI share the align/print path. ALSO STAND 2026-08-06 (TUI half): with archive_mode=off and pg_stat_reset_shared('archiver'), field-slicing the rendered row gave last_archived=[ ], archived_age=[ ], last_failed=[ ], failed_age=[ ] — all spaces, not '0', not '-', not 'n/a'; archived=0, failed=0; the subtitle was present on the cmdline exactly once on entry." + }, + { + "id": "US-3", + "criterion": "working archiving (/bin/true, 3x pg_switch_wal): archived +>=3, archived_age <= 00:00:30, last_archived set, ready back to 0, failed unchanged", + "status": "passed", + "evidence": "STAND 2026-08-06 (feature 27d9a8f, PostgresPro ent 18.4). archive_mode=on + archive_command='/bin/true', restart, 3x pg_switch_wal(): screen went 'Archiver 0 0 0' -> 'Archiver 0 3 000000010000~ 00:00:04 0'. SQL: archived_count=3, last_archived_wal=000000010000000000000023, age=00:00:05, ready files=0, failed_count=0. archived +3, archived_age<00:00:30, last_archived set, ready back to 0, failed unchanged. logs/working/qa-017-stand/manual-run-2026-08-06.txt" + }, + { + "id": "US-4", + "criterion": "broken archiving (/bin/false, 3x pg_switch_wal): failed grows tick to tick, ready >= 3 and not falling, last_failed set, failed_age <= 00:00:30, archived unchanged", + "status": "passed", + "evidence": "STAND 2026-08-06. archive_command='/bin/false' (reload only) + 3x pg_switch_wal(): screen 'failed' column over 25 s samples = 6,6,9,9,12 (SQL 6,7,9,9,12) -> grows; the plateaus are PostgreSQL's own archiver retry backoff (~60 s after consecutive failures), not a pgcenter stall. ready=3 at every tick and never falls; last_failed=000000010000000000000024 rendered; failed_age=00:00:01 on the first capture after the switches (<00:00:30); archived stayed 3." + }, + { + "id": "US-5", + "criterion": "'w' on wal opens archiver; 'w' on archiver opens wal; 'w' from any other screen opens wal", + "status": "passed", + "evidence": "STAND 2026-08-06, tmux 190x52. 'w' on wal -> archiver header 'source ready archived last_archived archived_age failed last_failed failed_age stats_age'; 'w' on archiver -> wal header 'source waldir_size wal,KiB records fpi buffers_full stats_age'; 'd' (databases) then 'w' -> wal. All three directions captured." + }, + { + "id": "US-6", + "criterion": "'W' opens a two-item menu, each item opening its screen", + "status": "passed", + "evidence": "STAND 2026-08-06. 'W' opened '┌─ Choose WAL / archiver mode (Enter to choose, Esc to exit): ─┐' with exactly two items, '│ pg_stat_wal │' and '│ pg_stat_archiver │'. Down+Enter opened the archiver screen; Enter on item 1 opened the wal screen. Both items work." + }, + { + "id": "US-7", + "criterion": "'Show archiver statistics (requires archive_mode=on)' appears EXACTLY ONCE per entry path, verified on BOTH paths separately (w hotkey and W menu)", + "status": "passed", + "evidence": "STAND 2026-08-06 — THE GATE CHECK, both entry paths captured SEPARATELY and grepped with 'grep -o ... | wc -l'. Hotkey 'w' path: count = 1. 'W' menu path: count = 1 (message on cmdline, pane line 5). No duplicate on either call site (switchViewTo vs menuSelect->viewSwitchHandler). Lifetime measured: present at 0.3 s and 1.3 s, cleared from ~2 s by printCmdline's clear timer — by design, shared with every screen's Msg." + }, + { + "id": "US-8", + "criterion": "the help screen carries the w,W line and the Q line lists archiver among the non-resettable stats", + "status": "passed", + "evidence": "The help screen IS the helpTemplate constant. Test_helpTemplate_walEntry pins the line word for word and its position directly after j,J with a shared description column; Test_helpTemplate_replicationEntry pins that the 'r' line lost its 'w' clause; Test_helpTemplate_resetCaveat pins 'pg_stat_io, bgwriter, wal, archiver'. Mutations M7-M10 confirmed red. A live capture of the rendered page was not taken (stand). ALSO STAND 2026-08-06 (live help page): line 11 \"w,W 'w' pg_stat_wal / pg_stat_archiver switch, 'W' WAL statistics menu.\" and line 37 \"('Q' does not reset shared stats: pg_stat_io, bgwriter, wal, archiver).\"" + }, + { + "id": "US-9", + "criterion": "pgcenter record writes archiver entries with no recorder change", + "status": "passed", + "evidence": "Real 4-tick recording on the PG 18 fixture contains archiver..json for every tick (cli-checks.log:11-14); the same on PG 14 and PG 19. git diff develop...HEAD touches no file under record/ except record_test.go." + }, + { + "id": "US-10", + "criterion": "report -W a replays recorded data; -W w gives the previous pg_stat_wal report; -d -W a prints the column description", + "status": "passed", + "evidence": "Real recording, real replay. -W a: 9-column header + 3 data rows, exit 0 (cli-checks.log:48-59). -W w: the unchanged 7-column PG 18 layout, exit 0 (:64-75). -d -W a: the full archiver description, exit 0 (:77-92)." + }, + { + "id": "US-11", + "criterion": "-W as the last token fails with \"flag needs an argument: 'W' in -W\"; -W -f dump.tar and -W x end with 'report type is not specified, quit'; exit code is 0 in all three cases", + "status": "passed", + "evidence": "Measured on the real binary: '-W x' -> report type is not specified, quit, exit=0; '-W -f rec18.tar' -> same message, exit=0; trailing '-W' -> flag needs an argument: 'W' in -W, exit=0 (cli-checks.log:99-109). Exit 0 is the documented pre-existing behaviour, quoted in the release notes and going to the tech-debt register." + }, + { + "id": "US-12", + "criterion": "a role without pg_monitor: the archiver screen shows the PostgreSQL error instead of a table, pgcenter does not crash and retries next tick; the wal screen behaves as before the feature", + "status": "passed", + "evidence": "STAND 2026-08-06, terminal half. Role qa_settings (pg_read_all_settings, NO pg_monitor): the archiver screen renders 'ERROR: permission denied for function pg_ls_archive_statusdir' IN THE TABLE AREA; header clock advanced 14:00:36 -> 14:00:40 with the error still shown and the TUI alive => retries next tick, no crash. A/B: on the master binary the wal screen shows the identical 'permission denied for function pg_ls_waldir' => wal unchanged, no regression. NOTE: a role with NO privileges at all (qa_plain) cannot start pgcenter on EITHER binary — it aborts on the startup 'shared_preload_libraries' probe (42501) — which is pre-existing and why pg_read_all_settings was used." + }, + { + "id": "US-13", + "criterion": "pgcenter record under a role without pg_monitor stops the whole recording with an error (existing recorder behaviour, not a defect of this feature)", + "status": "passed", + "evidence": "Measured: a purpose-created LOGIN role with no pg_monitor -> 'ERROR: permission denied to examine \"shared_preload_libraries\" (SQLSTATE 42501)', no tar written (cli-checks.log:200-203). The recording aborts even before the archiver query, on properties collection — pre-existing recorder behaviour, recorded as expected." + }, + { + "id": "US-14", + "criterion": "report -W w on a pre-0.12 archive (PG 18 and older) prints the previous wal column set — no new column appears", + "status": "passed", + "evidence": "A PG 18 archive recorded with the master-built binary and replayed by the feature binary prints the identical 7-column header and three data rows, exit 0 (cli-checks.log:131-142); byte-identical to what the master binary prints from the same tar (:123-129). The PG 19 counterpart correctly fails with 'diff failed: convert ...' (:151-158) — the accepted, release-noted limitation." + }, + { + "id": "US-15", + "criterion": "report -W a on an archive with no archiver entries prints no data rows and no column header (only the three INFO lines) and exits 0", + "status": "passed", + "evidence": "The master-recorded tar has no archiver entries; replaying it with -W a prints exactly 3 lines, all INFO, no header, exit 0 (cli-checks.log:144-149). Decision 15 as specified." + }, + { + "id": "US-16", + "criterion": "under a pg_monitor-only role (not superuser) the archiver query succeeds and returns 9 columns, and the [010] backlog aggregate returns a number — proven by test through SET ROLE, not only on the stand", + "status": "passed", + "evidence": "Test_StatArchiverQuery_PgMonitorRoleSucceeds (all six versions) and Test_ArchivingBacklogQuery_PgMonitorRole, plus Test_collectOverviewStat_PgMonitorRole asserting ArchivingBacklogValid=true through the real collector. Each guards with assertRestrictedSession before the query — not a superuser, pg_monitor and nothing else — so the test cannot silently decay into a superuser run." + }, + { + "id": "US-17", + "criterion": "under a role without pg_monitor the archiver query fails with permission denied for function pg_ls_archive_statusdir — all-or-nothing degradation proven by test", + "status": "passed", + "evidence": "Test_StatArchiverQuery_WithoutPgMonitorFails pins SQLSTATE 42501 AND the function name in the message (so a syntax typo cannot pass it) on PG 14-19; Test_ArchivingBacklogQuery_NoPrivilegeRole is the same check for the aggregate." + }, + { + "id": "US-18", + "criterion": "on PG 19 the wal screen carries fpi,KiB right after the fpi counter, rendered as a per-interval delta; on PG 14-18 the column set, order and diffed range are unchanged", + "status": "passed", + "evidence": "Test_SelectStatWALQuery_PG19ColumnOrder and Test_SelectStatWALQuery_LegacyBranchesUntouched; TestViews_Configure pins PgStatWALPG19/8/{2,6} at 190000 and PgStatWALPG14/11/{2,9} at 140000; Test_StatWALQueries runs the query live on all six versions. End to end on a real PG 19 recording: header source waldir_size wal,KiB records fpi fpi,KiB buffers_full stats_age (cli-checks.log:166), with fpi,KiB inside DiffIntvl{2,6} and stats_age outside it. The delta arithmetic itself is pinned by report/testdata/report_record_wal_pg19.golden (240.25)." + }, + { + "id": "US-19", + "criterion": "golden replay tests exist for archiver and for the wal screen on PG 18 and PG 19", + "status": "passed", + "evidence": "report/testdata/report_record_archiver.golden (9 columns), report_record_wal_pg18.golden (7 columns, no fpi,KiB), report_record_wal_pg19.golden (8 columns with fpi,KiB after fpi). Test_app_doReport_Archiver (populated + never_archived) and Test_app_doReport_WAL pass; task 8 recorded mutation control showing each golden red on a perturbed layout." + }, + { + "id": "US-20", + "criterion": "full make test green on the PG 14-19 fixtures; make lint and make vuln clean", + "status": "passed", + "evidence": "See testSuite and lint above: exit 0 / 0 races / 0 FAIL, golangci-lint 0 issues, gosec quiet, govulncheck 0 vulnerabilities." + }, + { + "id": "US-21", + "criterion": "under a pg_monitor role the [010] verbose panel shows a numeric backlog instead of n/a; unchanged for a superuser on a normal cluster; on a missing archive_status directory it now shows a bare 0, not '0 B'", + "status": "passed", + "evidence": "Test_collectOverviewStat_PgMonitorRole: ArchivingBacklogValid true under a pg_monitor-only role, and TotalSizeValid/DatabasesCount identical to the superuser baseline. Superuser path unchanged: Test_collectOverviewStat and _Degradation. The bare-0 rendering is code-level: pretty.Size(0) returns \"0\" (internal/pretty/pretty.go:12). The missing-directory case itself was measured during spec work on live PG 18.4 (Decision 11) and was NOT re-measured here; the A/B stand observation of the panel is part of the blocked cost run. ALSO STAND 2026-08-06 (A/B observation): under qa_monitor with verbose on, the feature binary's panel shows '3.1T archiving backlog' where the master binary shows 'n/a archiving backlog'. This is the zero->full change Decision 9 predicted, and its measured price is finding[0]." + }, + { + "id": "US-22", + "criterion": "report -d -W w documents the fpi,KiB column", + "status": "passed", + "evidence": "Real run: '- fpi,KiB\\twal_fpi_bytes\\tAmount of WAL generated by full page images, in KiB (PG 19+)' printed directly under the fpi row, exit 0 (cli-checks.log:94-97). Version-independence of describe text is the existing contract, stated in the row itself." + }, + { + "id": "US-23", + "criterion": "updated: the -W flag help line, release notes 0.12.0 (flag type change with the literal refusal text, and the PG 19 archive limitation with the literal 'diff failed'), and the built-in help screen", + "status": "passed", + "evidence": "pgcenter report --help prints ' -W, --wal SELECTOR show pg_stat_wal / pg_stat_archiver statistics ... 'w' - wal; 'a' - archiver' (cli-checks.log:206-208); the pflag usage string is pinned by Test_walFlagDefinition. doc/release-notes/v0.12.0.md carries both literals ('report type is not specified, quit', 'diff failed'), the exit-0 warning and the missing-archive_status note. Help screen: US-8. README deliberately untouched (Decision 13)." + }, + { + "id": "TS-1", + "criterion": "go test -race -p 1 ./... green inside the CI image with PG 14-19 fixtures, zero race reports", + "status": "passed", + "evidence": "exit 0, every package ok, grep -c 'DATA RACE' = 0 on both runs." + }, + { + "id": "TS-2", + "criterion": "make lint (golangci-lint + gosec) and make vuln clean on the host", + "status": "passed", + "evidence": "0 issues / quiet / no vulnerabilities, all exit 0. Note findings[2]: two files are gofmt-dirty, but identically so on master, and the golangci-lint v2 config enables no formatters." + }, + { + "id": "TS-3", + "criterion": "no existing test's expectations weakened: TestNew, TestView_VersionOK, Test_filterViews, Test_selectMenuStyle, Test_switchViewTo all carry new CORRECT numbers", + "status": "passed", + "evidence": "See testWeakeningAudit." + }, + { + "id": "TS-4", + "criterion": "the archiver query executes on every version PG 14-19 and returns exactly 9 columns", + "status": "passed", + "evidence": "Test_StatArchiverQueries — six live subtests, all PASS, each asserting 9 named columns and exactly one row. No SKIP in the range." + }, + { + "id": "TS-5", + "criterion": "the PG 19 wal query returns exactly 8 columns; PG 14-17 and PG 18 branches unchanged", + "status": "passed", + "evidence": "Test_SelectStatWALQuery (190000 -> 8 / {2,6}, 180000 -> 7 / {2,5}, 140000-170000 -> 11 / {2,9}), Test_SelectStatWALQuery_LegacyBranchesUntouched, Test_StatWALQueries live on six versions." + }, + { + "id": "TS-6", + "criterion": "SelectStatArchiverQuery is version-independent and its unused parameter is named _", + "status": "passed", + "evidence": "internal/query/archiver.go: func SelectStatArchiverQuery(_ int) (string, int, [2]int). revive's unused-parameter rule is enabled and lint is clean." + }, + { + "id": "TS-7", + "criterion": "the archiver view sets NotRecordable: false and needs no change in record/record.go", + "status": "passed", + "evidence": "The registration omits NotRecordable (zero value false) and TestNew_ArchiverView asserts False. git diff develop...HEAD contains no record/record.go. Confirmed by behaviour: archiver entries appear in a real recording (US-9)." + }, + { + "id": "TS-8", + "criterion": "golden replay tests exist for archiver and for wal at PG 18 and PG 19, and fail if the layout changes", + "status": "passed", + "evidence": "See US-19." + }, + { + "id": "TS-9", + "criterion": "column-name-driven assertions: tests reference columns by header name where the layout is pinned", + "status": "passed", + "evidence": "archiverColumns is a named list compared against the live field descriptions in Test_StatArchiverQueries and indexes the NULL-ness map in Test_StatArchiverQuery_NullsStayNull; Test_SelectStatWALQuery_PG19ColumnOrder asserts the alias order in the query text; the report goldens carry the rendered header line." + }, + { + "id": "TS-10", + "criterion": "a test proves the archiver query succeeds under a pg_monitor-only role and fails without it, and that the verbose backlog aggregate returns a number under the same role", + "status": "passed", + "evidence": "See US-16 and US-17. The load-bearing part is assertRestrictedSession / its stat-package twin: each asserts NOT superuser, pg_monitor present, and membership exactly [pg_monitor] BEFORE the subject query, so deleting the SET ROLE turns the test red instead of leaving it silently passing as the fixture superuser." + }, + { + "id": "TS-11", + "criterion": "all four stale comments about the old function's privileges are corrected", + "status": "passed", + "evidence": "internal/query/overview.go (now: pg_ls_archive_statusdir is superuser+pg_monitor, the predecessor was superuser-only), internal/stat/postgres.go (same, plus 'archive_mode=off is not an error'), internal/query/overview_test.go ('the fixtures role is postgres, a superuser'), internal/stat/postgres_test.go (same). All four read correctly in the diff; two new NotContains assertions keep pg_ls_dir from coming back." + } + ], + "standRun": { + "status": "completed", + "address": "pgpro@10.128.28.194", + "date": "2026-08-06", + "reachability": "Key auth worked on the first attempt and stayed stable for the whole run; no drop-outs. tmux was absent and installed with apt (passwordless sudo).", + "inventory": "Debian 12 (kernel 6.1), passwordless sudo. PostgresPro ent 18.4, unit postgrespro-ent-18.service, PGDATA /var/lib/pgpro/ent-18/data, socket /tmp. shared_preload_libraries = pgpro_scheduler, pg_query_state, pg_wait_sampling, pgpro_stats. pg_hba: 'local all all trust' first, so socket logins need no password. Disk 12 G / 6.9 G free, inodes 682882 free (200000 dummy files = 30% of headroom, so the full number was used, not scaled down). PPEM's own backend database 'ppem' (48 MB) lives IN this cluster, so every archive_mode restart interrupted ppem.service — done deliberately, three restarts total, all four services verified active after each.", + "geometry": "tmux 190x52 for the main pass, 60x40 for the narrow pass; send-keys, wait >= one refresh interval, capture-pane (-e where an attribute was the thing checked).", + "scenarios": [ + { + "name": "working archiving (/bin/true, 3x pg_switch_wal)", + "result": "PASS", + "detail": "archived 0->3, archived_age 00:00:04, last_archived=000000010000000000000023, ready back to 0, failed unchanged at 0" + }, + { + "name": "broken archiving (/bin/false, 3x pg_switch_wal)", + "result": "PASS", + "detail": "failed 6,6,9,9,12 over 25 s samples (grows; plateaus are PostgreSQL archiver backoff), ready=3 and never falling, last_failed=000000010000000000000024, failed_age 00:00:01 at first capture, archived unchanged at 3" + }, + { + "name": "archive_mode=off — blank cells and subtitle", + "result": "PASS", + "detail": "counters 0; the four text/age cells are all spaces (not '0', not '-', not 'n/a'), verified by slicing the row at header offsets; subtitle present once on entry" + }, + { + "name": "navigation: w cycle, W menu, help screen", + "result": "PASS", + "detail": "wal->archiver->wal, w from databases lands on wal, W menu has exactly two working items, help carries the w,W line and lists archiver in the Q caveat" + }, + { + "name": "subtitle exactly once per entry path, each path checked separately", + "result": "PASS", + "detail": "hotkey w path count=1; W menu path count=1; no duplicate on either call site" + }, + { + "name": "60-column terminal", + "result": "PASS", + "detail": "source stays frozen at every scroll position and carries bold ESC[1m that the rest of the header lacks (capture-pane -e); [ / ] reach every remaining column" + }, + { + "name": "privilege degradation on a terminal", + "result": "PASS", + "detail": "pg_monitor-only role renders the full row; a role without pg_monitor renders permission denied for function pg_ls_archive_statusdir in the table area and retries next tick without crashing; wal identical on master" + }, + { + "name": "Decision 9 cost measurement", + "result": "COMPLETED — numbers bad", + "detail": "see decision9" + } + ], + "blockedScenarios": [], + "cleanup": "Restored and verified against the recorded baseline. Order: stopped the recorder and all pgcenter processes, killed every tmux session, DELETED the 200000 dummy .ready files BEFORE touching archive_mode (200000 -> 0), removed the remaining real .ready files (back to the baseline 0), ALTER SYSTEM RESET archive_mode + archive_command, then one final restart. Verified: SHOW archive_mode = off (source=default, pending_restart=false), SHOW archive_command = (disabled) (source=default), wal_level = replica (source=default), 0 archive_ lines in postgresql.auto.conf, archive_status 0 files. The archive_status directory inode had grown to 11579392 bytes from holding 200k entries (ext4 never shrinks a directory), so it was rmdir'd and recreated at mode 700 while the cluster was stopped — back to 4096 bytes. Roles qa_monitor/qa_plain/qa_settings dropped (after revoking their CONNECT grants), table qa_junk1 dropped, both binaries and all scripts removed from /tmp, tmux server gone. systemctl is-active: postgrespro-ent-18, ppem, ppem-agent, pgpro-otel-collector = all active; the ppem database is reachable (49 MB). Sole residue: pg_stat_archiver's stats_reset timestamp is later than the original 2026-08-06 08:50:25 because scenario 3 required a counter reset; the counters themselves match the baseline 0/0." + }, + "decision9": { + "status": "applied", + "outcome": "THROTTLE — numbers are on the bad side of the pre-agreed threshold", + "conditions": "Taken exactly under the conditions Decision 9 fixes: role qa_monitor holding ONLY pg_monitor (usesuper=f, NOT a superuser), verbose mode on, a concurrent 'pgcenter record -i 1s -c 600' as the same role, archive_mode=off so the archiver could not drain the directory mid-run, and 200005 .ready files (200000 empty dummies named 00000002<16 hex>.ready, created in 5 s; no real WAL segments).", + "measurements": { + "queryWallTime": "SELECT count(*) FILTER (WHERE name LIKE '%.ready') FROM pg_ls_archive_statusdir(): 895.7, 1189.8, 1260.6, 1054.4, 1140.6 ms => mean ~1108 ms (range 0.90-1.26 s) on 200005 files, against ~0.9-1.2 ms on the empty directory. The verbose panel's backlog variant: 840.5, 1378.2, 886.3 ms => mean ~1035 ms. The default refresh interval is 1 s, so the query time is not merely comparable to it — it exceeds it on most samples.", + "viewSwitchLatency": "archiver open, press 'w', time to the wal header: 70, 240, 230 ms. Not visibly sticky — this is the one number that came out acceptable.", + "verbosePanelABCost": "A/B on an unrelated screen (activity) as qa_monitor, measured as the effective refresh rate (distinct header timestamps per wall second): feature verbose=off 17 ticks/15 s (~1.0 s); feature verbose=ON 8 ticks/15 s (~1.9 s); master verbose=ON 17 ticks/15 s (~1.0 s). A 20 s repeat: feature ON 10 ticks/21 s (~2.1 s) vs master ON 22 ticks/21 s (~1.0 s). The verbose panel field itself reads '3.1T archiving backlog' on the feature binary and 'n/a archiving backlog' on master, because master's pg_ls_dir() is superuser-only and fails for pg_monitor in 4-15 ms. Keypress latency was 8-17 ms on both, so the cost is collector-side and surfaces as a halved refresh rate, not as input lag." + }, + "reason": "Decision 9's trigger is 'query time comparable to the refresh interval (1 s by default) OR the view switch becomes visibly sticky'. The view switch is fine (70-240 ms), but the query time is ~1.1 s against a 1 s interval, and the A/B shows the consequence directly: for a pg_monitor role with verbose on, the feature halves the effective refresh rate on EVERY screen (1.0 s -> ~1.9-2.1 s) while master is unaffected. This is the zero->full change Decision 9 anticipated, and it is measured, not inferred.", + "carryForward": "Per Decision 9's pre-agreed outcome, throttling returns as ITS OWN DECISION, with these numbers attached, built on the machinery that already exists for exactly this: verboseCollectState + latencyGuardThreshold (ADR [010], today used only for the DB-size aggregate). Two call sites need it: (1) the verbose panel's OverviewArchivingBacklog walk, which is the one that halves the refresh rate because it runs on every screen and every tick, and (2) the archiver screen's own .ready sub-select. This task DECIDES and does not IMPLEMENT — no task of feature 017 owes a code fix for it. Because the outcome is 'throttle' and not 'acceptable', the remainder does NOT simply go to the tech-debt register." + }, + "findings": [ + { + "severity": "major", + "title": "The verbose panel's new pg_ls_archive_statusdir() walk halves the effective refresh rate for a pg_monitor role on a large archive_status", + "expected": "Enabling verbose mode should not change how often the screen refreshes.", + "actual": "With 200005 .ready files, a pg_monitor-only role and verbose on, the feature binary refreshes every ~1.9-2.1 s instead of the configured 1 s (8 distinct header ticks per 15 s vs 17 for the same binary with verbose off, and 17 for the master binary with verbose ON). The panel query itself measures ~1035 ms; master's superuser-only pg_ls_dir() equivalent fails for pg_monitor in 4-15 ms and therefore costs nothing.", + "reproduction": "Stand run 2026-08-06, scenario 7; see decision9.measurements and logs/working/qa-017-stand/manual-run-2026-08-06.txt.", + "abVerdict": "FEATURE-INTRODUCED. The master binary under identical conditions holds ~1.0 s. Root cause is the deliberate Decision 8 swap of pg_ls_dir() for pg_ls_archive_statusdir(), which is what makes the field work for pg_monitor at all.", + "owner": "new decision (Decision 9's pre-agreed 'throttle' branch) — NOT a defect of tasks 1-9; the behaviour is exactly what the specs asked for, and its price is what this measurement was commissioned to find", + "impact": "Only bites when all of: role is pg_monitor-and-not-superuser, verbose mode on, and archive_status is very large. That combination is precisely the incident an operator opens pgcenter to look at, which is Decision 9's own argument for measuring it." + }, + { + "severity": "minor", + "title": "last_archived / last_failed render truncated to the header width when the archiver screen is first opened on a cluster that has never archived", + "expected": "A 24-character WAL segment name is readable on the archiver screen.", + "actual": "Entering archiver while the columns are still NULL freezes their widths at the header lengths (13 and 11), and pgcenter's top never lowers view.Aligned again, so once values appear they print as '000000010000~' and '0000000100~' for the rest of the session. Because every segment on a timeline shares a long common prefix, the truncation removes exactly the discriminating suffix — all segments render as the same string. Entering the screen when values already exist gives the full 24-character names (verified: '000000010000000000000023' matching SQL).", + "reproduction": "Stand run 2026-08-06, scenarios 1-2 vs the fresh-launch capture; logs/working/qa-017-stand/manual-run-2026-08-06.txt.", + "abVerdict": "PRE-EXISTING, not feature-introduced. The mechanism is align.SetAlign + top/stat.go printDataCell + alignViewToResult, and 'git diff develop...HEAD' shows feature 017 changes none of them (its only touch in that area is +19 lines registering the view in internal/view/view.go). In top, Aligned is only ever set true, never reset. The archiver screen is a new place where an old mechanism shows.", + "owner": "tech-debt candidate (recorded here only; docs/tech-debt.md is /done's register), not a task of this feature", + "impact": "Cosmetic but real for the archiver screen specifically, since WAL names are distinguished only by their tail." + }, + { + "severity": "minor", + "title": "go.mod still marks github.com/spf13/pflag as // indirect although a test now imports it directly", + "owner": "task 4 (report CLI -W string flag)", + "expected": "go.mod lists a directly imported module in the direct require block.", + "actual": "Unchanged from the automated half. The working tree carries an uncommitted go.mod edit that moves pflag to the direct block; this QA task left it alone rather than committing code.", + "reproduction": "go mod tidy", + "abVerdict": "n/a — build metadata, not behaviour", + "impact": "Cosmetic." + }, + { + "severity": "minor", + "title": "Two files are gofmt-dirty — pre-existing, not this feature doing", + "owner": "pre-existing repo hygiene, no task of this feature", + "expected": "gofmt clean", + "actual": "Unchanged from the automated half.", + "reproduction": "gofmt -l .", + "abVerdict": "PRE-EXISTING", + "impact": "Cosmetic." + }, + { + "severity": "info", + "title": "A role with no privileges at all cannot start pgcenter on either binary", + "expected": "n/a — recorded so the US-12 method is auditable.", + "actual": "A role holding nothing (qa_plain) makes pgcenter abort before the TUI with 'permission denied to examine \"shared_preload_libraries\" (SQLSTATE 42501)', identically on the feature and master binaries. The archiver screen is therefore unreachable for such a role, and US-12's terminal half was verified with a role holding pg_read_all_settings but NOT pg_monitor — which still fails pg_ls_archive_statusdir() and is the meaningful case.", + "reproduction": "Stand run 2026-08-06, scenario 6.", + "abVerdict": "PRE-EXISTING (identical on master).", + "owner": "none — not filed as a defect", + "impact": "None on this feature; documents how US-12 was actually exercised." + } + ], + "deferredToPostDeploy": [] +} \ No newline at end of file diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-quality-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-quality-review.json similarity index 100% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-quality-review.json rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-quality-review.json diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-security-audit.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-security-audit.json similarity index 100% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-security-audit.json rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-security-audit.json diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-skeptic-techspec.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-skeptic-techspec.json similarity index 100% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-skeptic-techspec.json rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-skeptic-techspec.json diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-code-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-code-reviewer-review-round2.json new file mode 100644 index 00000000..740e8950 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-code-reviewer-review-round2.json @@ -0,0 +1,51 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "All three round-1 minors are addressed and nothing regressed: archiver.go is byte-identical to the reviewed round-1 version, the two test-side files changed only in the directions requested. The new code is sound on the points that carry weight — I empirically confirmed the SetupTestRole regex is genuinely anchored in Go (regexp.Perl sets OneLine, so 'role\\n; DROP ...' is rejected, unlike the PCRE/Python trailing-newline bypass) and that connectArchiverFixture's assert-then-Skipf really reports FAIL rather than a silent SKIP when a port mapping is missing. Zero critical, zero major; four optional minors below, the only substantive one being that the new regex guard itself has no test pinning it.", + "criticalIssues": [], + "suggestions": [ + { + "file": "internal/postgres/testing.go", + "line": 69, + "severity": "minor", + "category": "testing", + "suggestion": "The new guard is correct but entirely unpinned: nothing in the tree exercises it. internal/postgres/testing_test.go covers only NewTestConnectVersion, and the two callers in archiver_test.go pass valid literals, so widening testRoleNameRE to `.*` or deleting the four-line check breaks no test. That is the same class of gap the round-1 finding was about, just moved one level up — a later refactor can drop the invariant silently. The fix is a server-free table test in internal/postgres/testing_test.go and it can pass a nil *DB, because the MatchString check runs before the first db.Exec: `err := SetupTestRole(nil, \"a; DROP ROLE x\", false)` returns 'invalid test role name' with no dereference. Worth rows for the accepted literal, a semicolon, a space, an uppercase name, an empty string, a leading digit, and — the interesting one — a trailing newline, which is the case that documents why the Go anchoring is safe where PCRE's would not be.", + "benefit": "The 'literal constants only' invariant stops being load-bearing-but-unverified; the trailing-newline row also records the non-obvious Go-vs-PCRE anchoring fact for whoever touches the regex next.", + "optional": true + }, + { + "file": "internal/query/archiver_test.go", + "line": 291, + "severity": "minor", + "category": "maintainability", + "suggestion": "The membership sub-select aggregates pg_auth_members rows without DISTINCT: `array_agg(r.rolname::text ORDER BY r.rolname)`. Since PG 16 the same role can be granted to the same member more than once when the grantors differ, producing two catalog rows and therefore [\"pg_monitor\",\"pg_monitor\"], which would fail assert.Equal with a confusing message rather than the privilege regression the assertion is meant to catch. It cannot happen in the fixture environment today (SetupTestRole always grants as the fixture superuser, and a repeat grant by the same grantor does not duplicate the row), so this is hardening, not a defect. `array_agg(DISTINCT r.rolname::text ORDER BY r.rolname)` is a one-word change and stays valid SQL on 14-19.", + "benefit": "The membership guard keeps failing only for the reason it exists — an unexpected role membership — instead of for a duplicate-grant artifact on PG 16+.", + "optional": true + }, + { + "file": "internal/query/archiver_test.go", + "line": 83, + "severity": "minor", + "category": "readability", + "suggestion": "The order loop searches for `\" AS \"+col`, and two locked aliases are prefixes of later ones: 'archived' of 'archived_age', 'failed' of 'failed_age'. Deleting `archived_count AS archived, ` therefore does not trip the NotEqual(-1) assertion — strings.Index finds the archived_age occurrence instead. I reproduced this on a mutated copy of the query: the test still goes RED, but on `\"last_archived\" must follow the previous locked column / \"140\" is not greater than \"214\"`, which points at the wrong column. So the guard is sound, the diagnostics are not. Searching for `\" AS \"+col+\",\"` (with the last alias matched by the existing HasSuffix assertion, or by appending a space) removes the ambiguity.", + "benefit": "A dropped column is reported as a dropped column, so the structure test stays self-explanatory when it fires years from now.", + "optional": true + }, + { + "file": "internal/query/archiver_test.go", + "line": 130, + "severity": "minor", + "category": "maintainability", + "suggestion": "Test_StatArchiverQuery_NullsStayNull is 58 physical lines (130-187), which crosses the automatic '>50 lines' mapping. Reporting it as major would be noise: 13 of those lines are the wantValid fixture map and explanatory comments, the executable body is ~40 lines with a single version loop and no branching, and it grew only because the round-1 test review asked for value assertions here. Graded minor deliberately, and noted rather than fixed — splitting the NULL-ness loop from the value assertions would mean two live round-trips per version for no added coverage. If it grows again in a later task, lifting wantValid to a package-level var alongside archiverColumns is the cheap way back under the line.", + "benefit": "Records the threshold crossing and the reasoning, so a later reviewer does not re-litigate it or split the test at a cost in fixture round-trips.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 3, + "criticalIssuesCount": 0, + "majorIssuesCount": 0, + "minorIssuesCount": 4, + "testCoverageAssessment": "excellent" + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-code-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-code-reviewer-review.json new file mode 100644 index 00000000..1a4509e5 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-code-reviewer-review.json @@ -0,0 +1,42 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "The three files implement task 01 exactly as specified: a single version-independent 9-column pg_stat_archiver query in the locked alias order, a selector matching the io.go:99 form, and one shared append-only role helper in internal/postgres/testing.go with no testing import. Cross-file consistency checks out (postgres.DB.Exec/Query/QueryRow, query.Format/NewOptions, pgx v5 FieldDescription.Name, pgconn.PgError), the doc comments make the four required claims and each is verifiable against the code they cite (calculateDelta short-circuits on [2]int{0,0} at internal/stat/postgres.go:590-597). No critical or major findings; three optional improvements below.", + "criticalIssues": [], + "suggestions": [ + { + "file": "internal/postgres/testing.go", + "line": 59, + "severity": "minor", + "category": "best-practices", + "suggestion": "SetupTestRole is an exported symbol in a file with no build tag, so it is part of the released binary's API surface and it builds three DDL statements (CREATE ROLE / GRANT pg_monitor / SET ROLE) by fmt.Sprintf interpolation. The safety contract today is a comment ('Callers must pass literal constants - never user input'). Decision 19 forces the file location and gosec is clean, and no production code calls it, so this is exposure rather than a defect. It can be made enforced rather than documented at ~2 lines of cost: reject a name that does not match a plain identifier pattern before the first Exec, or quote it with pgx.Identifier{name}.Sanitize() for the two identifier positions (the pg_roles lookup stays a quoted literal). Task 03 consumes this signature, so any change must be agreed with it — the error return type is unaffected either way.", + "benefit": "The 'literal constants only' invariant becomes machine-checked instead of comment-checked, and a future caller in another package (or a later production caller) cannot turn the helper into a DDL injection point.", + "optional": true + }, + { + "file": "internal/query/archiver_test.go", + "line": 38, + "severity": "minor", + "category": "maintainability", + "suggestion": "The Test_SelectStatArchiverQuery table stops at 190000, but its comment claims 'a future branch cannot be added silently'. That holds only for PG 14-19: a later `if version >= PostgresV20 { return PgStatArchiverPG20, ... }` would leave every existing case green. wal_test.go:26-27 already carries the precedent for closing this — a `{version: 200000, ...}` row commented as 'Forward version: the branch must fire for every future major'. Adding the same row here makes the comment's claim true for all inputs; alternatively soften the comment to name the 14-19 range.", + "benefit": "The selector's version-independence stays pinned when PG 20 support is added, and the test comment matches what the test actually guards.", + "optional": true + }, + { + "file": "internal/query/archiver_test.go", + "line": 64, + "severity": "minor", + "category": "readability", + "suggestion": "Four of the five tests repeat the same eight-line preamble: SelectStatArchiverQuery -> NewOptions(version, \"f\", \"off\", 256, \"public\") -> Format -> assert.NoError -> NewTestConnectVersion -> t.Skipf -> defer conn.Close(). A single file-local helper returning (query, *postgres.DB) and skipping internally would remove ~24 duplicated lines and keep the skip message in one place. Note this duplication matches the existing style in wal_test.go / replication_slots_test.go / io_test.go, so leaving it as-is is a defensible consistency choice — take this only if the file grows further in later tasks.", + "benefit": "One place to change when the Options arguments or the skip convention move; the intent of each test (privilege, NULL-ness, shape) stands out from the boilerplate.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 3, + "criticalIssuesCount": 0, + "majorIssuesCount": 0, + "minorIssuesCount": 3, + "testCoverageAssessment": "excellent" + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-security-auditor-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-security-auditor-review-round2.json new file mode 100644 index 00000000..6b5a44b6 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-security-auditor-review-round2.json @@ -0,0 +1,41 @@ +{ + "status": "approved", + "summary": { + "totalFindings": 3, + "critical": 0, + "major": 0, + "minor": 3 + }, + "findings": [ + { + "severity": "minor", + "category": "A03: Injection", + "title": "Round-1 finding 1 is CLOSED - the regex admits nothing that can escape any of the four interpolation positions; the residual is semantic, not syntactic", + "description": "VERDICT: the fix closes the hole as described. I did not take the regex on inspection - I compiled `^[a-z_][a-z0-9_]*$` with regexp.MustCompile and ran the payload set against it, because the one bypass that would have mattered is invisible to reading.\n\nThat bypass is the trailing-newline gap. In PCRE, Python and Ruby, `$` matches before a final newline, so `^[a-z_][a-z0-9_]*$` accepts \"pgcenter_test\\n; DROP ROLE victim\" - and under pgx.QueryExecModeSimpleProtocol that is a second statement, executed. Go does NOT inherit that behaviour: regexp.Compile applies syntax.Perl, which includes the OneLine flag, so `$` anchors to end of text only. Confirmed empirically - \"ok\\n; DROP ROLE victim\" and a bare \"ok\\n\" are both rejected. Also rejected: \"ok; DROP ROLE victim\", \"ok'; DROP ROLE victim; --\", \"a$$; CREATE ROLE evil SUPERUSER; $$\", \"ok\\x00; DROP ROLE v\", backslash, tab, space, double quote, non-ASCII, leading digit, and the empty string. Accepted: the two existing constants and nothing else outside [a-z_][a-z0-9_]*.\n\nAgainst the four positions, checked one at a time as requested:\n\n1. `rolname = '%s'` inside the `$$ ... $$` DO block - needs `'` to escape the literal or `$` to terminate the dollar quoting. The accepted alphabet contains neither, and both are independently rejected above. Closed on two counts.\n2-4. `CREATE ROLE %s`, `GRANT pg_monitor TO %s`, `SET ROLE %s` - unquoted identifier positions, the sharp ones. The accepted alphabet is a strict subset of PostgreSQL's unquoted-identifier alphabet (it is that set minus `$`, minus uppercase, minus non-ASCII), and the first character is constrained to a letter or underscore. Every accepted string therefore lexes as exactly one identifier token: it cannot terminate the statement (no `;`), cannot open a comment (`-` and `/` rejected), cannot start a new token (no whitespace or control characters), and cannot be split by the lexer at all. Simple protocol's multi-statement capability is unreachable because no accepted name can produce a statement separator.\n\nValidation is also correctly placed - lines 69-71, before the first db.Exec, so no partial work precedes a rejection - and the error is returned rather than panicked, which the *testing.T-free signature Task 03 depends on requires.\n\nTHE RESIDUAL, and the only reason this is a finding rather than silence: the regex bounds SYNTAX, not SEMANTICS. Four accepted names are PostgreSQL keywords - `none`, `default`, `select`, `public` all match. Three of those fail loudly (a reserved word in the RoleId position is a syntax error, and `public` collides with an existing role), which is fine. `none` is the one that fails quietly: `SET ROLE NONE` is documented to reset the session to the session user rather than assume a role, and `SET ROLE DEFAULT` does the same. A caller passing either would get no error, no role restriction, and a session still running as the fixture superuser.\n\nThis is not injection - nothing extra executes - and it is not reachable today, since both call sites pass file-local constants (archiver_test.go:205, :243) and the tree contains no other caller. It is also already caught downstream: assertRestrictedSession asserts current_user equals the requested name and that rolsuper is false, so a `none` would redden on the guard rather than silently produce a superuser test. I record it only so that the regex is not later read as a stronger contract than it is - it guarantees \"this string cannot alter the statement's structure\", not \"this string names a role\".\n\nOne non-security note in the same area, recorded because it belongs to the same interpolation: names longer than 63 bytes are truncated by the server in CREATE ROLE while the DO block's `rolname = '%s'` comparison uses the untruncated string, so idempotency would break on re-run. Both constants are ~30 characters; no action implied.", + "location": "internal/postgres/testing.go:12 (testRoleNameRE), :69-71 (guard), :73-91 (four interpolation positions)", + "impact": "None. The injection surface described in round 1 is closed: no string the regex accepts can escape the dollar-quoted literal or split any of the three unquoted identifier positions, on any input including the newline-anchor bypass that defeats this same pattern in PCRE-family engines. The residual is a caller-error class (a keyword name that is accepted syntactically but does not mean what the caller intends), bounded to test code, unreachable from any pgcenter input path, and already trapped by the session guard in the tests.", + "recommendation": "No change required - the round-1 recommendation was implemented essentially verbatim and it works. Two things worth carrying into the decisions log rather than into code:\n\n1. The guard's strength depends on Go's OneLine `$` semantics. If this pattern is ever ported to another language or reused with (?m), the trailing-newline bypass reopens and yields multi-statement execution under simple protocol. Anchoring with \\A and \\z instead of ^ and $ would make that independent of the engine, and is the only hardening I would consider - it is optional here and changes no behaviour in Go.\n2. The regex is a structural guarantee, not a naming one. Keyword names (`none`, `default`) pass it; `none` in particular silently no-ops the SET ROLE. If a future caller ever derives a role name rather than passing a constant, reject the reserved set explicitly at that call site.", + "cwe": "CWE-89" + }, + { + "severity": "minor", + "category": "best-practice", + "title": "Decision 16 carried forward unchanged - a server-trust argument rather than an output-encoding control, accepted as written", + "description": "CARRIED FORWARD from round 1 finding 2, accepted with no code change, recorded here so this report stands alone. I re-read internal/query/archiver.go:32-37 and the query text; the doc comment and the two columns are byte-identical to what I reviewed in round 1, and the file is unmodified in the working tree.\n\nThe verified substance, unchanged: PostgreSQL's archiver admits a segment only through pgarch_readyXlog(), which rejects any candidate failing `strspn(d_name, VALID_XFN_CHARS) < basenamelen` with VALID_XFN_CHARS covering hex digits plus the .history/.backup/.partial suffix letters, under a 16..40 length bound. That surviving basename is exactly what pgstat_report_archiver() stores in last_archived_wal / last_failed_wal, on the success and failure branches alike. The character set contains no ESC (0x1B) and no control characters, so neither column can carry a terminal escape sequence - which is why the doc comment's stronger claim about a hand-placed bogus .ready file also holds.\n\nThe design still does not leak the unfiltered side channel it could have: the `ready` column takes count(*) over pg_ls_archive_statusdir() and never renders a `name` value. pg_ls_archive_statusdir() returns raw directory entries with no VALID_XFN_CHARS filtering at all, so selecting a filename from it would have widened [029] with a genuinely attacker-influenceable string. The argument depends on taking only the count, and the count is what is taken.\n\nThe boundary, restated because it is the part worth recording: this reasons about what a correctly-behaving PostgreSQL will send, not about what pgcenter will render. It does not cover a hostile or compromised endpoint, nor an attacker who already owns the postgres OS user and can write the pgstat file directly. Both are the general class tech-debt [029] tracks, and neither is introduced or widened here.", + "location": "internal/query/archiver.go:32-37 (doc comment), :44 (last_archived), :47 (last_failed)", + "impact": "None for this task. Under the trusted-server model the two columns are provably incapable of carrying a terminal escape sequence. The residual - a hostile endpoint emitting arbitrary bytes into any text column - is unchanged in scope and already tracked as [029].", + "recommendation": "Accept as written; no code change, as agreed after round 1. Record in the decisions log that the licence is per-column and per-source: it covers these two columns because the archiver path filters them upstream, and must not generalise into \"server text needs no sanitisation\". If [029] is ever resolved centrally in printDataCell, these columns need no exemption - a central control subsumes this decision rather than conflicting with it.", + "cwe": "CWE-150" + }, + { + "severity": "minor", + "category": "A01: Broken Access Control", + "title": "Persistent pg_monitor test role carried forward as designed - and the new membership assertion closes the decay risk that made it worth flagging", + "description": "CARRIED FORWARD from round 1 finding 3, accepted with no code change. The helper still creates roles idempotently and nothing drops them, so pgcenter_test_archiver_monitor persists on every cluster the fixture port map reaches, holding pg_monitor indefinitely. That is deliberate - the re-runnability criterion depends on creation meeting an existing role, and the task file forbids teardown - and remains the right trade-off for ephemeral CI containers.\n\nThe residual is unchanged and still small: the role is NOLOGIN so it cannot authenticate; it has no members so nothing inherits from it; assuming it via SET ROLE needs superuser or an explicit grant, at which point pg_monitor is redundant. The realistic exposure stays narrow - a developer with something other than a throwaway fixture on 127.0.0.1:21914-21919 gets a silent permanent pg_monitor grant from a test run.\n\nWHAT ROUND 2 IMPROVES, and it improves exactly the part I flagged: round 1 noted that the roles are cluster-global and that SetupTestRole never normalises a role that already exists, so a stray GRANT on a long-lived cluster could silently decay the positive test from \"pg_monitor is sufficient\" to \"some privileged role works\". The new assertRestrictedSession (archiver_test.go:278-312) now asserts direct membership exactly - array_agg over pg_auth_members joined to pg_roles, compared against []string{\"pg_monitor\"} for the grant role and asserted empty for the deny role - on top of current_user, rolsuper and pg_has_role. A role that accumulated an extra membership between runs now reddens instead of passing. That is the correct guard for the failure mode, and it is a net security improvement to the persistent-role design rather than a new risk.\n\nI reviewed the rest of the round-2 test changes for new issues and found none. The new server-free Test_StatArchiverQuery_Structure does pure string assertions with no connection. Every statement in assertRestrictedSession, resetRole and runArchiverQuery is a static literal - no fmt.Sprintf, no interpolation, so the new helpers add no injection surface (`grep -rE '(Exec|Query|QueryRow)\\(fmt\\.Sprintf'` over the tree returns only the two known lines in testing.go). The catalogs read are world-readable and pg_roles masks rolpassword, so scanning them under a restricted role exposes no credential material. No secrets, tokens or passwords appear in the fixture setup. The RESET ROLE defers are registered immediately after the successful SET ROLE, so a failing assertion cannot leak a restricted session onto a reused connection.\n\nOne residual on the membership assertion, for completeness rather than action: pg_auth_members lists direct grants only, so it would not catch a direct object-level GRANT EXECUTE on pg_ls_archive_statusdir to the test role. hasMonitor is asserted true independently, so the positive test cannot pass without pg_monitor actually being held; the gap is only that it could pass with pg_monitor plus a redundant direct grant. Not worth code.", + "location": "internal/postgres/testing.go:68-94 (no teardown by design); internal/query/archiver_test.go:30-33 (role constants), :278-312 (new membership guard)", + "impact": "A NOLOGIN role holding pg_monitor persists on any cluster the fixture port map reaches. It cannot be logged into and cannot be assumed without superuser, so it grants no practical access on its own; the concern is an unexpected privileged object left behind on a non-throwaway cluster. The privilege-decay consequence flagged in round 1 is now detected by the test suite rather than silent.", + "recommendation": "No change required - accepted as designed. If the fixture port map is ever pointed at anything longer-lived than the CI container, drop the roles explicitly (`DROP ROLE IF EXISTS pgcenter_test_archiver_monitor, pgcenter_test_archiver_norole`) rather than adding per-test teardown, which would defeat the idempotency the re-runnability criterion checks. Keep the note already present in the task's Acceptance Criteria that the two grant-related mutation gates stay vacuous until the roles are dropped on each cluster.", + "cwe": "CWE-269" + } + ] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-security-auditor-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-security-auditor-review.json new file mode 100644 index 00000000..51e7a201 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-security-auditor-review.json @@ -0,0 +1,41 @@ +{ + "status": "approved", + "summary": { + "totalFindings": 3, + "critical": 0, + "major": 0, + "minor": 3 + }, + "findings": [ + { + "severity": "minor", + "category": "A03: Injection", + "title": "SetupTestRole interpolates an unvalidated SQL identifier into three statements, guarded only by a doc comment, in a file that ships in the release binary", + "description": "SetupTestRole builds all three of its statements with fmt.Sprintf and no validation of `name`: a single-quoted literal position inside the DO block (`rolname = '%s'`), and three bare identifier positions (`CREATE ROLE %s`, `GRANT pg_monitor TO %s`, `SET ROLE %s`). The identifier positions are the sharp ones - they carry no quoting at all, and NewConfig sets pgx.QueryExecModeSimpleProtocol (internal/postgres/postgres.go:53), so a semicolon in `name` yields direct multi-statement execution. The DO-block literal is wrapped in dollar quoting, so a quote there escapes into PL/pgSQL rather than out of the statement - still arbitrary code, just one level in.\n\nI judge the residual surface ACCEPTABLE as shipped, and the reasoning is worth recording rather than just the verdict:\n\n1. Reachability today is zero. The two call sites (internal/query/archiver_test.go:166 and :207) pass package-level string constants (archiverRoleMonitor, archiverRoleNoRole). There is no path from any pgcenter input - flag, config, connection string, or server row - into this function.\n2. The blast radius is bounded by Go's internal-package rule. `github.com/lesovsky/pgcenter/internal/postgres` cannot be imported by any module other than pgcenter itself, so \"ships in the released binary\" means the code is present as dead weight, not that it is exposed as callable API to third parties. That is the decisive mitigation and it is structural, not a convention.\n3. The choice itself is forced, not lazy. A role name is an identifier; it cannot be a $1 placeholder. Every alternative (pgx.Identifier.Sanitize, a regex allowlist) is a validation strategy, not a parameterisation strategy - there is no \"do it properly with placeholders\" option being passed up here.\n\nWhat makes it worth a finding rather than silence is that the guard is a comment. Two facts sharpen that: this is the FIRST fmt.Sprintf-constructed SQL statement in the entire tree (`git grep -E 'Exec\\(fmt\\.Sprintf|Query\\(fmt\\.Sprintf' HEAD` returns nothing), so it establishes a pattern in a repo that previously had none; and gosec did not clear it - the clean `make lint` run is not evidence here. gosec's G201/G202 are keyed to database/sql call shapes and do not recognise the project's pgx wrapper `(*postgres.DB).Exec`, so no linter in the pipeline will fire if a future contributor copies this shape into a path that does take user input. Task 03 is documented to reuse this helper, which is one more caller multiplying the pattern.", + "location": "internal/postgres/testing.go:59-81", + "impact": "No exploitable impact in the shipped binary: the function is unreachable from any input path and the internal/ package rule prevents external callers. The finding is latent-hazard only. If a future change ever routes a non-constant value into `name`, the impact flips to arbitrary SQL execution under the connected role - which for pgcenter is routinely a superuser or pg_monitor DBA session against a production cluster - because simple protocol mode permits multi-statement strings at the unquoted identifier positions.", + "recommendation": "Make the doc comment's contract mechanically enforceable instead of advisory. Four lines at the top of the function converts \"callers must pass literal constants\" from a promise into an invariant, costs nothing at runtime, and keeps the pattern safe when Task 03 and any later caller copy it:\n\n```go\nvar testRoleNameRE = regexp.MustCompile(`^[a-z_][a-z0-9_]*$`)\n\nfunc SetupTestRole(db *DB, name string, pgMonitor bool) error {\n\tif !testRoleNameRE.MatchString(name) {\n\t\treturn fmt.Errorf(\"invalid test role name %q\", name)\n\t}\n\t...\n```\n\nThe lowercase-only pattern is not an arbitrary restriction: the identifier positions are unquoted, so PostgreSQL down-folds the name anyway, and both existing constants are already lowercase. `pgx.Identifier{name}.Sanitize()` is the alternative if quoted mixed-case names are ever wanted, but the regex is the better fit here because it also rejects rather than silently rewrites.\n\nNo change is required to merge this task. Recorded so the decision is a decision.", + "cwe": "CWE-89" + }, + { + "severity": "minor", + "category": "best-practice", + "title": "Decision 16 is sound, but it is a server-trust argument rather than an output-encoding control - boundary recorded", + "description": "I verified the factual claim behind Decision 16 and it holds. PostgreSQL's archiver picks up a segment only in pgarch_readyXlog(), which rejects any candidate failing `strspn(d_name, VALID_XFN_CHARS) < basenamelen` with `VALID_XFN_CHARS \"0123456789ABCDEF.history.backup.partial\"` (src/include/postmaster/pgarch.h), plus a 16..40 length bound. The basename that survives that filter is exactly what is handed to pgstat_report_archiver() and thus what lands in last_archived_wal / last_failed_wal. The character set is hex digits, '.', and the letters appearing in the three suffix words - no ESC (0x1B), no CSI, no control characters. Both columns are written from that same path, on the success and the failure branch alike. So the doc comment's stronger claim - that the columns stay clean \"even if an operator hand-places a bogus .ready file\" - is correct: a file named with an ESC in it is skipped by the filter and never reaches the statistics.\n\nThe implementation also does not leak the unfiltered side channel it easily could have: the `ready` column takes `count(*)` over pg_ls_archive_statusdir() and never renders the `name` values. pg_ls_archive_statusdir() returns raw directory entries with no VALID_XFN_CHARS filtering whatsoever, so selecting a filename from it would have widened [029] with a genuinely attacker-influenceable string. Taking only the count is the right call and the argument depends on it.\n\nOne further point in the design's favour: the argument does not lean on the self-healing repaint reasoning in [029], which 016-feat-pause-display explicitly weakened (a hostile value now survives on screen for the whole freeze). Decision 16 rests on the value being incapable of carrying ESC in the first place, which is the stronger and pause-independent form of the argument.\n\nThe boundary worth stating: this is an argument about what a correctly-behaving PostgreSQL will send, not a control on what pgcenter will render. It does not cover a hostile or compromised endpoint - pgcenter connects wherever the operator points it, and a server that is not genuine PostgreSQL can return arbitrary bytes in any column. Nor does it cover an attacker who already owns the postgres OS user and can hand-write the pgstat file. Both are precisely the general class tech-debt [029] tracks, and neither is introduced or widened by this task: the archiver screen adds two more text columns to a stream of server-supplied text columns that is already unsanitised everywhere.", + "location": "internal/query/archiver.go:32-37 (doc comment), :44,:47 (last_archived / last_failed columns)", + "impact": "None for this task. Under the trusted-server model the two columns are provably incapable of carrying a terminal escape sequence. The residual - a hostile or compromised PostgreSQL endpoint emitting arbitrary bytes into any text column - is unchanged in scope by this change and is already tracked as tech-debt [029].", + "recommendation": "Accept as written; no code change. Decision 16 is correctly reasoned and correctly scoped, and the task's statement that [029] is \"neither widened nor closed\" is accurate. The only thing worth carrying forward is that the argument is per-column and per-source: it licenses these two columns because the archiver path filters them, and it must not be generalised into \"server text needs no sanitisation\". Should [029] ever be resolved centrally in printDataCell, these columns need no exemption - a central control subsumes this decision rather than conflicting with it.", + "cwe": "CWE-150" + }, + { + "severity": "minor", + "category": "A01: Broken Access Control", + "title": "Privilege tests create a persistent pg_monitor-holding role with no teardown, by design", + "description": "SetupTestRole creates roles idempotently and nothing ever drops them, so pgcenter_test_archiver_monitor persists on every cluster it touches, holding pg_monitor indefinitely. This is deliberate - the re-runnability acceptance criterion depends on creation meeting an existing role, and the task file explicitly forbids adding teardown - and it is the correct trade-off for ephemeral CI containers.\n\nThe residual is small and I do not think it warrants a change. The role is NOLOGIN, so it cannot be used to authenticate; it has no members, so no existing role inherits its privileges; and assuming it via SET ROLE requires superuser or an explicit membership grant, at which point pg_monitor is already redundant. The realistic exposure is narrow: a developer whose environment has something other than a throwaway fixture listening on 127.0.0.1:21914-21919 gets a silent, permanent pg_monitor grant created on it by a test run.\n\nWorth noting alongside this: the task's own Acceptance Criteria already documents the operational consequence - the two grant-related mutation gates are vacuous until the role is dropped on each cluster, because the idempotent DO block sees the role exists and skips it. That caution is present and correctly worded in the task file, so the trap is handled procedurally.", + "location": "internal/postgres/testing.go:59-81; internal/query/archiver_test.go:31-34 (role name constants)", + "impact": "A NOLOGIN role holding pg_monitor persists on any cluster the fixture port map reaches. It cannot be logged into and cannot be assumed without superuser, so it grants no practical access on its own; the concern is an unexpected privileged object left on a non-throwaway cluster.", + "recommendation": "No change required - accepted as designed, recorded for the security file. If the fixture port map is ever pointed at anything longer-lived than the CI container, drop the roles explicitly (`DROP ROLE IF EXISTS pgcenter_test_archiver_monitor, pgcenter_test_archiver_norole`) rather than adding per-test teardown, which would defeat the idempotency the re-runnability criterion checks.", + "cwe": "CWE-269" + } + ] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-test-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-test-reviewer-review-round2.json new file mode 100644 index 00000000..71998fdd --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-test-reviewer-review-round2.json @@ -0,0 +1,62 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "passed", + "summary": "Both round-1 majors are genuinely closed, and closed with standing gates rather than one-off evidence. F1: Test_StatArchiverQuery_Structure pins 'count(*) FILTER (WHERE name LIKE '%.ready')' and 'FROM pg_ls_archive_statusdir()' by exact substring, and M9 shows the gate fires on '%.done'. I concede the pushback on the live VALUES check and withdraw that half of the recommendation: the substring pin reddens on every mutation the VALUES query would have caught (dropped FILTER, '%.done', '%ready%'), it reddens on a plain host run where the VALUES check would have skipped, and the VALUES query would in fact have asserted PostgreSQL's LIKE semantics rather than anything this repo owns. The offline pin is the better instrument, not the cheaper one. F2: assertRestrictedSession now asks the server for current_user, rolsuper, pg_has_role and the exact pg_auth_members set in one round trip, and M10 turns the cluster-pollution scenario from a hand-run experiment into a permanent gate that names its cause ('Should be empty, but was [pg_monitor]'). F3, F5, F6, F8, F9 are all applied as recommended; I verified the alias-position walk is sound including the prefix hazards it could have had (a deleted 'archived' or 'failed' column makes Index land on 'archived_age'/'failed_age' and the assert.Greater comparison against the equal-or-earlier previous index fires, so the walk does not false-green), and the stats_age regexp does redden on a dropped date_trunc because ::text of an untruncated interval carries fractional seconds. F4's first half is closed and I confirmed the mechanic empirically: a failed assert followed by t.Skipf reports FAIL, not SKIP, so a version missing from the port map now fails instead of skipping forever. On the two open judgement calls: the F7 scope call is right — acceptance criterion 1 names three files and testing_test.go is a fourth, the exists-branch is exercised by Verification Step 3's two-runs-in-one-container procedure which was performed, and automating it belongs in a follow-up, not in a deviation from the task's own file list. The M7 change is an improvement, not a loss of evidence: what M7 existed to prove is that the positive test's success depends on the grant, and that dependency is now asserted on every run by two independent gates (assertRestrictedSession's pg_has_role/membership check in the positive test, and the permanent 42501 assertion in Test_StatArchiverQuery_WithoutPgMonitorFails) instead of by one manual mutation in a container that no longer exists. Reddening earlier and on a line that names the cause is strictly better diagnostics; the only cost is that the acceptance-criterion text still predicts a 42501 that the mutation no longer reaches, which is a wording fix, not a coverage hole. What is new and worth fixing is small: the role-name regexp added to SetupTestRole on the security auditor's finding is production-compiled code with no test at all — deleting it leaves the whole suite green. Four minor items remain, none of them blocking; the two deferrals are correctly reasoned and belong in the decisions file as follow-ups.", + "findings": [ + { + "severity": "minor", + "category": "missing_coverage", + "location": "internal/postgres/testing.go:12 and :69-71 (testRoleNameRE guard) — no covering test anywhere", + "issue": "The role-name validation added in response to the security auditor is the only new production code in this round and it has zero coverage. Litmus: delete both the regexp var and the MatchString branch and every test in internal/query and internal/postgres stays green — the two callers pass valid literal constants, so the reject path never executes. This file carries no build tag and ships in the released binary, so the guard is production code by the task's own reasoning (acceptance criterion 2 treats a testing import here as a release-build regression for exactly that reason). A guard whose failure mode is invisible to the suite will not survive a future refactor of SetupTestRole intact, and its silent removal is precisely the class of change the security finding was raised to prevent.", + "recommendation": "One server-free test inside an already-in-scope file. The regexp check is the function's first statement and returns before any use of db, so the DB argument is never dereferenced on the reject path: in internal/query/archiver_test.go add Test_SetupTestRole_RejectsBadNames with a table over []string{\"bad-name\", \"1role\", \"role;DROP ROLE postgres\", \"Role\", \"\"} asserting require.Error and assert.Contains(err.Error(), \"invalid test role name\") for each, calling postgres.SetupTestRole(nil, name, false). Add one positive control — postgres.SetupTestRole(nil, \"pgcenter_test_ok\", false) must NOT return that error (it will return a create-role failure or panic-free nil-db error instead, so assert only that the message is not the validation one) — otherwise a regexp that rejects everything would pass the table. If the team would rather not pass a nil *DB, the same test belongs in internal/postgres/testing_test.go and should be batched with the F7 follow-up below; either location closes the litmus failure, the nil-DB form is simply the one that fits inside the task's three-file limit.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/query/archiver_test.go:333-344 (connectArchiverFixture) — F4 residue", + "issue": "The half that was applied is genuinely closed: I confirmed that a failed assert.NotContains followed by t.Skipf produces FAIL rather than SKIP, so an unmapped version now fails the run instead of skipping forever. The remaining half is untouched by design: a cluster that is down still yields a skip indistinguishable from a pass, so with one of six clusters unavailable the suite is green and non-verbose output shows nothing. This round's evidence for it is a human reading a -v run and counting zero skipped Archiver subtests — the same manual procedure that tech-debt [019] describes, restated rather than closed. I accept the reason given for not adding PGCENTER_REQUIRE_FIXTURES here (it would change a CI command shared by other tasks in this feature) and this must not block the task; but the gap is now the only thing standing between a partial fixture outage and a false green for the entire live half of this file.", + "recommendation": "Do not add the env var in this task. Record it instead as a feature-level follow-up in 017-feat-wal-archiver-decisions.md: connectArchiverFixture gains 'if os.Getenv(\"PGCENTER_REQUIRE_FIXTURES\") != \"\" { t.Fatalf(\"postgres %d required but unavailable: %v\", version, err) }' and the feature's CI command gains -e PGCENTER_REQUIRE_FIXTURES=1, both landed once at the end of the feature so every task's live tests are covered by the same switch. That is a one-line change per test file and it retires tech-debt [019] for this package rather than working around it per task.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "internal/postgres/testing.go:68-94 (SetupTestRole) / internal/postgres/testing_test.go — F7 residue", + "issue": "Carried forward unchanged and the deferral is correct — recording it so it is not lost, not to reopen it. Acceptance criterion 1 states the three named files are the only ones that may be modified and testing_test.go is a fourth, so declining was the right call; the deviation ledger for this task should not grow a second entry for a minor. The property is also not unverified: Verification Step 3's two-runs-in-one-container procedure exercises the DO block's exists branch on all six clusters and was performed green this round. What remains missing is automation — SetupTestRole is a published cross-task contract that Task 03 consumes in Wave 2, and its idempotency is guarded only by a procedure a human must remember to run.", + "recommendation": "Follow-up item for Task 03 or a feature-closing cleanup, batched with the regexp test above: TestSetupTestRole_isIdempotent in internal/postgres/testing_test.go following the local skip style at :28-32 — connect to 170000, call SetupTestRole(db, \"pgcenter_test_setup_idem\", true) twice, require.NoError on both, assert QueryRow(\"SELECT current_user\") returns the role name, then Exec(\"RESET ROLE\") and assert current_user is back to postgres. Add the pre-existing-role case (call once with pgMonitor true, once with false, assert pg_has_role is still true) so the 'reverting a mutation to role setup does not undo it on the server' caution becomes an asserted property rather than prose in an acceptance-criteria preamble.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-01.md:195-199 (the M7 acceptance criterion) vs internal/query/archiver_test.go:215", + "issue": "Verdict on the question asked: the change is an improvement, and the report should say so explicitly rather than leave a future reader to rediscover it. M7 was written to prove two things — that the helper's pgMonitor flag is wired, and that the positive test's success depends on the grant. The new membership guard proves the first directly and earlier, at the line that names the cause; the second is now asserted permanently on every run by Test_StatArchiverQuery_WithoutPgMonitorFails rather than once by hand. No coverage is lost, because no scenario exists in which the guard passes and the query would still have succeeded without pg_monitor — that is exactly what the negative test forbids. The residue is textual: the acceptance criterion still promises 'turns red with SQLSTATE 42501', which the mutation can no longer produce, so an engineer re-running Step 4 in six months will read a different red and be unable to tell an improvement from a regression.", + "recommendation": "Amend the M7 checkbox in the task file (or record the amendment in 017-feat-wal-archiver-decisions.md alongside the test-3 deviation) to read: after DROP ROLE on the target cluster, passing pgMonitor:false makes Test_StatArchiverQuery_PgMonitorRoleSucceeds turn red on assertRestrictedSession's pg_has_role / membership assertions, BEFORE the query runs; the 42501 dependency it used to demonstrate is asserted permanently by Test_StatArchiverQuery_WithoutPgMonitorFails and needs no mutation. State that reddening before the query is the intended behaviour, so a future run does not treat the missing 42501 as evidence the gate weakened.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/query/archiver_test.go:291-293 (the array_agg over pg_auth_members inside assertRestrictedSession)", + "issue": "Defensive, not observed — the assertion is correct today on all six clusters, as M10 demonstrates. The membership set is built with array_agg without DISTINCT. From PG 16 pg_auth_members holds one row per (roleid, member, grantor), so the same membership granted twice by different grantors produces two rows and the aggregate becomes [pg_monitor pg_monitor], failing assert.Equal against []string{\"pg_monitor\"} with a message that reads like the pollution scenario the guard was built to detect while actually meaning something harmless. It cannot fire from SetupTestRole itself (the fixture connection is always the bootstrap superuser, and a repeated GRANT from the same grantor adds no row), but the task's own Step 4 procedure instructs engineers to drop and re-grant these roles by hand, and a re-grant issued from a different superuser session is exactly how the second row appears.", + "recommendation": "One word: array_agg(DISTINCT r.rolname::text ORDER BY r.rolname::text) — with DISTINCT the ORDER BY expression must match the aggregated expression, hence the cast in both positions. The assertion keeps its full strength (an extra role still fails, since it is a distinct name) and stops being sensitive to how many grantors happened to issue the same grant.", + "litmusTestFailed": false + } + ], + "metrics": { + "filesReviewed": 5, + "litmusTest": { + "checked": 21, + "passed": 20, + "failed": 1 + }, + "coverageAssessment": "adequate", + "pyramidBalance": { + "unit": 2, + "integration": 3, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-test-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-test-reviewer-review.json new file mode 100644 index 00000000..a18374d6 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01-dev-test-reviewer-review.json @@ -0,0 +1,94 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "needs_improvement", + "summary": "Five tests, no vacuous or mock-only tests, and the mutation evidence (M1-M8) is genuine: every named mutation reddens the test it was supposed to redden, and M6 in particular proves the positive privilege test guards itself rather than riding the superuser fixture. The helpers are sound: runArchiverQuery correctly collects the error from both Query and the drain (the pgx simple-protocol path that would otherwise let the negative test miss the failure), assertRestrictedSession asks the server rather than the test's own state and returns a bool so the caller reddens on the guard, and the defer order (resetRole registered after conn.Close, so it runs first) is right. Two gaps are worth blocking on. First, the '.ready' FILTER predicate — the only real logic in the query and the feature's headline number — has no guard at all: the fixtures return zero directory rows, so count(*) FILTER (WHERE name LIKE '%.ready') and a bare count(*) are both 0, and no test asserts the value of ready anywhere; dropping the FILTER or changing it to '%.done' leaves all five tests green on all six clusters. Second, the privilege guard proves 'a named non-superuser' but never 'pg_monitor and nothing else', while the roles are cluster-global, never normalized by SetupTestRole on a pre-existing role, and explicitly mutated by the AC's own Step 4 procedure — so the positive test can decay from 'pg_monitor is sufficient' to 'some privileged role works' with nothing to signal it. The rest are minor: no value assertions at all (the 'Archiver' literal and the date_trunc truncation are both unfalsifiable), a skip that cannot distinguish an unmapped version from a downed cluster even though this same package already hardened against exactly that (testing_test.go:10-23), no offline guard on the locked column order, no unmapped/future version in the selector table despite the comment claiming one, SetupTestRole's exists-branch never executing in a CI run, and resetRole being the one assertion in the file with no falsifying mutation. On the deviation: the split of test 3 is the right call and should stand — see finding 9.", + "findings": [ + { + "severity": "major", + "category": "missing_coverage", + "location": "internal/query/archiver.go:39 (ready sub-select) — no covering assertion in internal/query/archiver_test.go", + "issue": "The '.ready' FILTER predicate is the only piece of logic in the query (every other column is a plain reference or a date_trunc), and nothing guards it. The fixtures run archive_mode=off, so pg_ls_archive_statusdir() returns zero rows and count(*) FILTER (WHERE name LIKE '%.ready') is indistinguishable from a bare count(*) — both 0, both Valid. No test asserts the value of the ready column at all: Test_StatArchiverQueries asserts only its NAME (line 83) and Test_StatArchiverQuery_NullsStayNull only Valid == true (line 141). Deleting the FILTER clause, or changing the pattern to '%.done' or '%ready%', leaves all five tests green on all six clusters. M3 caught deletion of the whole column, not corruption of the predicate inside it. This is the feature's headline metric — the archiving backlog — and it is the one thing here a reader would assume is tested.", + "recommendation": "Two additions, both reachable without fixtures or a forbidden seam. (a) Offline structural pin, mirroring the in-package precedent Test_SelectStatIOQuery_NullSafety (internal/query/io_test.go:66-107): assert.Contains(t, PgStatArchiverDefault, \"count(*) FILTER (WHERE name LIKE '%.ready')\") and assert.Contains(t, PgStatArchiverDefault, \"FROM pg_ls_archive_statusdir()\"). (b) A live semantic check of the predicate against synthetic input, which needs no archived segment and no writable directory: conn.QueryRow(\"SELECT count(*) FILTER (WHERE name LIKE '%.ready') FROM (VALUES ('000000010000000000000001.ready'),('000000010000000000000002.done'),('000000010000000000000003.ready.tmp')) AS t(name)\").Scan(&n), assert.Equal(t, 2, n) — this reddens on a dropped FILTER, on '%.done', and on '%ready%' (which would wrongly count the .ready.tmp). If the team instead prefers to defer the predicate to the stand run, patterns.md requires that be stated: the test file is currently silent about it, and the task's 'the tests cannot prove archiving behaviour' note lives only in the task file.", + "litmusTestFailed": true + }, + { + "severity": "major", + "category": "missing_coverage", + "location": "internal/query/archiver_test.go:236-254 (assertRestrictedSession), used at :176 and :214", + "issue": "The guard proves the session runs as a named non-superuser role. It does not prove the positive role holds pg_monitor AND NOTHING ELSE, nor that the deny role holds nothing — both stated requirements of the task. That matters because the roles are cluster-global and outlive the run, and SetupTestRole never normalizes a role that already exists: the DO block skips creation, and pgMonitor=false issues no REVOKE. So any grant that lands on pgcenter_test_archiver_monitor — including through the AC's own Step 4 procedure, which instructs engineers to drop and re-grant these very roles, or through a future task reusing the helper — permanently downgrades the positive test from 'pg_monitor is sufficient for this query' to 'some privileged role can run this query', with nothing to signal the change. M7 proved that dependency once, by hand, in a container that no longer exists; nothing keeps it proven. The mirror case on the deny role is loud rather than silent (it fails with 'An error is expected but got nil'), but that message names neither the role nor the grant, so the diagnosis costs a full re-derivation.", + "recommendation": "Give assertRestrictedSession an expected-membership parameter and assert it in the same round trip: SELECT current_user, (SELECT rolsuper FROM pg_roles WHERE rolname = current_user), pg_has_role(current_user, 'pg_monitor', 'USAGE'), coalesce((SELECT array_agg(r.rolname ORDER BY r.rolname) FROM pg_auth_members m JOIN pg_roles r ON r.oid = m.roleid WHERE m.member = (SELECT oid FROM pg_roles WHERE rolname = current_user)), '{}'). Assert the membership array equals []string{\"pg_monitor\"} and pg_has_role is true in Test_StatArchiverQuery_PgMonitorRoleSucceeds; equals []string{} and pg_has_role is false in Test_StatArchiverQuery_WithoutPgMonitorFails. pg_has_role covers indirect membership that array_agg over pg_auth_members alone would miss. This turns M7 from a one-off manual mutation into a standing gate, and makes a polluted cluster fail on a line that names the cause.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "internal/query/archiver_test.go:140-142", + "issue": "Test_StatArchiverQuery_NullsStayNull asserts NULL-ness for all nine columns and not one value, so several documented invariants are unfalsifiable. The 'Archiver' literal (archiver.go:38, documented there as 'the stable row identity across samples') can be changed to any non-empty string and every test stays green — the alias 'source' is what the column-name assertion checks, and any string is Valid. archived and failed can be swapped with each other, or replaced by any other non-NULL bigint expression, and stay green (both are 0 on fixtures, both Valid). date_trunc('seconds', ...) can be dropped from all three age columns: the value merely gains fractional seconds, Valid stays true, and the rendering an operator actually sees changes with no test noticing.", + "recommendation": "Assert values inside the existing loop, on the premise wantValid already relies on (the fixtures never archived): assert.Equal(t, \"Archiver\", values[0].String); assert.Equal(t, \"0\", values[2].String) and the same for values[5] (archived and failed counters); assert.Regexp(t, `^-?(\\d+ days? )?\\d{2}:\\d{2}:\\d{2}$`, values[8].String) to pin the truncation on stats_age so a dropped date_trunc reddens. For ready, do not pin 0 — assert it parses as a non-negative integer via strconv.Atoi, which guards the type and non-NULL-ness without coupling the suite to the state of the archive_status directory.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/query/archiver_test.go:73-76 (also :120-123, :160-163, :201-204)", + "issue": "Every live test skips on any error from NewTestConnectVersion, collapsing three different causes into one message: an unmapped version, a cluster that is down, and a connection refused for a reason the suite itself created. archiverVersions (line 17) is a hand-maintained list, so adding a future major there before internal/postgres/testing.go gains its port entry produces a permanent silent skip — precisely the failure this package already hardened against and documented (internal/postgres/testing_test.go:10-23: 'made a forgotten port entry invisible: every subtest for the new version passed while exercising a completely different server'), with the accepted mitigation shown at :30. archiver_test.go does not use it. The partial-skip case is the sharper one: with five of six clusters up, the suite is green and non-verbose output shows nothing, so a version-specific regression lands unnoticed. Verification Step 2 catches this only by a human reading -v subtest names, which is tech-debt [019] restated rather than closed.", + "recommendation": "Before each t.Skipf, add assert.NotContains(t, err.Error(), \"no test cluster port mapping\") exactly as internal/postgres/testing_test.go:30 does, so a bad entry in archiverVersions fails instead of skipping forever. Then make the skip refusable: if os.Getenv(\"PGCENTER_REQUIRE_FIXTURES\") != \"\" { t.Fatalf(\"postgres %d required but unavailable: %v\", version, err) } else { t.Skipf(...) }, and export that variable in the CI-image command of Verification Step 2. That converts Step 2's manual 'check the subtest names before believing the green' into an automatic gate and removes the whole class from this feature.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "internal/query/archiver_test.go:34-59 (selector test), :54 in particular", + "issue": "Everything about the query text except 'contains no coalesce' is asserted only behind live fixtures. assert.Equal(t, PgStatArchiverDefault, gotQuery) at :54 compares the selector's return value against the very constant it returns — it can fail only if a version branch is introduced, never if the query text is wrong — so it is not the content guard its position suggests. Consequently a host run (all live subtests skipped) leaves the locked nine-column order, the aliases and the FROM clause with no guard whatsoever, and M4 (swap ready/archived) is catchable only inside the CI image. The package already has two precedents for pinning SQL structure with no server: Test_SelectStatIOQuery_NullSafety (io_test.go:66) and Test_SelectStatWALQuery_PG19ColumnOrder (wal_test.go:44).", + "recommendation": "Add a server-free Test_StatArchiverQuery_ColumnOrder that walks archiverColumns and pins each alias's position in the constant: prev := -1; for _, col := range archiverColumns { idx := strings.Index(PgStatArchiverDefault, \" AS \"+col); assert.NotEqual(t, -1, idx, \"query must select %q\", col); assert.Greater(t, idx, prev, \"%q must follow the previous locked column\", col); prev = idx }, plus assert.True(t, strings.HasSuffix(PgStatArchiverDefault, \"FROM pg_stat_archiver\")). M3 and M4 then redden on the host as well as in the CI image, which also blunts the skip risk in the previous finding.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "internal/query/archiver_test.go:40-45, comment at :52-53", + "issue": "The comment claims the table exists 'so a future branch cannot be added silently', but the table covers only 140000-190000. A branch keyed on a future major — if version >= 200000, the exact shape wal.go already carries and wal_test.go:27 deliberately covers with a 200000 row — passes this table untouched, as does a branch on any version below 14. The cited precedent (io_test.go:43-48) has the same gap, but wal_test.go is the newer in-package standard and treats a forward version as required coverage for a version-keyed selector. For a selector that ignores its argument entirely, the invariant is 'any argument', and the table currently tests six specific ones.", + "recommendation": "Add rows {version: 200000}, {version: 130000} and {version: 0}, all with wantNcols 9 and wantDiffIntvl [2]int{0, 0}. Three lines, and the comment at :52-53 becomes true.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "internal/postgres/testing.go:59-81 (SetupTestRole) / internal/postgres/testing_test.go", + "issue": "The DO block's 'role already exists' branch never executes in a CI run. The image starts fresh, each test uses a distinct role name, and each name is created exactly once per cluster — so a single go test run only ever takes the CREATE path, on every cluster. Idempotency is both an explicit acceptance criterion and the property Task 03 will lean on when it adds its own roles through this helper, and it is verified only by the manual 'run the suite twice inside one container' procedure that CI does not perform. The helper is a published cross-task contract and has no test in internal/postgres/testing_test.go, which already exists and covers its two neighbours.", + "recommendation": "Add TestSetupTestRole_isIdempotent to internal/postgres/testing_test.go, following the local skip style at :28-32: connect to 170000, call SetupTestRole(db, \"pgcenter_test_setup_idem\", true) twice in a row, require.NoError on both, assert QueryRow(\"SELECT current_user\") returns the role name, then Exec(\"RESET ROLE\") and assert current_user is back to postgres. Add a second case that documents the helper's real contract on a pre-existing role — call it once with pgMonitor true and once with false and assert pg_has_role(current_user,'pg_monitor','USAGE') is still true — so the 'reverting a mutation to role setup does not undo it on the server' caution the AC spells out in prose becomes an asserted property instead of tribal knowledge.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/query/archiver_test.go:257-262 (resetRole), registered at :174 and :212", + "issue": "Both privilege tests open a dedicated connection and close it at the end of the same subtest, so a leaked SET ROLE cannot reach any later assertion — the hazard Decision 18 and the task's Edge cases describe ('a leaked SET ROLE poisons later tests on the same connection') is structurally absent as written. No mutation reddens resetRole: deleting both defers leaves the suite green, and the assert.NoError inside it is the one assertion in this file with no falsifying mutation. It also never verifies the reset took effect — only that the Exec did not error. Keeping the call is correct (Decision 18 mandates it and it is the belt to the connection-per-test braces), but it must not be counted as a guard, and the AC line 'both privilege tests RESET ROLE even when they fail' is likewise closed only by a manual broken-assertion experiment.", + "recommendation": "Keep the defer and make it assert something observable: after RESET ROLE, run QueryRow(\"SELECT current_user\").Scan(&u) and assert.NotEqual(t, wantRole, u) — pass the role name to resetRole for that. Add one line to its comment stating that the per-test connection, not RESET ROLE, is what currently contains a leak, so a later refactor that shares one connection between the two privilege tests does not assume a guard that is not yet exercised.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/query/archiver_test.go:95-145 (Test_StatArchiverQuery_NullsStayNull) — the known deviation", + "issue": "Verdict on the deviation first: the split is the right call and should stand. The TDD Anchor contradicts itself — its preamble says tests 1 and 3 run without PostgreSQL, while its own bullet for test 3 requires scanning fixture rows into sql.NullString receivers and asserting Valid flags, which no amount of care makes server-free. The implementation resolved the contradiction the only way that keeps both halves, and it placed the coalesce assertion at :96-97, outside the version loop and before any connect, which is what makes Verification Step 1 ('on the host only tests 1 and 3 produce a real red') satisfiable at all; inside the loop it would have skipped on the host and the red-first evidence for Decision 3 would have been unobtainable without the CI image. M5 confirms both halves redden. The residual cost is only in naming: one test name now spans a server-free assertion and a per-version live loop, so a host run reports PASS while exactly one of its ten assertions executed, and the deviation itself is recorded nowhere in the tree.", + "recommendation": "Split by name, not by logic: move :96-97 into Test_StatArchiverQuery_NoCoalesce (server-free, the host-side red-first evidence for Decision 3) and leave the fixture loop in Test_StatArchiverQuery_NullsStayNull, so a PASS names exactly what ran and a skipped run cannot report a green for the coalesce guard's neighbour. Then record the deviation in 017-feat-wal-archiver-decisions.md per Post-completion: state that the anchor's preamble contradicts its own bullet for test 3, and that the bullet won.", + "litmusTestFailed": false + } + ], + "metrics": { + "filesReviewed": 5, + "litmusTest": { + "checked": 16, + "passed": 12, + "failed": 4 + }, + "coverageAssessment": "adequate", + "pyramidBalance": { + "unit": 1, + "integration": 4, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-01.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01.md similarity index 96% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-01.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01.md index dfb4ad54..aeba3206 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-01.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-01.md @@ -1,5 +1,5 @@ --- -status: planned # planned -> in_progress -> done +status: done # planned -> in_progress -> done depends_on: [] # ID задач-зависимостей (строки: ["01", "02"]) wave: 1 # волна параллельного выполнения skills: [code-writing] # МАССИВ скиллов для загрузки @@ -181,9 +181,11 @@ Two cautions before running any of them: red on the column-name order (the count alone must not be what catches it). - [ ] Mutation — wrap `last_archived_wal` in `coalesce(last_archived_wal, '-')`: `Test_StatArchiverQuery_NullsStayNull` turns red. -- [ ] Mutation — delete the `SET ROLE` line from `Test_StatArchiverQuery_PgMonitorRoleSucceeds`, so the - test runs on the fixture superuser connection: the test turns red **on its own - `current_user` / `rolsuper` guard**, before it ever reaches the query. This is the mutation that +- [ ] Mutation — in `Test_StatArchiverQuery_PgMonitorRoleSucceeds`, skip the `SetupTestRole` call and + the `SET ROLE` it performs, so the test runs on the fixture superuser connection: the test turns + red **on its own `current_user` / `rolsuper` guard**, before it ever reaches the query. Mutate + the call site in the test, not the helper — the helper is shared with task 03, so editing it + would redden that task's tests too and obscure which gate actually fired. This is the mutation that proves the positive privilege test is exercising privileges rather than riding the superuser fixture connection. If it stays green, the guard is missing or asserts nothing. *Do not substitute the older "swap `pg_ls_archive_statusdir()` for `pg_ls_dir('pg_wal/archive_status')`" @@ -302,7 +304,7 @@ Two cautions before running any of them: already carry the mutated grants. Run `DROP ROLE IF EXISTS ` on every cluster under test, or restart the container, before trusting the result in either direction. - Step 5 — production-build guard: `grep -n '"testing"' internal/postgres/testing.go` finds nothing, and - `go build ./cmd` succeeds (the main package is `./cmd` — the repository root holds no Go files, so + `make build` (note: `go build ./cmd` fails — Go refuses to write an executable named `cmd` next to the `cmd/` directory; use `make build` or `go build -o /dev/null ./cmd` as a compile check) succeeds (the main package is `./cmd` — the repository root holds no Go files, so `go build .` / `go run .` do not work here). - Step 6 — `make lint` and `make vuln` on the host: clean, with no revive complaint about the unused selector parameter. @@ -440,7 +442,7 @@ Two cautions before running any of them: - This task does **not** add the view, so nothing renders yet. Verification is entirely `go test ./internal/query/... ./internal/postgres/...` inside the CI image — resist the urge to wire `view.go` "just to see it", that is Task 5 and would create a wave conflict. If you want a build - sanity check, it is `go build ./cmd`: the repository root contains no Go files and the main package + sanity check, it is `make build` (note: `go build ./cmd` fails — Go refuses to write an executable named `cmd` next to the `cmd/` directory; use `make build` or `go build -o /dev/null ./cmd` as a compile check): the repository root contains no Go files and the main package lives in `./cmd`, so `go build .` and `go run .` fail with "no Go files". ## Reviewers diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02-dev-code-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02-dev-code-reviewer-review.json new file mode 100644 index 00000000..9c4bd6f1 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02-dev-code-reviewer-review.json @@ -0,0 +1,51 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "Task 02 is implemented exactly as specified: a third PG 19 branch in SelectStatWALQuery returning (PgStatWALPG19, 8, [2]int{2,6}), guarded by the PostgresV19 constant, with the PG 14-17 and PG 18 constants and return statements byte-identical (verified: the wal.go diff is additions only). The new tests are strong — I independently re-ran mutations M1-M4 in an isolated worktree and all four turned the named tests red, and the full query suite in the CI image (lesovsky/pgcenter-testing:0.0.11) is green with the PG 19 subtest of Test_StatWALQueries actually running (not skipped) and asserting the 8 live column names. No critical or major findings; three minor, optional observations, two of which conflict with an explicit acceptance criterion and should not be applied blindly.", + "criticalIssues": [], + "suggestions": [ + { + "file": "internal/query/wal.go", + "line": 13, + "severity": "minor", + "category": "readability", + "suggestion": "The PgStatWALDefault doc comment still reads \"defines query for pg_stat_wal (PG 18+)\", which is no longer true — PG 19+ now resolves to PgStatWALPG19, so this constant covers PG 18 only. The same drift affects the name: with three branches, \"Default\" no longer denotes the newest layout, unlike the bgwriter file where every constant is version-bounded (PgStatBgwriterPG14 \"PG 14-16\", PgStatBgwriterPG17 \"PG 17\", PgStatBgwriterPG18 \"PG 18+\"). Note the tension before acting: this task's acceptance criteria require that the wal.go diff shows only additions and that the PG 18 constant is left exactly as it is, so a comment or rename edit here would violate the task contract. Recommend either a one-line comment fix agreed with the task owner, or a follow-up item to rename PgStatWALDefault -> PgStatWALPG18 (it is referenced in internal/view/view.go only indirectly; the static QueryTmpl there is PgStatWALPG14, so the rename is confined to internal/query).", + "benefit": "Keeps the version-range comments accurate as the file grows a fourth branch; prevents a future reader from assuming PgStatWALDefault is still the fallthrough for the newest PG.", + "optional": true + }, + { + "file": "internal/query/wal_test.go", + "line": 99, + "severity": "minor", + "category": "best-practices", + "suggestion": "assert.NoError(t, err) after conn.Query is non-fatal, so a failing query (e.g. wal_fpi_bytes renamed at PG 19 RC — the caveat this task carries) lets the subtest continue into FieldDescriptions() and produce a cascade of secondary failures around the real cause. There is no panic risk (pgx v5 returns a non-nil &baseRows{err:...} on every Query error path — verified in pgx v5.9.2/v5.10.0 conn.go:760), so this is purely about failure readability. require.NoError would stop at the first, meaningful message; the test already imports require for exactly this reason at line 113. Note this mirrors the io_test.go precedent the task told you to follow, so leaving it as-is for consistency is a defensible choice.", + "benefit": "A failed live query reports one actionable error instead of three derived ones.", + "optional": true + }, + { + "file": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver-decisions.md", + "line": 0, + "severity": "minor", + "category": "maintainability", + "suggestion": "The acceptance criteria require the M1-M4 mutation runs to be recorded in the feature decisions log, and the Post-completion checklist points at 017-feat-wal-archiver-decisions.md — that file does not exist in docs/features/017-feat-wal-archiver/ yet. Create it (or append to it) with the task-02 entry, including the mutation results. For the record, I re-verified all four independently in a detached worktree: M1 (DiffIntvl {2,7}) reddens Test_SelectStatWALQuery at 190000 and 200000; M2 (fpi,KiB moved after stats_age) reddens Test_SelectStatWALQuery_PG19ColumnOrder; M3 (== PostgresV19) reddens the 200000 row; M4 (wal_fpi_bytes added to PgStatWALDefault) reddens Test_SelectStatWALQuery_LegacyBranchesUntouched.", + "benefit": "Closes the task's own acceptance criterion and leaves the mutation evidence discoverable for the next reviewer.", + "optional": true + }, + { + "file": "internal/query/wal.go", + "line": 43, + "severity": "minor", + "category": "readability", + "suggestion": "The function now mixes the named constant (version >= PostgresV19) with a bare literal (version >= 180000) in adjacent branches. The task explicitly rules this out of scope for Task 02 (\"The 180000 literal in the existing branch stays a literal\"), so no action here — recording it only so it can be picked up by whichever task next owns this file, together with the PgStatWALDefault rename above.", + "benefit": "One consistent version-constant style across the selector once the file is next opened for edit.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 2, + "criticalIssuesCount": 0, + "majorIssuesCount": 0, + "minorIssuesCount": 4, + "testCoverageAssessment": "excellent" + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02-dev-security-auditor-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02-dev-security-auditor-review.json new file mode 100644 index 00000000..7b68f5d3 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02-dev-security-auditor-review.json @@ -0,0 +1,51 @@ +{ + "status": "approved", + "summary": { + "totalFindings": 0, + "critical": 0, + "major": 0, + "minor": 0 + }, + "findings": [], + "scope": { + "task": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-02.md", + "auditedFiles": [ + "internal/query/wal.go", + "internal/query/wal_test.go" + ], + "excluded": [ + "cmd/report/report.go", + "cmd/report/report_test.go", + "internal/postgres/testing.go", + "internal/query/archiver.go", + "internal/query/archiver_test.go" + ], + "excludedReason": "Out of scope for Task 02 — owned by a concurrent agent on other tasks in the same branch." + }, + "verification": { + "injectionSurface": "PgStatWALPG19 is a compile-time const built only from string literals concatenated with '+'. No fmt.Sprintf, no '%s', no variable interpolation, no user input. grep for '{{' in internal/query/wal.go returns nothing, so query.Format (text/template) renders the template with zero actions and the executed SQL is byte-identical to the constant — the Options struct (including operator-supplied fields such as PGSSSchema and PgSSQueryLen) never reaches this query. Runtime path confirmed end to end: internal/view/view.go:390 assigns QueryTmpl from SelectStatWALQuery, view.go:418-422 sets view.Query = query.Format(view.QueryTmpl, opts), and nothing concatenates user input into it afterwards. CWE-89 surface: none introduced.", + "identifierQuoting": "The new alias contains a comma and is correctly double-quoted as \"fpi,KiB\" (Go backtick literal), matching the existing \"wal,KiB\" precedent. Both are static, so quoting is not an escaping decision made at runtime.", + "informationDisclosure": "The added expression round(wal_fpi_bytes / 1024, 2) exposes aggregate full-page-image WAL volume from pg_stat_wal — an operational counter with no PII, credentials, query text or table contents. No new disclosure relative to PgStatWALDefault. The pg_ls_waldir() subselect (superuser / pg_monitor only) is copied verbatim from the two pre-existing constants and is unchanged, so this task neither widens nor narrows the privilege requirement of the wal screen.", + "secrets": "No hardcoded credentials, tokens, connection strings or key material in either file. The test connects via postgres.NewTestConnectVersion (out-of-scope helper) and passes no literal credentials.", + "errorAndResourceHandling": "Test_StatWALQueries now defers conn.Close() instead of closing only on the success path, and closes rows before the connection — a leak fix, not a regression. pgx v5 Conn.Query returns a non-nil *baseRows even on error (conn.go:760 in pgx v5.9.2/v5.10.0), so the FieldDescriptions() call after a failed Query cannot nil-panic; a query failure degrades to a normal assertion failure. Query results are never logged or printed.", + "dependencies": "No new modules. github.com/stretchr/testify v1.11.1 (source of the newly imported require package) is already in go.mod. No CVE surface added.", + "toolsRun": [ + "gofmt -l internal/query/wal.go internal/query/wal_test.go — clean", + "go vet ./internal/query/ — clean", + "gosec ./internal/query/ — no issues reported", + "go test ./internal/query/ -run Test_SelectStatWALQuery -v — all subtests pass (no PostgreSQL required)" + ] + }, + "owaspCoverage": { + "A01_BrokenAccessControl": "N/A — no authorization logic; access is governed by the PostgreSQL role the operator connects with, unchanged by this task.", + "A02_CryptographicFailures": "N/A — no crypto, no secret storage.", + "A03_Injection": "Clean — see verification.injectionSurface.", + "A04_InsecureDesign": "Clean — version branching follows the established SelectStatBgwriterQuery pattern; the '>=' boundary is pinned by the 200000 test row.", + "A05_SecurityMisconfiguration": "N/A — no configuration or defaults introduced.", + "A06_VulnerableComponents": "Clean — no new dependencies.", + "A07_AuthFailures": "N/A — no authentication code.", + "A08_DataIntegrity": "N/A — no deserialization, no update or CI/CD mechanism touched.", + "A09_LoggingMonitoring": "N/A — no security-relevant events in scope; no sensitive data written to logs or test output.", + "A10_SSRF": "N/A — no URLs or outbound requests." + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02-dev-test-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02-dev-test-reviewer-review.json new file mode 100644 index 00000000..697b25fe --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02-dev-test-reviewer-review.json @@ -0,0 +1,54 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "needs_improvement", + "summary": "Strong, deliberately mutation-driven test set: the version table pins both boundaries (179999/189999 via the legacy-identity test, 200000 via the forward row), the column-order guard pins fpi,KiB between its neighbours and stats_age at the tail, and Test_StatWALQueries was correctly upgraded from an execution smoke test into a live named-column contract with a version-generic stats_age-outside-DiffIntvl assertion. M1-M4 are genuinely covered. One real hole remains: the only new production expression this task adds is round(wal_fpi_bytes / 1024, 2), and nothing anywhere asserts the /1024 KiB conversion — replacing it with a bare wal_fpi_bytes keeps all four tests green while the screen would show bytes under a KiB header (verified empirically, not reasoned). Secondary: the no-Postgres guards inspect only three substrings, so the declared Ncols=8 is reconciled against the real select list only inside the skippable live test.", + "findings": [ + { + "severity": "major", + "category": "missing_coverage", + "location": "internal/query/wal.go:31 (guarded by internal/query/wal_test.go:44-60 and :118-124)", + "issue": "The KiB conversion in the one expression this task adds — round(wal_fpi_bytes / 1024, 2) AS \"fpi,KiB\" — is not pinned by any test. Test_SelectStatWALQuery asserts only Ncols/DiffIntvl; Test_SelectStatWALQuery_PG19ColumnOrder matches only the alias substring `AS \"fpi,KiB\"` and its relative position; Test_StatWALQueries asserts only column NAMES, which are unchanged by any edit to the expression body. Verified empirically rather than by reasoning: mutating the constant to `wal_fpi_bytes AS \"fpi,KiB\"` (or `/ 1000`) leaves every one of the four guard assertions passing, including the ordered live-header list. The resulting user-visible defect is a column labelled KiB rendering raw bytes — a 1024x error on the wal screen that this task's own mutation battery (M1-M4) does not probe, since all four mutations target position, interval, branch condition and constant identity, never the expression body.", + "recommendation": "Add one no-Postgres assertion that derives the PG 19 constant from the PG 18 one, which pins the expression text, its exact insertion point, and the untouched-ness of every other column in a single line. Verified to pass against the current constants: in Test_SelectStatWALQuery_PG19ColumnOrder add `assert.Equal(t, PgStatWALPG19, strings.Replace(PgStatWALDefault, \"wal_fpi AS fpi, wal_buffers_full\", `wal_fpi AS fpi, round(wal_fpi_bytes / 1024, 2) AS \"fpi,KiB\", wal_buffers_full`, 1), \"PG 19 query must be the PG 18 query plus exactly the fpi,KiB expression\")`. Under the dropped-/1024 mutation this assertion fails, and it also fails under M2. Optionally strengthen further inside the PG 19 branch of Test_StatWALQueries, where a live connection is already open: scan the single pg_stat_wal row into a float and compare against an independently issued `SELECT round(wal_fpi_bytes / 1024, 2) FROM pg_stat_wal` — that additionally proves the diffed cell is strconv.ParseFloat-parseable, which is the invariant diffPair (internal/stat/postgres.go:792) keys on for every column inside DiffIntvl.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "internal/query/wal_test.go:44-60", + "issue": "The declared Ncols=8 is never reconciled with the PG 19 select list except inside Test_StatWALQueries, which is skippable. The no-Postgres guards look up exactly three substrings (wal_fpi AS fpi, the fpi,KiB alias, wal_buffers_full) plus a suffix, so dropping or renaming any of the remaining five columns — source, waldir_size, wal,KiB, records, stats_age's date_trunc body — inside PgStatWALPG19 leaves all host-runnable tests green while SelectStatWALQuery keeps claiming 8 columns to view.Configure. Test_SelectStatWALQuery_LegacyBranchesUntouched pins constant identity for the PG 14 and PG 18 branches but nothing pins the PG 19 branch's returned string at all — it is asserted only by property, never by identity or completeness.", + "recommendation": "The derivation assertion recommended in the previous finding closes this too: because it equates PgStatWALPG19 to PgStatWALDefault with exactly one expression inserted, the 7+1 column count and every carried-over column become structurally guaranteed without a fixture. If it is preferred to keep the two concerns separate, add to Test_SelectStatWALQuery_PG19ColumnOrder: `q, ncols, _ := SelectStatWALQuery(PostgresV19); assert.Equal(t, PgStatWALPG19, q); assert.Equal(t, ncols, strings.Count(q, \" AS \") - 1)` — noting the waldir_size subquery contributes two ` AS ` occurrences, so the derivation assertion is the cleaner form.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/query/wal_test.go:92-95", + "issue": "The strongest assertions this task introduces — the live 8-column count and the ordered PG 19 header — sit behind `t.Skipf(\"postgres %d not available\")`, and a skip is green. The task file itself flags this (\"A skipped PG 19 subtest is not a pass — check the subtest output\"), and the orchestrator confirmed the subtest ran in this round, but nothing in the test enforces it: a future CI image whose PG 19 fixture fails to start reports success while the entire live contract for the new column goes unverified. The pattern is pre-existing across the package (io_test.go does the same), but this task is the first to place a genuinely new behavioural contract behind it rather than a smoke exec.", + "recommendation": "Make the skip opt-out rather than unconditional in CI: replace `t.Skipf(...)` with `if os.Getenv(\"PGCENTER_FIXTURES_REQUIRED\") != \"\" { require.NoError(t, err, \"postgres %d fixture must be available in CI\", version) }; t.Skipf(...)`, and export that variable in the CI-image invocation from the task's Verification Steps. Host runs keep skipping; the gate run can no longer pass by skipping. If touching the shared skip idiom is considered out of scope for this task, record the decision in the decisions log so the exposure is tracked rather than forgotten.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/query/wal_test.go:98-106", + "issue": "`assert.NoError(t, err)` after `conn.Query(q)` is non-fatal, so on a query error the test continues into `rows.FieldDescriptions()` with a closed error-carrying Rows, collects a nil names slice, and then reports a cascade of three further failures (Len, require.Greater, Equal) that obscure the actual cause. This is exactly the failure mode the task's own caveat predicts — if `wal_fpi_bytes` is renamed at PG 19 beta3/RC, the undefined-column error is the signal, and it should be the first and only thing the output shows.", + "recommendation": "Use `require.NoError(t, err)` for the Query call so the subtest aborts at the real cause; `require` is already imported. Same reasoning applies to the `Format` call at line 89-90, where a template error makes every later assertion meaningless.", + "litmusTestFailed": false + } + ], + "metrics": { + "filesReviewed": 2, + "litmusTest": { + "checked": 18, + "passed": 17, + "failed": 1 + }, + "coverageAssessment": "adequate", + "pyramidBalance": { + "unit": 3, + "integration": 1, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-02.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02.md similarity index 99% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-02.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02.md index d51bd4af..af03cc13 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-02.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-02.md @@ -1,5 +1,5 @@ --- -status: planned # planned -> in_progress -> done +status: done # planned -> in_progress -> done depends_on: [] # ID задач-зависимостей (строки: ["01", "02"]) wave: 1 # волна параллельного выполнения skills: [code-writing] # МАССИВ скиллов для загрузки @@ -189,7 +189,7 @@ at the repository root (no `.go` files there) — `go run .` fails; the binary i **Files:** -- `internal/query/wal.go` (33 lines today) — holds two constants, `PgStatWALPG14` (lines 5-11, PG 14-17, +- `internal/query/wal.go` (32 lines today) — holds two constants, `PgStatWALPG14` (lines 5-11, PG 14-17, 11 columns) and `PgStatWALDefault` (lines 15-21, PG 18+, 7 columns), plus `SelectStatWALQuery(version int) (string, int, [2]int)` (lines 25-32) with two branches: `>= 180000` → `(PgStatWALDefault, 7, {2,5})` and the fallthrough → `(PgStatWALPG14, 11, {2,9})`. Add the third diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-code-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-code-reviewer-review-round2.json new file mode 100644 index 00000000..0257c5d1 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-code-reviewer-review-round2.json @@ -0,0 +1,89 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "Round-1's single major is fixed: PgstatOverview.ArchivingBacklogValid (internal/stat/postgres.go:84) now reads \"false when the aggregate failed (e.g. missing privilege, 42501) -> n/a\", which agrees with Decision 11, with the consumer comment at :288-291, and with Test_collectOverviewStat_Degradation. All four round-1 minors are either applied or structurally resolved, and the two findings from the other reviewers that landed in these files (the serverless structure test, the falsifiable parity check) are correct and strengthen the change. I re-verified the mechanical claims independently: go vet clean on both packages, gofmt clean on all four in-scope files, `grep -rn \"has pg_monitor\" internal/` empty, and every surviving `pg_ls_dir` occurrence is explanatory prose or the NotContains guard. Two optional minors remain, neither of which needs a round 3.", + "criticalIssues": [], + "suggestions": [ + { + "file": "internal/stat/postgres.go", + "line": 291, + "severity": "minor", + "category": "readability", + "suggestion": "The new closing clause reads \"archive_mode=off is not an error: the status directory is simply empty, so the field is a real 0.\" The conclusion is right and the ambiguity round 1 flagged is gone, but \"simply empty\" overstates it: .ready files are only created when XLogArchivingActive(), so archive_mode=off produces none — yet .done files left over from a period when archiving WAS on persist in pg_wal/archive_status, and the directory is then non-empty while the backlog is still 0. The precise statement is that it contains no .ready files. Suggested wording: \"archive_mode=off is not an error: nothing writes .ready files, so the FILTER counts none and the field is a real 0.\" Purely a wording nit on a line that is already correct in its conclusion — record it or fold it into the finalization comment pass; it does not warrant another review round.", + "benefit": "Keeps the last remaining approximation out of the comment block this task exists to make trustworthy.", + "optional": true + }, + { + "file": "internal/stat/postgres_test.go", + "line": 282, + "severity": "minor", + "category": "maintainability", + "suggestion": "Carry-forward from round 1, now sharpened rather than resolved. Adding the pg_auth_members membership assertion was the right call — the two packages' guards are equal-strength again, and the stat-side guard can no longer decay from \"pg_monitor is sufficient\" into \"some privileged role works\". The cost is that internal/stat/postgres_test.go:282-307 is now a near-verbatim copy of internal/query/archiver_test.go:285-321, including the four-branch SELECT and the PG-16 DISTINCT rationale. The duplication remains structurally forced (assertRestrictedSession is package-level in `query`; internal/postgres/testing.go is task-01 territory), so this is not a defect in the delivered change. The finalization move is unchanged and now has a second call site arguing for it: promote assertRestrictedSession next to SetupTestRole in internal/postgres so both packages share one implementation and one DISTINCT rationale.", + "benefit": "One home for the privilege-test scaffolding once the task-ownership fences are lifted; today's two copies cannot drift apart silently.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 4, + "criticalIssuesCount": 0, + "majorIssuesCount": 0, + "minorIssuesCount": 2, + "testCoverageAssessment": "excellent" + }, + "round1FindingsVerification": [ + { + "round1Finding": "major — internal/stat/postgres.go:84 struct field comment claimed archive_mode=off invalidates ArchivingBacklogValid", + "resolution": "fixed", + "evidence": "Now reads `// false when the aggregate failed (e.g. missing privilege, 42501) -> n/a`. Consistent with Decision 11, with :288-291, and with Test_collectOverviewStat_Degradation:227-232. No comment in the tree now attributes n/a to archive_mode=off." + }, + { + "round1Finding": "minor — :288 consumer comment presented archive_mode=off as a failure cause via the shared colon", + "resolution": "fixed", + "evidence": "Split into two sentences: privilege as the failure reason, archive_mode=off as a separate note that the count is legitimately zero. Only the wording nit above remains." + }, + { + "round1Finding": "minor — the stat-package inline guard omitted the pg_auth_members membership check present in internal/query", + "resolution": "fixed", + "evidence": "postgres_test.go:288-307 now scans array_agg(DISTINCT rolname) and asserts memberOf == []string{\"pg_monitor\"}; the four gate results are ANDed before collect runs. The deferred RESET ROLE additionally re-reads current_user and asserts it left the role, matching query.resetRole:327-337." + }, + { + "round1Finding": "minor — backlogRoleMonitor was declared as an untied literal in two packages", + "resolution": "resolved structurally", + "evidence": "internal/stat now owns pgcenter_test_backlog_collect. Verified the four test roles in the tree are pairwise distinct (pgcenter_test_archiver_* and pgcenter_test_backlog_monitor/norole in internal/query, pgcenter_test_backlog_collect in internal/stat), so no rename hazard and no cross-package name to keep in sync." + }, + { + "round1Finding": "minor — shared role name coupled the mutation procedure and could race outside `go test -p 1`", + "resolution": "obsolete", + "evidence": "Both consequences followed from the shared name and disappear with it. The mutation `SetupTestRole(..., false)` on the query call site no longer needs a matching DROP ROLE for the stat package, and SetupTestRole's non-atomic DO block can no longer be reached concurrently for the same role name by two packages. The comments at overview_test.go:17-22 and postgres_test.go:240-242 both state this rationale correctly." + } + ], + "newWorkVerification": [ + { + "item": "Test_ArchivingBacklogQuery_Structure (test reviewer's major)", + "assessment": "correct", + "evidence": "All three Contains substrings match OverviewArchivingBacklog character-for-character. The NotContains guard is sound and not self-defeating: \"pg_ls_archive_statusdir\" does not contain the substring \"pg_ls_dir\" (after `pg_ls_` comes `archive`), so the assertion passes on the fixed query and reddens on the FROM-reversion mutation, which the reported mutation run confirms. Mirrors Test_StatArchiverQuery_Structure. The stated motivation is accurate: with archive_mode=off and an empty status dir, every live backlog assertion reduces to 0 >= 0, so the FILTER and pg_size_bytes mutations were genuinely unguarded before this test." + }, + { + "item": "Superuser baseline parity check replacing assert.True(t, got.Valid)", + "assessment": "correct, and a justified deviation from the task's TDD anchor", + "evidence": "collectOverviewStat sets s.Valid = true unconditionally at postgres.go:202, so the anchor's `Valid` assertion was vacuous and could never redden. The replacement compares TotalSizeValid and DatabasesCount against a superuser sample taken before the role switch, which is falsifiable: pg_database_size needs pg_read_all_stats (held via pg_monitor), so a privilege regression in the neighbouring size aggregate now shows as a divergence. Stability checked — no t.Parallel anywhere in internal/stat or internal/query and no CREATE/DROP DATABASE in the suite, so DatabasesCount cannot drift between the two calls. `base, _ :=` discards a time.Duration, not an error (the function returns (PgstatOverview, time.Duration)), so nothing is silently swallowed. Worth one line in the decisions file so the anchor's mention of `Valid` is not read later as an unmet acceptance item." + }, + { + "item": "Cross-file consistency (dimension 10)", + "assessment": "clean", + "evidence": "postgres.SetupTestRole(db *DB, name string, pgMonitor bool) error, query.resetRole(t, conn, role), query.assertRestrictedSession(t, conn, wantRole, wantPgMonitor) bool, collectOverviewStat(db, props, itv, prev, skipDatabasesSize) (PgstatOverview, time.Duration) and GetPostgresProperties(db) (PostgresProperties, error) all match their call sites. New imports fmt and pgconn in overview_test.go are both used; internal/stat needed no new import. go vet exit 0 on both packages." + }, + { + "item": "Scope fence", + "assessment": "held", + "evidence": "The diff touches exactly the four permitted files. collectOverviewStat's degrade path (postgres.go:292-296) is byte-identical apart from its comment; top/stat.go, docs/decisions-log.md and internal/postgres/testing.go are untouched; no CREATE ROLE/GRANT SQL was written in either test file. Note for the record: gofmt -l reports internal/stat/procpidstat_test.go, but that file is unmodified here and its deviation predates the feature (commit 99c8413) — out of scope, not a finding against this task." + } + ], + "pushbackAccepted": [ + { + "suggestion": "Replace the inline connect + t.Skipf in the two new query tests with connectArchiverFixture", + "verdict": "accepted, do not change", + "reasoning": "Two independent reasons hold. (1) The task's Implementation Hints (task-03.md:332-333) prescribe postgres.NewTestConnectVersion(version) with t.Skipf and conn.Close() per iteration, explicitly as the existing shape of this file — adopting the helper would mean renaming it inside internal/query/archiver_test.go, a task-01 file this task must not restructure. (2) I checked what the helper's extra guard actually buys here: connectArchiverFixture hard-fails instead of skipping when a version is absent from the port map, and internal/postgres/testing.go:30-35 maps all six versions in overviewVersions (140000-190000). The guard is therefore not load-bearing for these tests today. Sharing it is a finalization cleanup, not a task-03 correction." + } + ] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-code-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-code-reviewer-review.json new file mode 100644 index 00000000..af5d3628 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-code-reviewer-review.json @@ -0,0 +1,60 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "The SQL change is minimal and correct: pg_ls_archive_statusdir() supplies the same named `name` column, the SELECT list is byte-identical, the result stays a non-NULL bigint, and the degrade-to-n/a path in collectOverviewStat is untouched as the scope fence requires. The three new tests are strong — the anti-vacuous current_user/rolsuper guard runs before the aggregate, the negative half pins SQLSTATE 42501 plus the function name, and role setup goes through the task-01 helper with RESET ROLE deferred right after the successful SET ROLE. One finding: the PgstatOverview.ArchivingBacklogValid field comment (internal/stat/postgres.go:84) was not part of the enumerated four and now contradicts both Decision 11 and the corrected comment 200 lines below it.", + "criticalIssues": [], + "suggestions": [ + { + "file": "internal/stat/postgres.go", + "line": 84, + "severity": "major", + "category": "maintainability", + "suggestion": "The struct field comment still reads `ArchivingBacklogValid bool // false on archive_mode=off / missing privilege (42501) -> n/a`. The archive_mode=off half is wrong and now contradicts (a) Decision 11, (b) the comment this task rewrote at :288-291, (c) Test_collectOverviewStat_Degradation:230-232 which states archive_mode=off is a real 0 with Valid=true, and (d) the constant's own doc comment about missing_ok=true. pg_wal/archive_status is created by initdb and exists regardless of archive_mode, so the aggregate returns 0 with Valid=true on the fixtures — it never degrades for that reason (it did not under pg_ls_dir either). The privilege half is still accurate, but only for a role holding neither superuser nor pg_monitor. Suggested text: `// false only when the aggregate itself errs (e.g. 42501 for a role holding neither superuser nor pg_monitor) -> n/a`. Scope note: this line is not one of the four comments the task enumerates, and the task's fence says only the SQL function and those four comments change — so this is deliberately filed as a suggestion, not a blocker. It is, however, the same class of defect the task exists to eliminate (a stale privilege/degradation comment that a future reader would trust), it lives in an in-scope file, and leaving it means the tree carries two contradictory accounts of when the field is n/a. Either fix the one line here or record it explicitly as a finalization carry-forward in the decisions file so it is not lost.", + "benefit": "Removes the last comment in the touched files that misstates when the backlog degrades — the exact failure mode (a trusted-but-wrong comment) that let the ADR [010] privilege error survive unnoticed.", + "optional": false + }, + { + "file": "internal/stat/postgres.go", + "line": 288, + "severity": "minor", + "category": "readability", + "suggestion": "The rewritten consumer comment reads \"OWN QueryRow so any failure degrades this field alone to n/a instead of aborting the sample: the aggregate needs superuser or pg_monitor (pg_ls_archive_statusdir), and archive_mode=off leaves nothing to count.\" Each clause is true in isolation, but the colon presents both as reasons a failure occurs, and archive_mode=off is not a failure — it yields a successful 0 with Valid=true. Splitting the sentence (privilege as the failure reason; archive_mode=off as a separate note that the count is legitimately zero) removes the ambiguity.", + "benefit": "The comment stops implying the pre-change (incorrect) degradation story it was rewritten to remove.", + "optional": true + }, + { + "file": "internal/stat/postgres_test.go", + "line": 258, + "severity": "minor", + "category": "maintainability", + "suggestion": "The inline restricted-session guard (current_user / rolsuper / pg_has_role) re-implements internal/query/archiver_test.go:285-321 in a weaker form: it omits the pg_auth_members check that pins the role to `pg_monitor and nothing else`. The duplication is structurally forced — assertRestrictedSession is a package-level helper in `query`, and internal/postgres/testing.go is task-01 territory this task must not edit — so this is not a defect in the delivered change. Two options worth recording: add the membership assertion here for parity (a role that accumulated extra grants from an earlier mutation run would currently pass in `stat` but fail in `query`), and propose promoting the guard next to SetupTestRole in internal/postgres at feature finalization so both packages share one implementation.", + "benefit": "Equal-strength guards in both packages, and a single home for the privilege-test scaffolding once the task-ownership fences are lifted.", + "optional": true + }, + { + "file": "internal/stat/postgres_test.go", + "line": 240, + "severity": "minor", + "category": "maintainability", + "suggestion": "`backlogRoleMonitor = \"pgcenter_test_backlog_monitor\"` is declared as a literal in both internal/query/overview_test.go:19-21 and here, with only a comment tying them together. Renaming one side leaves the other silently creating a second role on the fixture clusters rather than failing to compile. Same structural constraint as the guard above (no shared test-only package), so the pragmatic fix is the same finalization move: one exported constant beside SetupTestRole.", + "benefit": "Removes a rename hazard that no compiler or test would catch.", + "optional": true + }, + { + "file": "internal/query/overview_test.go", + "line": 19, + "severity": "minor", + "category": "best-practices", + "suggestion": "Because the same role name is now created from two packages, two consequences deserve one clause in the comment. (1) The mutation procedure in the acceptance criteria — flip a call site to SetupTestRole(..., false) and drop the role first — must drop the role for BOTH packages on the PG 17 cluster, otherwise the stat-package test re-grants pg_monitor and the query-package mutation reads as a false pass. (2) SetupTestRole's DO-block CREATE ROLE is not concurrency-safe, so a bare `go test ./...` without `-p 1` could have both packages create the role simultaneously and fail with a duplicate-key error. `make test` and the task's verification command both pin `-p 1`, so impact is low, but the comment claiming the shared name is free of coupling understates it slightly.", + "benefit": "Keeps the mutation evidence honest for future rounds and documents the one invocation that could flake.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 4, + "criticalIssuesCount": 0, + "majorIssuesCount": 1, + "minorIssuesCount": 4, + "testCoverageAssessment": "excellent" + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-security-auditor-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-security-auditor-review-round2.json new file mode 100644 index 00000000..0f0de06e --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-security-auditor-review-round2.json @@ -0,0 +1,31 @@ +{ + "status": "approved", + "summary": { + "totalFindings": 2, + "critical": 0, + "major": 0, + "minor": 2 + }, + "findings": [ + { + "severity": "minor", + "category": "best-practice", + "title": "Role-creating helper with interpolated DDL is still compiled into the released pgcenter binary (carried over from round 1, unchanged)", + "description": "Round-1 finding 2, re-verified in round 2 and unchanged. postgres.SetupTestRole builds CREATE ROLE / GRANT pg_monitor / SET ROLE with fmt.Sprintf (internal/postgres/testing.go:73-91). The identifier cannot be a $1 placeholder, so interpolation is unavoidable; the file carries no build tag (its own doc comment at line 63-64 states this explicitly, as the reason it returns an error instead of taking *testing.T) and therefore ships in the released binary. The sole defense remains testRoleNameRE (`^[a-z_][a-z0-9_]*$`, line 12), pinned by Test_SetupTestRole_RejectsUnsafeName (internal/query/archiver_test.go:348-366), which asserts the name is refused before any statement is built (nil *postgres.DB suffices). Re-checked every call site in the module this round: internal/query/overview_test.go:193 (backlogRoleMonitor), :227 (backlogRoleNoRole), internal/stat/postgres_test.go:267 (backlogRoleCollect), internal/query/archiver_test.go:212, :250 (archiverRoleMonitor/NoRole), and the negative test at archiver_test.go:361. Every one passes an untyped string constant declared in the same file - no variable, argument, env value, table-driven fixture name or format string reaches the helper. The round-2 fix additionally moved the stat package off the query package's role name onto its own literal constant, so the constant-only invariant is preserved by the fix. Task 03 does not own internal/postgres/testing.go and is forbidden to modify it.", + "location": "internal/postgres/testing.go:58-94 (helper, owned by task 01); call sites this task owns: internal/query/overview_test.go:193,227 and internal/stat/postgres_test.go:267", + "impact": "A privilege-management primitive (role creation plus pg_monitor grant) is linked into production builds and callable by any future in-module caller. With the regex in place there is no injection path today; the residual risk is a later caller passing a non-constant name and relying on the regex alone, or the regex being relaxed.", + "recommendation": "Unchanged from round 1: move SetupTestRole behind a build tag or into a *_test.go / test-only package so it is not linked into the release binary. This is task 01's file - raise with the lead rather than patching from task 03.", + "cwe": "CWE-89" + }, + { + "severity": "minor", + "category": "A05: Security Misconfiguration", + "title": "Test roles and their pg_monitor grant remain permanent cluster state with no cleanup and no fixture-cluster assertion (carried over from round 1)", + "description": "Round-1 finding 3, re-verified and accepted as designed (Decision 18: no DROP ROLE, no REVOKE). The round-2 change adds a third permanent role rather than reusing an existing one: internal/stat now creates pgcenter_test_backlog_collect (granted pg_monitor) alongside internal/query's pgcenter_test_backlog_monitor / pgcenter_test_backlog_norole and the archiver task's pgcenter_test_archiver_monitor / pgcenter_test_archiver_norole. The de-sharing is the correct trade for the isolation it buys - a stray grant to one package's role can no longer silently weaken the other package's assertion, and the two packages no longer race on pg_authid via SetupTestRole's non-atomic DO block outside `go test -p 1` - but it does widen the standing residue by one pg_monitor grantee per cluster. The target is still chosen by postgres.NewTestConnectVersion, which hardcodes 127.0.0.1 and fixture ports 21910-21919/21994-21996 with user postgres / database pgcenter_fixtures (internal/postgres/testing.go:27-56), so the blast radius is normally a disposable CI container; there is still no assertion that the connected cluster is actually a throwaway fixture before DDL is issued. All roles are created NOLOGIN NOSUPERUSER, so the residue is not directly usable to authenticate.", + "location": "internal/stat/postgres_test.go:241,267 (new third role), internal/query/overview_test.go:193,227, internal/postgres/testing.go:68-94 (no drop/revoke)", + "impact": "Persistent monitoring-privileged catalog state on any cluster the suite is pointed at, outliving the test process and the checkout, now one role larger. On a fixture container this is accepted design; on a misdirected run against something else listening on those loopback ports it is an unnoticed standing grant.", + "recommendation": "Accepted as-is for this task. If tightened later, do it once in the shared helper rather than at the five call sites: refuse to issue any DDL unless the target matches a recognised fixture (port present in the NewTestConnectVersion map AND database = pgcenter_fixtures).", + "cwe": "CWE-732" + } + ] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-security-auditor-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-security-auditor-review.json new file mode 100644 index 00000000..e15f4436 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-security-auditor-review.json @@ -0,0 +1,41 @@ +{ + "status": "approved", + "summary": { + "totalFindings": 3, + "critical": 0, + "major": 0, + "minor": 3 + }, + "findings": [ + { + "severity": "minor", + "category": "A01: Broken Access Control", + "title": "Restricted-session guard in the internal/stat role test is weaker than the internal/query one", + "description": "Test_collectOverviewStat_PgMonitorRole re-implements the anti-vacuous guard inline (current_user, rolsuper, pg_has_role(current_user,'pg_monitor','USAGE')) instead of the internal/query equivalent assertRestrictedSession, which additionally asserts the role's membership set is EXACTLY {pg_monitor} (archiver_test.go:312-318). Roles are cluster-global, SetupTestRole never normalises an already-existing role, and Decision 18 mandates no DROP ROLE / no REVOKE. On a long-lived container a stray GRANT to pgcenter_test_backlog_monitor (e.g. pg_read_server_files, pg_read_all_settings, or the role mutation described in the task's Acceptance Criteria) silently decays the assertion from 'pg_monitor is sufficient' to 'some privileged role works' — the exact class of false-green that let the wrong pg_ls_dir privilege assumption survive into ADR [010]. Secondarily, the deferred RESET ROLE at internal/stat/postgres_test.go:267-270 only asserts the Exec returned no error; it does not verify current_user actually left the test role, unlike resetRole() in internal/query. Blast radius is currently bounded because the connection is closed immediately after, but the guard is strictly weaker than the one the same change ships in the neighbouring package.", + "location": "internal/stat/postgres_test.go:271-287 (guard), internal/stat/postgres_test.go:267-270 (reset)", + "impact": "A privilege regression could pass undetected: the end-to-end proof that a pg_monitor-only role gets a backlog number would still go green under a role that has accumulated grants beyond pg_monitor, so the change's core security claim (the field is readable with pg_monitor and nothing more) would no longer be pinned by this test.", + "recommendation": "Add the membership assertion to the inline guard so it matches the internal/query one, e.g. extend the probe with `coalesce((SELECT array_agg(DISTINCT r.rolname::text) FROM pg_auth_members m JOIN pg_roles r ON r.oid = m.roleid WHERE m.member = (SELECT oid FROM pg_roles WHERE rolname = current_user)), ARRAY[]::text[])` and `assert.Equal(t, []string{\"pg_monitor\"}, memberOf)`; and assert current_user != backlogRoleMonitor after RESET ROLE. Alternatively, promote assertRestrictedSession/resetRole into internal/postgres so both packages share one guard — but that edits a file this task is forbidden to touch, so raise it with the lead rather than doing it here.", + "cwe": "CWE-1071" + }, + { + "severity": "minor", + "category": "best-practice", + "title": "Role-creating helper with interpolated DDL is compiled into the released pgcenter binary", + "description": "postgres.SetupTestRole executes CREATE ROLE / GRANT pg_monitor / SET ROLE built with fmt.Sprintf (internal/postgres/testing.go:73-91). The identifier cannot be a $1 placeholder, so interpolation is unavoidable, and the file's own comment states it carries no build tag and therefore ships in the released binary. The only defense is testRoleNameRE (`^[a-z_][a-z0-9_]*$`), which is correct and is pinned by Test_SetupTestRole_RejectsUnsafeName. Task 03 does not introduce this — the file is owned by Task 01 and is read-only here — but Task 03 is its second consumer and cements the exported API. Verified for this task's call sites: internal/query/overview_test.go:18-21 and internal/stat/postgres_test.go:249 pass untyped string constants only, never a variable, argument, env value, or fixture-derived name; both roles are created NOLOGIN NOSUPERUSER; the deny role receives no GRANT at all.", + "location": "internal/postgres/testing.go:68-94 (helper, task 01), internal/query/overview_test.go:177,211 and internal/stat/postgres_test.go:262 (this task's call sites)", + "impact": "A privilege-management primitive (role creation plus pg_monitor grant) is present in production builds and reachable from any future in-module caller. With the regex in place there is no injection path today; the risk is that a later caller passes a non-constant name and relies on the regex alone, or that the regex is relaxed.", + "recommendation": "Move SetupTestRole behind a build tag or into a *_test.go / test-only package so it is not linked into the release binary. This belongs to Task 01's file and must not be patched from Task 03 — raise it with the lead per the task's own instruction ('raise it with the lead if the helper does not hold them — do not patch around it locally').", + "cwe": "CWE-89" + }, + { + "severity": "minor", + "category": "A05: Security Misconfiguration", + "title": "Test roles and their pg_monitor grant are permanent cluster state with no cleanup and no fixture-cluster assertion", + "description": "The two new tests create pgcenter_test_backlog_monitor (granted pg_monitor) and pgcenter_test_backlog_norole on every cluster the suite connects to, and by Decision 18 nothing ever drops or revokes them. The target is chosen by postgres.NewTestConnectVersion, which hardcodes 127.0.0.1 and fixture ports 21914-21919 with user postgres / db pgcenter_fixtures, so the blast radius is normally a disposable CI container. There is, however, no assertion that the connected cluster is actually a throwaway fixture; anyone who runs `go test ./internal/query/... ./internal/stat/...` while something else listens on those loopback ports leaves a persistent pg_monitor grantee behind. The task file itself documents the consequence ('a git checkout does not un-grant pg_monitor') and that a poisoned deny role makes a correct tree fail. The roles are NOLOGIN, so the residue is not directly usable to authenticate; it becomes usable only in combination with a later GRANT ... TO or SET ROLE by a privileged user.", + "location": "internal/query/overview_test.go:177,211 and internal/stat/postgres_test.go:262 (role creation), internal/postgres/testing.go:68-94 (no drop/revoke)", + "impact": "Persistent monitoring-privileged catalog state on any cluster the suite is pointed at, outliving the test process and the checkout. On a fixture container this is accepted design; on a misdirected run it is an unnoticed standing grant.", + "recommendation": "Accepted as-is for this task (Decision 18 is explicit and the ports are fixture-only). If tightened later, do it in the shared helper rather than at these call sites: refuse to run when the target is not a recognised fixture (e.g. assert the port is in the NewTestConnectVersion map and the database is pgcenter_fixtures) before issuing any DDL.", + "cwe": "CWE-732" + } + ] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-test-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-test-reviewer-review-round2.json new file mode 100644 index 00000000..c7bc08cb --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-test-reviewer-review-round2.json @@ -0,0 +1,55 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "passed", + "round": 2, + "previousReport": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-test-reviewer-review.json", + "summary": "All four applied fixes verified; the one declined fix is accepted on its merits. The major finding is closed: Test_ArchivingBacklogQuery_Structure (internal/query/overview_test.go:159-172) now pins the aggregate's arithmetic without a server, and I re-ran the three mutations myself on this host rather than taking the executor's word — dropping the .ready FILTER, dropping the pg_size_bytes(current_setting('wal_segment_size')) product, and reverting FROM to pg_ls_dir('pg_wal/archive_status') AS name each turn the test RED, the last one on two assertions at once (Contains pg_ls_archive_statusdir and NotContains pg_ls_dir). That closes the hole where every live assertion on the backlog reduced to 0 >= 0 on the archive_mode=off fixtures. Minor 1 is closed correctly: assert.True(t, got.Valid) is gone and the superuser baseline taken before SetupTestRole makes the 'rest of the sample' claim falsifiable — and it cannot collapse into a 0==0 tautology, because assert.GreaterOrEqual(got.DatabasesCount, 1) pins the compared value and Test_collectOverviewStat_Degradation:236 independently pins TotalSizeValid absolutely as superuser on the same cluster. Minor 2 is closed: the collect-level guard now scans the same DISTINCT membership array as assertRestrictedSession and gates on assert.Equal([]string{\"pg_monitor\"}, memberOf), and the RESET ROLE defer re-reads current_user. Minor 4 is closed by construction: internal/stat owns pgcenter_test_backlog_collect, internal/query owns pgcenter_test_backlog_monitor / _norole, so no cluster-global object is shared across packages and the SetupTestRole DO-block race is gone regardless of -p. Minor 3 (connectArchiverFixture) is accepted as declined — see roundOneDisposition for the reasoning. Verified independently: go vet clean, go build clean, gofmt clean on both changed files (the one gofmt hit, internal/stat/procpidstat_test.go, is unmodified pre-existing state and out of scope), and the working tree is byte-restored after my mutation runs. One coverage gap remains by design, not by omission: no test observes a NON-zero backlog, because the fixtures run archive_mode=off with an empty status directory — the task file defers that to the Task 10 stand run, and the structure test is the correct substitute at this layer.", + "findings": [], + "roundOneDisposition": [ + { + "roundOneSeverity": "major", + "category": "missing_coverage", + "disposition": "fixed", + "verification": "Test_ArchivingBacklogQuery_Structure added at internal/query/overview_test.go:159-172 with the four prescribed assertions. I re-ran all three mutations on this host, not just the one the executor reported. (a) count(*) FILTER (WHERE name LIKE '%.ready') -> count(*): RED, 'does not contain count(*) FILTER (WHERE name LIKE %.ready)'. (b) removing * pg_size_bytes(current_setting('wal_segment_size')) from the constant: RED, 'does not contain pg_size_bytes(current_setting(wal_segment_size))'. (c) FROM pg_ls_archive_statusdir() -> FROM pg_ls_dir('pg_wal/archive_status') AS name: RED on two assertions simultaneously. Note on method: the multiplication mutation must be anchored on the constant, since the same substring also appears in the doc comment above it — a comment-only edit leaves the test green, correctly, because assert.Contains reads the constant's value and not the file. Acceptance criterion 1 is now gated by a test that reproduces on a bare host with no CI image." + }, + { + "roundOneSeverity": "minor", + "category": "empty_test", + "disposition": "fixed", + "verification": "assert.True(t, got.Valid) is removed. internal/stat/postgres_test.go:265 captures the superuser baseline before SetupTestRole; :314-316 assert base.TotalSizeValid == got.TotalSizeValid and base.DatabasesCount == got.DatabasesCount. I checked the fix for the obvious way a baseline parity check goes vacuous — both sides degrading to the same wrong value — and it is covered from two directions: assert.GreaterOrEqual(got.DatabasesCount, int64(1)) forbids the 0==0 case, and Test_collectOverviewStat_Degradation:236 already asserts TotalSizeValid absolutely as the fixture superuser on the same cluster, so a suite-wide break of the size aggregate reddens there rather than sliding through here as false parity." + }, + { + "roundOneSeverity": "minor", + "category": "anti_pattern", + "disposition": "fixed", + "verification": "internal/stat/postgres_test.go:288-306 now selects the fourth column verbatim from assertRestrictedSession, including the DISTINCT that PG 16+ per-grantor rows require, scans it into memberOf, and adds okMembers = assert.Equal([]string{\"pg_monitor\"}, memberOf) to the early-return condition — so the test can no longer decay from 'pg_monitor is sufficient' to 'some privileged role works'. The defer at :272-280 follows RESET ROLE with a current_user re-read and assert.NotEqual, matching resetRole. The remaining duplication of assertRestrictedSession's body across the two packages is forced (it lives in package query; internal/postgres/testing.go is read-only in this task) and was noted as forced in round 1." + }, + { + "roundOneSeverity": "minor", + "category": "anti_pattern", + "disposition": "fixed", + "verification": "internal/stat/postgres_test.go:243 defines its own const backlogRoleCollect = \"pgcenter_test_backlog_collect\"; internal/query/overview_test.go:20-21 keeps pgcenter_test_backlog_monitor / pgcenter_test_backlog_norole. The two packages no longer touch a common role, so the non-atomic NOT EXISTS / CREATE ROLE window in SetupTestRole cannot be entered concurrently by them, and the tests are now independent of whether the invocation passes -p 1. This is option (a) from the round-1 recommendation, the one I preferred: it fixes the property rather than constraining how the suite may be invoked." + }, + { + "roundOneSeverity": "minor", + "category": "anti_pattern", + "disposition": "accepted_as_declined", + "verification": "The pushback holds on all three of its claims and I verified each. (1) The task file's Implementation Hints prescribe this shape explicitly, at 017-feat-wal-archiver-task-03.md:332-333: 'Follow the existing live-PG test shape in this file: postgres.NewTestConnectVersion(version) with t.Skipf when the cluster is unavailable, and conn.Close() per iteration.' Re-raising the finding would ask the executor to contradict its own task file. (2) It is the file's prevailing convention, including the pre-existing Test_ArchivingBacklogQuery_Degrades, so the concern is file-wide rather than something these tests introduced. (3) internal/query/archiver_test.go is not among task 03's Context Files, and adopting the helper without the rename would leave an archiver-named function connecting overview fixtures. The impact I flagged is latent, and I confirmed it is latent today: all six entries of overviewVersions (140000-190000) are present in the NewTestConnectVersion port map at internal/postgres/testing.go:28-44, so no subtest can currently skip for the wrong reason. Follow-up for a later task, not a blocker for this one: rename connectArchiverFixture to connectFixture and route every live-PG test in package query through it, so that adding a version to a version list without adding it to the port map fails loudly instead of skipping silently." + } + ], + "metrics": { + "filesReviewed": 4, + "litmusTest": { + "checked": 14, + "passed": 14, + "failed": 0 + }, + "coverageAssessment": "excellent", + "pyramidBalance": { + "unit": 1, + "integration": 3, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-test-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-test-reviewer-review.json new file mode 100644 index 00000000..57471e4f --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03-dev-test-reviewer-review.json @@ -0,0 +1,62 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "needs_improvement", + "summary": "The three new tests are non-vacuous where it matters most: all three gate on a restricted-session guard that runs BEFORE the aggregate and returns early, so a forgotten SET ROLE reddens on the guard rather than passing as the fixture superuser. The collected mutation evidence (1-4) is consistent with the code as written and covers the privilege claim in both directions at query level and end-to-end at collect level. The gap is elsewhere: nothing in the tree falsifies the query's actual arithmetic. On the fixtures (archive_mode=off, empty archive_status) every live assertion on the backlog reduces to 0 >= 0, so removing the '.ready' FILTER or the pg_size_bytes multiplication keeps all three new tests and both pre-existing tests green — the very reason task 01 added a serverless Test_StatArchiverQuery_Structure for its twin query. Four smaller issues: one tautological assertion (got.Valid is set unconditionally), a collect-level guard weaker than its query-level counterpart on a role both packages share, skip-on-any-connect-error where the same package already has a helper that distinguishes a missing port map from an unavailable cluster, and a latent cross-package role race outside -p 1.", + "findings": [ + { + "severity": "major", + "category": "missing_coverage", + "location": "internal/query/overview_test.go:159-233 (both new tests) and internal/stat/postgres_test.go:249-296", + "issue": "No test — new or pre-existing — falsifies the only logic in OverviewArchivingBacklog. The fixtures run archive_mode=off with an empty archive_status directory, so the aggregate returns exactly 0 on every cluster; `assert.GreaterOrEqual(backlog, int64(0))` and `assert.GreaterOrEqual(got.ArchivingBacklog, int64(0))` therefore hold for a bare `count(*)`, for a wider LIKE pattern, and for a query with the `* pg_size_bytes(current_setting('wal_segment_size'))` factor deleted. Acceptance criterion 1 ('the SELECT list is unchanged and the query still yields one non-NULL bigint in bytes') is consequently gated by nothing. Litmus: delete `FILTER (WHERE name LIKE '%.ready')` from internal/query/overview.go:106 → Test_ArchivingBacklogQuery_PgMonitorRole, Test_ArchivingBacklogQuery_NoPrivilegeRole, Test_ArchivingBacklogQuery_Degrades and Test_collectOverviewStat_PgMonitorRole all stay GREEN. The same is true for deleting the pg_size_bytes product. This is precisely the hole task 01 identified for its twin query and closed with a serverless structure test (internal/query/archiver_test.go:73-104, whose comment states the reason verbatim: 'the fixtures have an empty archive status directory, so count(*) FILTER (...) and a bare count(*) are both 0 on every cluster and no live assertion can tell them apart').", + "recommendation": "Add a no-server test next to the two new ones, mirroring Test_StatArchiverQuery_Structure: func Test_ArchivingBacklogQuery_Structure(t *testing.T) with (a) assert.Contains(t, OverviewArchivingBacklog, \"count(*) FILTER (WHERE name LIKE '%.ready')\", \"only .ready files are backlog - a bare count(*) is a different number\"); (b) assert.Contains(t, OverviewArchivingBacklog, \"pg_size_bytes(current_setting('wal_segment_size'))\", \"the backlog is bytes, not a segment count\"); (c) assert.Contains(t, OverviewArchivingBacklog, \"FROM pg_ls_archive_statusdir()\", \"the pg_monitor-granted function is the whole point of Decision 8\"); (d) assert.NotContains(t, OverviewArchivingBacklog, \"pg_ls_dir\", \"the superuser-only predecessor must not come back\"). Assertion (d) also turns AC mutation 1 (revert the FROM clause) into a failure that reproduces on a bare host, not only in the CI image. No mock changes — the test touches no connection.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "empty_test", + "location": "internal/stat/postgres_test.go:292 (assert.True(t, got.Valid) in Test_collectOverviewStat_PgMonitorRole)", + "issue": "`got.Valid` is set unconditionally at internal/stat/postgres.go:202 (`s.Valid = true`) and collectOverviewStat has no early return — it always falls through to `return s, sizeLatency` at :304. The assertion therefore cannot fail for any state of the cluster, the role, or the query, yet it is one of the two assertions carrying the test's stated claim that 'the rest of the sample must be unaffected by running under a restricted role'. Only the neighbouring `DatabasesCount >= 1` actually proves anything about the restricted role. The TDD anchor named `Valid` explicitly, so the vacuity originates in the task file, not in the executor's judgement.", + "recommendation": "Replace the tautology with a falsifiable parity check against a superuser baseline. Before SetupTestRole, capture `base, _ := collectOverviewStat(conn, props, 1, PgstatOverview{}, false)`; after the guard, collect as `got` and assert: assert.Equal(t, base.TotalSizeValid, got.TotalSizeValid, \"the db-size aggregate must not degrade under pg_monitor\"); assert.Equal(t, base.DatabasesCount, got.DatabasesCount, \"the databases aggregate must see the same clusters under pg_monitor\"); keep assert.GreaterOrEqual(t, got.DatabasesCount, int64(1)). Drop assert.True(t, got.Valid). This makes the 'rest of the sample' claim reddenable — a privilege regression in any neighbouring aggregate now diverges from the baseline instead of being absorbed by an availability flag nobody asserts.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/stat/postgres_test.go:265-287 (inline guard in Test_collectOverviewStat_PgMonitorRole)", + "issue": "The collect-level guard is a weakened re-implementation of internal/query/archiver_test.go:285-321 assertRestrictedSession: it checks current_user, rolsuper and pg_has_role(...,'pg_monitor'), but omits the role-membership assertion ('pg_monitor and nothing else'). The omission matters more here than at query level, not less: the two packages deliberately share ONE cluster-global role (pgcenter_test_backlog_monitor, internal/stat/postgres_test.go:243 and internal/query/overview_test.go:21) that SetupTestRole never normalises once it exists and nothing ever drops. After any stray GRANT to that role — including a hand-run AC mutation — this test silently decays from 'pg_monitor is sufficient' to 'some privileged role works', which is the exact class of false confidence that put the wrong privilege claim into ADR [010]. The RESET ROLE defer at :277-280 is likewise thinner than query's resetRole: it asserts the Exec returned no error but not that current_user actually left the role. (The duplication itself is forced — assertRestrictedSession lives in package query and internal/postgres/testing.go is read-only in this task — so the fix is to close the gap, not to deduplicate.)", + "recommendation": "Extend the inline SELECT with the membership column from assertRestrictedSession and gate on it: add `coalesce((SELECT array_agg(DISTINCT r.rolname::text ORDER BY r.rolname::text) FROM pg_auth_members m JOIN pg_roles r ON r.oid = m.roleid WHERE m.member = (SELECT oid FROM pg_roles WHERE rolname = current_user)), ARRAY[]::text[])` as a fourth column, scan into `memberOf []string`, and add `okMembers := assert.Equal(t, []string{\"pg_monitor\"}, memberOf, \"the test role must hold pg_monitor and nothing else\")` to the existing `if !okUser || !okSuper || !okMonitor` early-return condition. In the RESET ROLE defer, follow the Exec with `var u string; if assert.NoError(t, conn.QueryRow(\"SELECT current_user::text\").Scan(&u)) { assert.NotEqual(t, backlogRoleMonitor, u) }`, matching resetRole.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/query/overview_test.go:162-166 and :196-200 (connect + t.Skipf in both new tests)", + "issue": "Both new tests skip on ANY error from postgres.NewTestConnectVersion, including 'postgres version %d has no test cluster port mapping' — the failure mode patterns.md documents as having previously made subtests named after a new version pass while exercising nothing at all. The same package already solved this: connectArchiverFixture (internal/query/archiver_test.go:371-382) asserts the error is not a missing port-map entry before skipping. Since archiver_test.go is `package query`, that helper is directly callable from overview_test.go and was not used. Impact today is latent (all six overviewVersions are in the map), but the moment a version is added to overviewVersions and not to the ports map, both privilege tests go green while proving nothing — the same silent-pass shape the anti-vacuous guards exist to prevent one layer down.", + "recommendation": "Replace `conn, err := postgres.NewTestConnectVersion(version); if err != nil { t.Skipf(...) }` in both new tests with `conn := connectArchiverFixture(t, version)`, keeping `defer conn.Close()`. If the helper's archiver-specific name is undesirable in this file, rename it to `connectFixture` at its definition site (internal/query/archiver_test.go) and update its two existing call sites — it is package-scoped test code, not a task-01-owned production file.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/stat/postgres_test.go:243 (const backlogRoleMonitor) vs internal/query/overview_test.go:21", + "issue": "The two packages share one cluster-global role name by design, and both reach the SAME PG 17 cluster (postgres.NewTestConnect() == NewTestConnectVersion(170000), and overviewVersions contains 170000). SetupTestRole's idempotent creation is a DO block guarded on pg_roles, which is not atomic: two concurrent executions can both observe NOT EXISTS and both issue CREATE ROLE, so one loses on the pg_authid unique index and SetupTestRole returns an error, reddening a correct tree. The sanctioned commands serialise packages (`-p 1` in the Makefile test target and in the task's Verification Steps docker command), but the task frontmatter's own verify line — 'go test ./internal/query/... ./internal/stat/...' — does not, and neither does an ad-hoc `go test ./internal/...`. This is a flaky-test risk introduced by the cross-package name sharing, whose stated benefit (one role instead of two on the fixture clusters) is cosmetic.", + "recommendation": "Pick one: (a) give the stat package its own role name, e.g. `pgcenter_test_backlog_collect`, removing the shared-object race entirely at the cost of one extra NOLOGIN role per cluster; or (b) keep the shared name and add `-p 1` to the frontmatter verify line so every documented invocation serialises the two packages. Option (a) is preferable — it makes the test independent of how it is invoked, which is the property the sharing trades away.", + "litmusTestFailed": false + } + ], + "metrics": { + "filesReviewed": 4, + "litmusTest": { + "checked": 12, + "passed": 9, + "failed": 3 + }, + "coverageAssessment": "adequate", + "pyramidBalance": { + "unit": 0, + "integration": 3, + "e2e": 0, + "assessment": "unbalanced" + } + } +} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-03.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03.md similarity index 89% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-03.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03.md index efe8dc8a..390e2bc8 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-03.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-03.md @@ -1,5 +1,5 @@ --- -status: planned # planned -> in_progress -> done +status: done # planned -> in_progress -> done depends_on: ["01"] # ID задач-зависимостей (строки: ["01", "02"]) wave: 2 # волна параллельного выполнения skills: [code-writing] # МАССИВ скиллов для загрузки @@ -115,16 +115,22 @@ Green alone is not evidence (patterns.md, "Extract the decision out of the unrea - [ ] **Mutation:** revert the `FROM` clause to `pg_ls_dir('pg_wal/archive_status') AS name` → `Test_ArchivingBacklogQuery_PgMonitorRole` and `Test_collectOverviewStat_PgMonitorRole` must go RED. If they stay green, the tests are not running under the restricted role. -- [ ] **Mutation:** delete the `SET ROLE` statement from `Test_ArchivingBacklogQuery_PgMonitorRole` - (leaving the session as the fixture superuser) → the test must go RED on its own - `current_user` / `rolsuper` guard. A test that would still pass here is a vacuous gate. -- [ ] **Mutation (role-state-sensitive — read the note below):** remove the `GRANT pg_monitor` from - the role setup → `Test_ArchivingBacklogQuery_PgMonitorRole` must go RED with a permission error. - Counts only when run against a cluster where the role does not already hold `pg_monitor`. -- [ ] **Mutation (role-state-sensitive — read the note below, and it poisons the cluster):** add - `GRANT pg_monitor` to the deny role in `Test_ArchivingBacklogQuery_NoPrivilegeRole` → that test - must go RED (it asserts a permission error). Counts only on a cluster where the deny role does - not already hold the grant, and the cluster must be discarded afterwards. +> **All three mutations below act on THIS task's call sites, never on the helper.** `SET ROLE` and the +> grant now live inside `postgres.SetupTestRole`, which Task 01 owns and this task must not edit — so +> a mutation phrased as "delete the SET ROLE line" or "remove the GRANT" would send the executor into +> a file its own acceptance criteria forbid touching. Mutate the call instead. + +- [ ] **Mutation:** in `Test_ArchivingBacklogQuery_PgMonitorRole`, skip the `SetupTestRole` call and + run the query on the plain fixture connection (session stays the superuser) → the test must go + RED on its own `current_user` / `rolsuper` guard. A test that still passes here is a vacuous gate. +- [ ] **Mutation (role-state-sensitive — read the note below):** flip the call site to + `SetupTestRole(db, , false)` so the role is created without `pg_monitor` → + `Test_ArchivingBacklogQuery_PgMonitorRole` must go RED with a permission error. Counts only on a + cluster where that role does not already hold `pg_monitor` from an earlier run. +- [ ] **Mutation (role-state-sensitive — read the note below, and it poisons the cluster):** flip the + deny call site to `SetupTestRole(db, , true)` → `…_NoPrivilegeRole` must go RED (it + asserts a permission error). Counts only on a cluster where the deny role does not already hold + the grant, and the cluster must be discarded afterwards. - [ ] All role setup goes through the shared helper in `internal/postgres/testing.go` (Decision 19): this task adds **no** role helper of its own and no inline `CREATE ROLE`/`GRANT` SQL in either package, and `internal/postgres/testing.go` is not modified here. Role creation is therefore @@ -138,9 +144,12 @@ Green alone is not evidence (patterns.md, "Extract the decision out of the unrea claims the fixtures role holds `pg_monitor`. - [ ] `collectOverviewStat`'s error-swallow + `ArchivingBacklogValid` degradation path is byte-identical apart from its comment; `top/stat.go` is not touched. -- [ ] A cluster whose `archive_status` directory is absent now reports `0 B` instead of `n/a` - (`pg_ls_archive_statusdir()` is `missing_ok=true`). This is accepted by Decision 11 — it must - **not** be "fixed", worked around, or guarded against in this task. +- [ ] A cluster whose `archive_status` directory is absent now reports a zero backlog instead of + `n/a` (`pg_ls_archive_statusdir()` is `missing_ok=true`). Note the rendering: the panel prints a + bare `0`, not `0 B` — a check written against the string `0 B` would report a false failure. + This is accepted by Decision 11 and must **not** be "fixed", worked around, or guarded against + here. Not gated by a test: reproducing it means moving the directory aside on a live cluster, + which belongs to the stand run (Task 10), not to this task. - [ ] `go test ./internal/query/... ./internal/stat/...` passes inside the CI image; `make lint` and `make vuln` are clean on the host. (Those two packages are not host-runnable — on a machine without the fixture clusters `./internal/stat/...` panics rather than skipping.) @@ -213,8 +222,12 @@ tests that touch no cluster; `make lint` and `make vuln` are the genuinely host- comment mutations first; the two **role** mutations last, each on clean role state (dropped role or fresh containers), then rebuild the containers — see the note in Acceptance Criteria for why a reverted role mutation does not restore the cluster. -- Grep the tree for the stale claims — no hit may remain: - `grep -rn "pg_ls_dir" internal/` and `grep -rn "has pg_monitor" internal/`. +- Grep the tree for the stale CLAIMS, not for the string itself. `pg_ls_dir` legitimately survives in + the new doc comment that explains what was wrong with it, so a zero-hit rule is unsatisfiable — the + same defect that was fixed in task 04's grep-count criterion. Check instead that no remaining + occurrence still asserts the old privilege requirement: + `grep -rn "pg_ls_dir" internal/` must show only the explanatory comment, and + `grep -rn "has pg_monitor" internal/` must return nothing. - `make lint` and `make vuln` on the host. ## Details @@ -255,7 +268,10 @@ and `internal/stat`. This task defines **no** helper of its own — not in `inte `internal/stat`, not "overview-specific", not a copy under a different name. Read `internal/postgres/testing.go` for the helper's actual signature and call it. If it is not there when you start, the dependency has not landed and this task is not startable yet — say so rather than -writing a second helper. The SQL role names come from the helper too; do not invent parallel ones. +writing a second helper. **Role names are this task's to choose**, since the helper takes the name as +an argument — pick names distinct from task 01's (`pgcenter_test_archiver_*`), e.g. +`pgcenter_test_backlog_monitor` / `pgcenter_test_backlog_norole`, following the existing +`pgcenter_test_*` convention. Sharing task 01's roles would couple the two tasks' cluster state. **Edge cases:** diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-code-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-code-reviewer-review-round2.json new file mode 100644 index 00000000..0c90ea0f --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-code-reviewer-review-round2.json @@ -0,0 +1,44 @@ +{ + "reviewer": "dev-code-reviewer", + "round": 2, + "status": "approved", + "summary": "Round-1 finding 1 (assert.NotNil -> require.NotNil) is applied correctly. Round-1 finding 2 was declined, and the decline is right: the implemented alternative — pinning NoOptDefVal directly in Test_walFlagDefinition — is strictly stronger than what I proposed, because a Lookup-derived mirror built with fs.StringP() always carries NoOptDefVal == \"\" regardless of the real flag and would therefore have hidden exactly the cobra shim Decision 7 rejects. cmd/report/report.go is byte-identical to round 1; all round-2 changes are in the test file. Seven mutations were run against the new assertions on a scratch copy and every one turned the suite red, each killed by precisely the test that claims to guard that property. No findings.", + "criticalIssues": [], + "suggestions": [], + "metrics": { + "filesReviewed": 2, + "criticalIssuesCount": 0, + "majorIssuesCount": 0, + "minorIssuesCount": 0, + "testCoverageAssessment": "excellent" + }, + "round1FindingsDisposition": [ + { + "finding": "Test_walFlagDefinition dereferenced f after assert.NotNil, so a renamed flag panicked and took the whole package binary down.", + "disposition": "applied", + "verdict": "resolved", + "evidence": "cmd/report/report_test.go:151 is now require.NotNil(t, f), with a comment at :149-150 stating the reason (stop this test rather than panic and take the rest of the package down). testify/require is imported at :7 and used nowhere else in the file, so the import is justified and minimal." + }, + { + "finding": "Build the pflag mirror in Test_walFlagPflagFailureShapes from CommandDefinition.Flags().Lookup(\"wal\") fields instead of hardcoded literals.", + "disposition": "declined", + "verdict": "decline accepted — the reasoning is correct and my round-1 suggestion was the weaker option", + "evidence": "Three independent reasons, checked rather than assumed. (1) The suggestion would not have improved fidelity where it matters: fs.StringP(f.Name, f.Shorthand, f.DefValue, f.Usage) sets NoOptDefVal to \"\" unconditionally, so a derived mirror reproduces an empty NoOptDefVal even when the real flag has a shim installed — Parse([]string{\"-W\"}) would keep returning 'flag needs an argument' in the test while the real CLI silently accepted bare -W. The derived mirror would have been a lie precisely in the dimension the task cares about. (2) The direct assertion that replaced it is load-bearing: mutation A below installs NoOptDefVal = \"w\" on the real flag and Test_walFlagDefinition goes red — the derived-mirror variant would not have caught it. (3) The task file prescribes the hardcoded mirror explicitly (task-04.md:246-249, 'build a local pflag.FlagSet mirroring the definitions instead, and let Test_walFlagDefinition guard that the mirror still matches reality'), and the independence argument stands on its own: two tests hardcoding the same literal fail separately, whereas a derived mirror cannot contradict the definition it is derived from. I withdraw the suggestion; it should not be re-raised." + } + ], + "verifiedProperties": [ + "Production code unchanged since round 1. git diff -- cmd/report/report.go is identical to the round-1 diff (three edits: struct field :25 bool->string, flag definition :70 BoolVarP->StringVarP, selectReport arm :151-160 guarded outer case + inner switch with no default). The round-1 verification of the closed-whitelist mechanism therefore still holds and was not re-derived from scratch, only re-confirmed by mutation.", + "Mutation A (the rejected Decision 7 shim): CommandDefinition.Flags().Lookup(\"wal\").NoOptDefVal = \"w\" appended to init() -> Test_walFlagDefinition FAILS. This is the new assertion at :159 and it is genuinely load-bearing; before this round no test in the package reacted to that mutation.", + "The behavioural claim in the new comment at :156-158 is accurate, verified by driving pflag directly rather than by reading: with NoOptDefVal = \"w\", Parse([\"-W\"]) returns nil with value \"w\" (bare -W keeps working) and Parse([\"-W\", \"a\"]) returns nil with value \"w\" and \"a\" pushed into positional Args (the 'a' is silently dropped and the user gets the wal report). The comment describes the real failure mode, not a plausible-sounding one.", + "Mutation B (arm hoisted above showFunctions) -> Test_selectReport_WALPrecedence FAILS. Mutation C (arm hoisted above showDatabases) -> FAILS. Mutation D (arm sunk below showBgwriter) -> FAILS. The 4-row table pins both neighbours as it claims; under the round-1 single-row version, mutations B and D both survived, so the widening closed a real hole rather than adding decoration.", + "Mutation E (switch strings.TrimSpace(opts.showWAL)) -> Test_selectReport_WALWhitelistIsClosed FAILS on the new 'w ' row. Trimming is a normalisation Decision 17 prohibits, and it was the one prohibited normalisation with no test behind it in round 1 (case-folding was already covered by the 'W' row). Mutation G (strings.ToLower, the round-1 mutation) re-run and still killed.", + "Mutation F (a stray alias between selectReport and report.Config: 'if r == \"archiver\" { r = \"wal\" }' inserted in validate() at :93) -> Test_selectReport_WALWhitelistIsClosed FAILS while Test_selectReport stays GREEN. This is the exact scenario the new end-to-end assertions at :115-121 were added for, and the split result proves they cover ground the selectReport-level table cannot reach: ReportType is what keys the view map and filters tar entries, so a rewrite in validate() is the same silently-wrong-report failure the whitelist exists to prevent.", + "The corrected comment at :182-183 is accurate: Parse([\"-W\", \"-f\", \"dump.tar\"]) yields value \"-f\" and Args [\"dump.tar\"] whether or not -f is defined in the FlagSet — confirmed by running both variants. A string flag consumes the next token unconditionally, so the outcome genuinely does not depend on -f existing, and the comment now says so instead of implying otherwise.", + "The mirror-honesty chain is now complete. Every property of the mirror that can affect parsing — name, shorthand, value type, default, and NoOptDefVal — is asserted against the real flag in Test_walFlagDefinition (:152-159). Usage is also pinned there; it cannot affect parsing, so its duplication in the mirror is inert rather than misleading.", + "Tests are order-independent and state-free: go test ./cmd/report/... -race -shuffle=1|2|3 all pass. The new validate() calls at :108-121 operate on options literals, and validate/selectReport/setReportInterval/parseFilterString are pure with value receivers, so nothing touches the package-level opts var that CommandDefinition's flags are bound to. The task's order-dependence prohibition (task-04.md:246-249) is honoured by the added end-to-end assertions too, not just by the pflag test.", + "Containment unchanged: grep -rn showWAL --include=*.go . hits only cmd/report/report.go (4) and cmd/report/report_test.go (12). No production file outside cmd/report mentions it.", + "Gates: gofmt -l cmd/report empty; go vet ./cmd/report/... clean; go test ./cmd/report/... -v green with all six -W-related test functions named in the output. make lint could not be run — golangci-lint is not installed in this environment (make lint dies with 'golangci-lint: No such file or directory'), which is an environment gap, not a property of this change; the code introduces no construct golangci-lint's default set would flag.", + "All mutation work was done on a full copy of the tree at scratchpad/r2. The repository working tree was not modified: git diff -- cmd/report/ is unchanged and the parallel edits in internal/query/ and internal/postgres/ from other agents were left untouched.", + "Out of scope per the task and deliberately not reported: -W a maps to \"archiver\" while the archiver view (Task 5) and the describeReport entry (Task 7) land in later waves. No temporary alias or guard is wanted here — task-04.md:211-216 rules one out explicitly, and adding one would be the silently-wrong-report failure Decision 17 exists to prevent." + ] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-code-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-code-reviewer-review.json new file mode 100644 index 00000000..42de53e0 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-code-reviewer-review.json @@ -0,0 +1,47 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "The -W bool -> string conversion is implemented exactly as Decision 7/17 prescribe: three surgical edits in cmd/report/report.go, the inner switch copied verbatim from the -J arm with no default, and selectReport remains the single producer of ReportType, gated by validate()'s r == \"\" check. The closed-whitelist property was verified empirically, not by reading: seven mutations (default: return \"wal\", unconditional outer return, swapped a->wal, strings.ToLower normalisation, widened case \"w\",\"W\", help-text char edit, arm moved above showActivity) were each applied to a scratch copy of the tree and each turned the suite red. Only two minor test-robustness nits remain; nothing blocks.", + "criticalIssues": [], + "suggestions": [ + { + "file": "cmd/report/report_test.go", + "line": 121, + "severity": "minor", + "category": "best-practices", + "suggestion": "Test_walFlagDefinition uses assert.NotNil(t, f) and then dereferences f on the next line. assert only records the failure and continues, so if Lookup(\"wal\") ever returns nil the test panics with a nil pointer dereference at f.Value.Type() instead of failing cleanly. Verified by renaming the flag to \"walx\" in a scratch copy: the run ends in 'panic: runtime error: invalid memory address or nil pointer dereference' and the panic takes down the whole test binary, so the other nine tests in the package never report. Use require.NotNil(t, f) (testify/require is already used in this repo, e.g. report/report_test.go:1249,1362) so the failure is a clean, localised FAIL.", + "benefit": "A renamed or dropped -W flag produces one readable failure naming the flag, instead of a package-wide binary crash whose stack trace hides which assertion tripped.", + "optional": true + }, + { + "file": "cmd/report/report_test.go", + "line": 137, + "severity": "minor", + "category": "maintainability", + "suggestion": "The mirror FlagSet in Test_walFlagPflagFailureShapes re-types the -W definition as literals (\"wal\", \"W\", \"\", and the full usage string), and the comment says Test_walFlagDefinition is what keeps the mirror honest — which is true only indirectly, since both tests happen to hardcode the same literals. The mirror can be made structurally honest without parsing through CommandDefinition (the constraint the task rightly imposes): f := CommandDefinition.Flags().Lookup(\"wal\"); wal := fs.StringP(f.Name, f.Shorthand, f.DefValue, f.Usage). Lookup is read-only and does not touch the package-level opts var, so the order-independence property the comment protects is preserved.", + "benefit": "Removes a duplicated user-visible literal and makes the mirror track the real flag by construction rather than by two tests agreeing on the same copy-pasted string.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 2, + "criticalIssuesCount": 0, + "majorIssuesCount": 0, + "minorIssuesCount": 2, + "testCoverageAssessment": "excellent" + }, + "verifiedProperties": [ + "Closed whitelist is genuinely enforced. selectReport (cmd/report/report.go:132-211) is the only producer of the r value passed to report.Config.ReportType, and validate() (:93-96) rejects r == \"\" before the struct at :117 is built. grep for 'report.Config{' outside the report package returns only cmd/report/report.go's three zero-value error returns plus that one populated literal, so no unmapped -W value can reach ReportType by any path.", + "Go semantics of the nested switch are correct: an unmatched inner switch has no implicit fallthrough and no default, so control leaves the outer switch entirely and reaches the final 'return \"\"' at :210. Confirmed by running every value in the whitelist table.", + "Mutation 1 (default: return \"wal\" inside the inner switch) -> Test_options_validate, Test_selectReport_WALWhitelistIsClosed and Test_walFlagPflagFailureShapes all FAIL. Mutation 1b (outer arm replaced by an unconditional return \"wal\") -> same three plus Test_selectReport FAIL.", + "Mutation 2 (case \"a\" returns \"wal\") -> Test_selectReport FAILs. Mutation 3 (one character changed in the help text: 'archiver' -> 'archivers') -> Test_walFlagDefinition FAILs.", + "Extra mutations beyond the acceptance criteria, all killed: switch strings.ToLower(opts.showWAL) -> WALWhitelistIsClosed FAILs on the 'W' row; case \"w\", \"W\" -> same; hoisting a 'case opts.showWAL == \"a\": return \"archiver\"' arm above showActivity -> Test_selectReport_WALPrecedence FAILs. The normalisation and precedence prohibitions in Decision 17 are therefore load-bearing tests, not decorative ones.", + "All eight whitelist rows required by the acceptance criteria are present and each carries a why-string explaining the danger, not just the invalidity: c, t, g, W, wal, archiver, x, -f (cmd/report/report_test.go:89-96).", + "Flag precedence unchanged: the showWAL arm sits between showFunctions (:149) and showBgwriter (:161), exactly where the boolean arm was.", + "Containment: grep -rn showWAL --include=*.go . hits only cmd/report/report.go (4) and cmd/report/report_test.go (7). No production file outside cmd/report mentions it; nothing else consumes options.", + "Style conformance: the arm is a byte-for-byte shape match with the showStatIO arm at :165-171 (guarded outer case, inner switch, no default), which is the precedent the task names. gofmt -l cmd/report is empty, go vet ./cmd/report/... is clean, go test ./cmd/report/... passes with all six -W-related test functions named in -v output.", + "Comment quality: the four-line comment at :152-154 explains why the default arm is absent (the zero-value view / silently empty report failure mode) rather than restating what the code does — this is the 'answer a linter with a comment, not a default' instruction from the task, honoured.", + "Behavioural side effect checked and found acceptable: because the outer case matches on showWAL != \"\", an invalid combination such as -W x -B now yields \"\" rather than falling through to \"bgwriter\". This is identical to the pre-existing behaviour of the -D, -J, -X and -P arms and is the fail-closed outcome Decision 17 wants; it is not a regression.", + "Out of scope per the task and deliberately not reported as defects: the archiver view (Task 5) and its describeReport entry (Task 7) do not exist yet, so -W a maps correctly but has nothing behind it inside the feature branch." + ] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-security-auditor-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-security-auditor-review.json new file mode 100644 index 00000000..2e7d88dd --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-security-auditor-review.json @@ -0,0 +1,51 @@ +{ + "status": "approved", + "summary": { + "totalFindings": 4, + "critical": 0, + "major": 1, + "minor": 3 + }, + "findings": [ + { + "severity": "major", + "category": "A04:2021 Insecure Design (fail-closed validation is not observable to callers)", + "title": "Rejected -W value produces exit code 0 — the whitelist fails closed but silently for automation", + "description": "selectReport correctly maps only 'w' and 'a', and validate() rejects everything else with 'report type is not specified, quit' (cmd/report/report.go:94-96). Verified end-to-end on a built binary: `pgcenter report -W -f dump.tar` prints the error and exits **0**. Root cause is outside the audited files — main() in cmd/pgcenter.go prints the error and returns without os.Exit(1) — and it is pre-existing and repo-wide (`-J q` behaves identically). This change is what makes it consequential for -W: the task's own Details and Edge cases state that on the legacy `-W -f dump.tar` shape 'the command exits non-zero', and Decision 17's entire safety argument rests on the failure being loud. It is loud on stderr/stdout only, not to the caller. A legacy cron/monitoring wrapper of the form `pgcenter report -W -f dump.tar > wal-report.txt || alert` previously produced a WAL report; after this breaking change it produces an empty file and a success status, and the `|| alert` never fires. Neither the new tests nor the acceptance criteria assert an exit code, so nothing in the task would catch this.", + "location": "cmd/report/report.go:94-96 (rejection path introduced/relied on by this change); root cause cmd/pgcenter.go:66-70 (outside audit scope)", + "impact": "Automation that invoked the old boolean -W keeps reporting success while producing no data. The breaking change becomes invisible to exactly the consumers — scripts, cron jobs, CI report collectors — that cannot read the printed message. For a troubleshooting/report tool this converts a loud CLI break into a silent gap in captured incident data.", + "recommendation": "Do not add a compensating branch in selectReport (the task explicitly forbids it and it would re-open Decision 17). Two options, in order of preference: (1) fix the root cause in a separate task/feature — `func main() { if err := pgcenter.Execute(); err != nil { fmt.Println(err); os.Exit(1) } }`; this makes every subcommand's validation observable, not just -W. (2) At minimum, correct the claim in Task 04 Details/Edge cases and in the Task 9 release notes: state that the command prints 'report type is not specified, quit' and exits 0, so operators know an exit-status check will not catch the migration. If (1) is taken, add an exit-code assertion to the acceptance criteria so it cannot regress.", + "cwe": "CWE-390 (Detection of Error Condition Without Action)" + }, + { + "severity": "minor", + "category": "A04:2021 Insecure Design (release gate, intra-branch state)", + "title": "-W a currently renders a silently empty report with exit 0 until Tasks 5 and 7 land", + "description": "Verified against report/testdata/pgcenter.stat.golden.tar: `pgcenter report -W a -f ` prints the three INFO header lines and then nothing, exit 0. `pgcenter report -d -W a` prints 'unknown description requested', exit 0. The reason is that 'archiver' is not yet a key in view.New() (internal/view/view.go registers 'wal' and 'stat_io' but no 'archiver') nor in describeReport's map (report/report.go:666-694), so newApp's `views[config.ReportType]` yields a zero-value view.View and isFilenameOK matches no tar entry. This is the exact silently-empty-report outcome Decision 17 exists to prevent — reached from *inside* the whitelist rather than around it. It is a sequencing state, not a defect of this task: the task Details explicitly assign the view to Task 5 and the description to Task 7 and forbid compensating here (a 'not implemented yet' guard would have to be removed two tasks later). Recording it so it is not lost as a merge/release gate.", + "location": "cmd/report/report.go:158-159 (maps 'a' -> \"archiver\") -> report/report.go:85 (views[config.ReportType]) and report/report.go:666-694 (describeReport map)", + "impact": "If the feature branch merged or shipped with Task 04 but without Tasks 5 and 7, `-W a` would be a documented, help-text-advertised flag that produces a clean, empty, exit-0 report — an operator would reasonably conclude the archive contains no archiver data. Same failure class as a widened whitelist, different route.", + "recommendation": "No code change in this task. Gate the branch: do not merge 017 to develop until Task 5 registers the 'archiver' view and Task 7 adds its describeReport entry; the feature's verification step should include `pgcenter report -W a -f ` printing actual rows and `-d -W a` printing a real description. If a general guard is ever wanted, the right shape is a startup assertion that every value selectReport can return exists in view.New() — a single test over the whitelist, not a runtime branch per report type.", + "cwe": "CWE-1188 (Initialization of a Resource with an Insecure Default)" + }, + { + "severity": "minor", + "category": "dependency", + "title": "github.com/spf13/pflag is imported directly by the test but still marked // indirect in go.mod", + "description": "cmd/report/report_test.go now imports github.com/spf13/pflag directly (Test_walFlagPflagFailureShapes builds a local FlagSet), while go.mod:27 still carries `github.com/spf13/pflag v1.0.10 // indirect`. The build and `go test ./cmd/report/...` succeed because the module is present in the require block, and CI has no `go mod tidy -diff` check (.github/workflows/default.yml runs lint, gosec, govulncheck only), so nothing flags it today. The module graph nevertheless misrepresents the dependency: pflag is now a first-class requirement of this module's own code, not merely something cobra drags in.", + "location": "package: github.com/spf13/pflag@v1.0.10 (go.mod:27); import at cmd/report/report_test.go:5", + "impact": "Supply-chain hygiene rather than an exploitable flaw. An `// indirect` marker tells maintainers and tooling the version is cobra's to choose; a cobra upgrade that changed or dropped its pflag requirement would silently change (or break) what this test compiles against, and dependency-review tooling that treats indirect deps as lower priority would under-weight it.", + "recommendation": "Run `go mod tidy` and commit the resulting go.mod, which promotes pflag into the direct require block (drops the `// indirect` comment). No version bump and no go.sum change is involved. Coordinate the run with the other agents currently editing this working tree so the tidy does not pick up half-finished imports from internal/query or internal/postgres.", + "cwe": "CWE-1104 (Use of Unmaintained Third Party Components — dependency-management hygiene)" + }, + { + "severity": "minor", + "category": "best-practice", + "title": "assert.NotNil followed by an unguarded dereference in Test_walFlagDefinition", + "description": "Test_walFlagDefinition does `f := CommandDefinition.Flags().Lookup(\"wal\")`, then `assert.NotNil(t, f)` and immediately dereferences f on the next four lines. testify's `assert` records a failure and continues, unlike `require`, so if the flag were ever renamed or its registration dropped, Lookup returns nil and the following `f.Value.Type()` panics with a nil dereference. A panic aborts the whole test binary for package cmd/report, so a single renamed flag would take down every other test in the package and report a stack trace instead of the one clean assertion failure the test was written to produce. The task explicitly designates this test as the guard that keeps the local FlagSet mirror in Test_walFlagPflagFailureShapes honest, which makes its failure mode worth getting right.", + "location": "cmd/report/report_test.go:126-131", + "impact": "Test-suite robustness only; no runtime security impact. Degrades the diagnostic quality of the one test that guards the user-visible flag contract, and can mask unrelated failures in the same package behind a panic.", + "recommendation": "Import github.com/stretchr/testify/require and use `require.NotNil(t, f)` in place of `assert.NotNil(t, f)`; the remaining assertions can stay `assert`. require halts this test function only, leaving the rest of the package to run and report normally.", + "cwe": "CWE-476 (NULL Pointer Dereference — test code)" + } + ] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-test-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-test-reviewer-review-round2.json new file mode 100644 index 00000000..a33992d0 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-test-reviewer-review-round2.json @@ -0,0 +1,116 @@ +{ + "reviewer": "dev-test-reviewer", + "round": 2, + "status": "passed", + "summary": "Both round-1 majors are genuinely closed, and I re-ran every mutation myself on a byte-identical copy of the tree outside the repo rather than taking the claims on trust. (1) NoOptDefVal is now pinned: re-applying the rejected Decision 7 shim (`Flags().Lookup(\"wal\").NoOptDefVal = \"w\"` in init()) turns Test_walFlagDefinition RED, and so do a changed default (\"w\"), a changed shorthand (W->V) and a one-character help-text edit. (2) Precedence is now pinned on both sides: I moved the showWAL arm in three independent directions - above showDatabases, one slot above showFunctions, and one slot below showBgwriter - and all three redden Test_selectReport_WALPrecedence, so AC 5's second clause is now covered in every direction it could move. The three minors that mattered are closed too: strings.TrimSpace, strings.ToLower, first-character prefix matching, `default: return \"wal\"`, an unconditional `return \"wal\"` in the outer arm, `case \"a\"` returning \"wal\", ReportType hardcoded to \"wal\" in validate(), and a stray archiver->wal alias in validate() are all RED now. The declined suggestion (deriving the mirror FlagSet from CommandDefinition instead of hardcoding it) is defensible and I would not press it: both drift directions are already guarded - a wrong real definition reddens Test_walFlagDefinition, and I confirmed that a mirror-only drift in the load-bearing direction (setting NoOptDefVal on the mirror alone) reddens Test_walFlagPflagFailureShapes through its own Parse assertion. Deriving would trade that independence for nothing. The -f half of the mirror is still inert (deleting the fs.StringP(\"file\", ...) line keeps the test green), but the comment no longer claims otherwise, which was the actual round-1 complaint - the test now says what it proves. One residual minor: only the trailing-whitespace row was added, so a leading-only trim widening still survives. Hygiene re-checked on the real tree: -race -shuffle=on x3 green, each new test green in isolation, go vet clean, gofmt -l empty, and every showWAL hit contained to cmd/report/. Two pre-existing items noted below the findings are deliberately left alone as out of this task's scope.", + "findings": [ + { + "severity": "minor", + "category": "missing_coverage", + "location": "cmd/report/report_test.go:98 (Test_selectReport_WALWhitelistIsClosed table)", + "issue": "The round-1 recommendation named two whitespace rows - a leading-padded \" w\" and a trailing-padded \"a \" - and only the trailing one landed (`{value: \"w \", ...}`). That kills the widening that actually matters: I re-confirmed `switch strings.TrimSpace(opts.showWAL)` is RED, and so is `strings.TrimRight(..., \" \")`. But the asymmetry leaves one narrow hole: VERIFIED MUTATION (green) - `switch strings.TrimLeft(opts.showWAL, \" \")` keeps the whole suite passing, because no table value is left-padded. This is a much less likely widening than TrimSpace (a maintainer being 'forgiving about shell quoting' reaches for TrimSpace, not TrimLeft), which is why this stays minor rather than blocking - the dominant mutation in the class is dead.", + "recommendation": "One row, no other change - the loop and the assert message already handle it: `{value: \" w\", why: \"leading whitespace - no trimming is done on purpose, from the other side; a quoted '-W \\\" w\\\"' must not map\"}`. That closes the class symmetrically and turns the TrimLeft mutation red.", + "litmusTestFailed": false + } + ], + "resolvedFromRound1": [ + { + "round1Severity": "major", + "item": "NoOptDefVal not asserted in Test_walFlagDefinition", + "verdict": "closed", + "evidence": "report_test.go:159 asserts f.NoOptDefVal == \"\" with an explanatory message naming Decision 7. Re-ran the shim mutation (`CommandDefinition.Flags().Lookup(\"wal\").NoOptDefVal = \"w\"` appended to init()) on an isolated copy: --- FAIL: Test_walFlagDefinition. Perimeter around the same test also re-probed: default \"\" -> \"w\" RED, shorthand W -> V RED, one-character Usage edit RED." + }, + { + "round1Severity": "major", + "item": "Test_selectReport_WALPrecedence pinned only that -A beats -W", + "verdict": "closed", + "evidence": "Now a 4-row table (report_test.go:127-142) with a per-row `why` carried into the assert message. Three independent relocations of the showWAL arm all RED: hoisted above showDatabases, hoisted one slot above showFunctions (the tightest upper move), sunk one slot below showBgwriter (the tightest lower move). Both clauses of AC 5 are now covered." + }, + { + "round1Severity": "minor", + "item": "The -f half of the mirror is inert but presented as load-bearing", + "verdict": "closed as documented, not as fixed - and that is the right call", + "evidence": "The comment at report_test.go:180-183 now states that a string flag consumes the next token unconditionally and that -f is defined for realism, not because the outcome depends on it. I re-confirmed the underlying fact - deleting the fs.StringP(\"file\", ...) line leaves the test GREEN - so the inertness is real, but the test no longer misrepresents what it proves, which was the finding. Keeping -f for scenario realism is legitimate; the alternative (asserting *file and fs.Args()) would have been fine too, and nothing is lost by not taking it." + }, + { + "round1Severity": "minor", + "item": "assert.NotNil before dereferencing f can panic and take the package's other tests down", + "verdict": "closed", + "evidence": "report_test.go:151 uses require.NotNil, with a comment explaining why require rather than assert. A renamed flag now stops this one test instead of panicking through the binary. The nil-guard has no custom message, which round 1 suggested; require.NotNil's own output names the file and line, so this is cosmetic and not re-raised." + }, + { + "round1Severity": "minor", + "item": "No whitespace row - a TrimSpace widening passed unnoticed", + "verdict": "closed for the dominant mutation, one narrow hole remains", + "evidence": "The `\"w \"` row makes strings.TrimSpace RED and strings.TrimRight RED. strings.TrimLeft survives - see the single finding above. Also re-confirmed in the same table: strings.ToLower RED (the \"W\" row) and first-character prefix matching `opts.showWAL[:1]` RED." + }, + { + "round1Severity": "minor", + "item": "Nothing asserted that selectReport's result reaches report.Config.ReportType", + "verdict": "closed", + "evidence": "report_test.go:115-121 runs both mapped values end-to-end through validate() and asserts cfg.ReportType == \"archiver\" / \"wal\". Two mutations confirm it bites: ReportType hardcoded to \"wal\" in validate() RED, and a stray archiver->wal alias applied to r in validate() RED. This is the exact wrong move the task's Dependencies section anticipates while Task 5 is unlanded, and it is now caught. The anchor's ban on restructuring Test_options_validate's table was respected - the coverage lives in the dedicated whitelist test." + }, + { + "round1Severity": "n/a (suggestion)", + "item": "Derive the mirror FlagSet from CommandDefinition.Flags().Lookup(\"wal\") instead of hardcoding literals - DECLINED", + "verdict": "declining is defensible; I would not press it", + "evidence": "Three reasons the decline holds. (a) The task file explicitly prescribes a hardcoded mirror guarded by Test_walFlagDefinition, and the guard is now complete on the axis that governs parsing: NoOptDefVal. (b) Independence is real value - a derived mirror is self-consistent with a wrong definition by construction, so it could only ever fail on pflag's behaviour, never on the definition being wrong. (c) I probed the drift risk the derivation was meant to remove and it is already covered from both sides: a wrong real definition reddens Test_walFlagDefinition (verified for NoOptDefVal, DefValue, Shorthand, Usage), and a mirror-only drift in the load-bearing direction - setting NoOptDefVal on the mirror alone while the real flag stays correct - reddens Test_walFlagPflagFailureShapes, because bare `-W` then parses without the expected error. Both halves of the drift are caught without coupling the tests." + } + ], + "preExistingNotRaised": [ + { + "location": "cmd/report/report_test.go:31 (Test_options_validate positive branch)", + "note": "`assert.NotNil(t, got)` on a non-pointer report.Config is vacuously true for any struct value, and the table's declared `want report.Config` field is still never read - so the positive branch of that table still asserts nothing about -J, -X or -P. Round 1 recorded this as pre-existing and not this task's doing; the WAL-specific consequence is now closed by the end-to-end ReportType assertions in Test_selectReport_WALWhitelistIsClosed, and the TDD Anchor forbids restructuring this table. Correctly left alone here - it belongs in a separate cleanup, not in task 04." + }, + { + "location": "cmd/report/report_test.go:134 (Test_selectReport_WALPrecedence, the showDatabases row)", + "note": "Strictly redundant against the showFunctions row: any hoist that reddens the showDatabases row also reddens the showFunctions row, since showFunctions is the arm's immediate upper neighbour (verified - the one-slot hoist above showFunctions is caught). Not raised as a finding: it is one documented line that pins a distinct stated boundary and keeps holding if the chain above is ever reordered. Cheap, honest, and it fails with a message naming which boundary moved." + } + ], + "verificationPerformed": { + "method": "Full tree copied to a scratch directory outside the repo; every mutation applied and reverted there. The working tree was never modified - re-confirmed by git diff --stat after the run (cmd/report/report.go +16, cmd/report/report_test.go +117, unchanged from the starting diff). Parallel agents' changes in internal/query/ and internal/postgres/ were ignored throughout.", + "mutationsRun": [ + {"mutation": "NoOptDefVal = \"w\" on the real -W flag in init()", "result": "RED", "killedBy": "Test_walFlagDefinition"}, + {"mutation": "flag default \"\" -> \"w\"", "result": "RED", "killedBy": "Test_walFlagDefinition"}, + {"mutation": "shorthand W -> V", "result": "RED", "killedBy": "Test_walFlagDefinition"}, + {"mutation": "one-character help-text edit (archiver -> archivers)", "result": "RED", "killedBy": "Test_walFlagDefinition"}, + {"mutation": "showWAL arm hoisted above showDatabases", "result": "RED", "killedBy": "Test_selectReport_WALPrecedence"}, + {"mutation": "showWAL arm hoisted one slot above showFunctions", "result": "RED", "killedBy": "Test_selectReport_WALPrecedence"}, + {"mutation": "showWAL arm sunk one slot below showBgwriter", "result": "RED", "killedBy": "Test_selectReport_WALPrecedence"}, + {"mutation": "switch strings.TrimSpace(opts.showWAL)", "result": "RED", "killedBy": "Test_selectReport_WALWhitelistIsClosed"}, + {"mutation": "switch strings.TrimRight(opts.showWAL, \" \")", "result": "RED", "killedBy": "Test_selectReport_WALWhitelistIsClosed"}, + {"mutation": "switch strings.TrimLeft(opts.showWAL, \" \")", "result": "GREEN - SURVIVED", "killedBy": "none - the single open finding"}, + {"mutation": "switch strings.ToLower(opts.showWAL)", "result": "RED", "killedBy": "Test_selectReport_WALWhitelistIsClosed"}, + {"mutation": "switch opts.showWAL[:1] (prefix matching)", "result": "RED", "killedBy": "Test_selectReport_WALWhitelistIsClosed"}, + {"mutation": "default: return \"wal\" in the inner switch", "result": "RED", "killedBy": "Test_selectReport_WALWhitelistIsClosed, Test_options_validate, Test_walFlagPflagFailureShapes"}, + {"mutation": "outer arm returns \"wal\" unconditionally", "result": "RED", "killedBy": "5 tests including Test_selectReport and Test_selectReport_WALPrecedence"}, + {"mutation": "case \"a\" returns \"wal\"", "result": "RED", "killedBy": "Test_selectReport, Test_selectReport_WALPrecedence, Test_selectReport_WALWhitelistIsClosed"}, + {"mutation": "ReportType hardcoded to \"wal\" in validate()", "result": "RED", "killedBy": "Test_selectReport_WALWhitelistIsClosed"}, + {"mutation": "archiver -> wal alias applied to r in validate()", "result": "RED", "killedBy": "Test_selectReport_WALWhitelistIsClosed"}, + {"mutation": "delete fs.StringP(\"file\", \"f\", ...) from the mirror", "result": "GREEN - SURVIVED (expected; the comment now says so)", "killedBy": "none - by design"}, + {"mutation": "mirror-only NoOptDefVal drift (mirror lies, real flag correct)", "result": "RED", "killedBy": "Test_walFlagPflagFailureShapes"} + ], + "hygieneOnRealTree": [ + "go test ./cmd/report/... -race -shuffle=on -count=1, three runs: all ok", + "each of the four new tests run in isolation via -run: all PASS", + "go vet ./cmd/report/...: clean", + "gofmt -l cmd/report: empty", + "grep -rn showWAL --include=*.go .: all 10 hits inside cmd/report/ (containment holds)" + ] + }, + "metrics": { + "filesReviewed": 2, + "litmusTest": { + "checked": 14, + "passed": 14, + "failed": 0 + }, + "coverageAssessment": "excellent", + "pyramidBalance": { + "unit": 14, + "integration": 0, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-test-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-test-reviewer-review.json new file mode 100644 index 00000000..ff38ba95 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04-dev-test-reviewer-review.json @@ -0,0 +1,70 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "needs_improvement", + "summary": "The five new/changed tests in cmd/report/report_test.go are honest work: every assertion is non-vacuous, every one can fail for the right reason, the whitelist table carries per-row 'why' text into its failure message, error-message contracts are pinned with assert.EqualError rather than bare assert.Error, and the local pflag.FlagSet mirror correctly avoids mutating the package-level opts (verified: -shuffle=on x3, -race, and each new test run in isolation all pass). All six TDD Anchor items are present and match the anchor's wording. The problem is not test quality but perimeter: I ran six additional mutations against the finished code and four stayed GREEN. Two are major. (1) Setting NoOptDefVal=\"w\" on the -W flag - the shim Decision 7 explicitly rejected - leaves the whole suite green while destroying both documented failure shapes and making 'report -W a' silently produce the wal report (confirmed by running the real binary under the mutation). Test_walFlagDefinition does not assert NoOptDefVal, so the property that makes the pflag mirror faithful is exactly the one property the mirror's guard does not check. (2) Test_selectReport_WALPrecedence pins only that -A beats -W; moving the showWAL arm up above showDatabases/showTables/showIndexes/showFunctions keeps the suite green, so half of AC 5 ('the arm sits between showFunctions and showBgwriter') is unpinned. Four minors follow, all cheap to close. The three mutations named in the Acceptance Criteria were re-confirmed as taken facts and were not re-run.", + "findings": [ + { + "severity": "major", + "category": "missing_coverage", + "location": "cmd/report/report_test.go:119-126 (Test_walFlagDefinition)", + "issue": "NoOptDefVal is not asserted, so the exact shim Decision 7 rejected is invisible to the suite. VERIFIED MUTATION (green): adding `CommandDefinition.Flags().Lookup(\"wal\").NoOptDefVal = \"w\"` to init() leaves all 10 tests passing, while the real CLI changes behaviour in both documented shapes - `report -W -f x.tar` no longer prints 'report type is not specified, quit' (it proceeds to open the file), and `report -W a -f x.tar` silently resolves to the WAL report with 'a' dropped as a positional arg, i.e. exactly the 'silently fails' outcome the roadmap owner rejected the shim for. This is also the load-bearing gap in the mirror contract: Test_walFlagPflagFailureShapes asserts pflag's 'flag needs an argument' message against a locally built FlagSet, and its comment claims Test_walFlagDefinition keeps the mirror honest - but whether the flag demands an argument at all is governed by NoOptDefVal, the one field the guard does not read. Type/shorthand/DefValue/Usage can all still match while parsing behaviour diverges.", + "recommendation": "Add one line to Test_walFlagDefinition: `assert.Equal(t, \"\", f.NoOptDefVal, \"-W must demand an argument; a non-empty NoOptDefVal is the rejected shim from Decision 7 and makes '-W a' silently mean 'wal'\")`. That single assertion turns the mutation red and makes the mirror's honesty claim true. Optionally also assert `assert.False(t, f.Changed)` is not needed - NoOptDefVal is the whole gap.", + "litmusTestFailed": false + }, + { + "severity": "major", + "category": "missing_coverage", + "location": "cmd/report/report_test.go:112-114 (Test_selectReport_WALPrecedence)", + "issue": "The test pins one direction only - that showActivity outranks showWAL - so it survives any repositioning of the arm that keeps it below the showActivity case. VERIFIED MUTATION (green): relocating the whole `case opts.showWAL != \"\":` arm to directly after `case opts.showReplication:` (above showDatabases, showTables, showIndexes and showFunctions) keeps the entire suite passing, yet `report -F -W a` changes from 'functions' to 'archiver' and `report -D g -W a` from 'databases_general' to 'archiver'. AC 5 has two clauses - '-A still beats -W' AND 'the showWAL arm sits between showFunctions and showBgwriter' - and only the first is tested. Test_selectReport cannot help: every row sets exactly one flag, so the first-match-wins ordering is never exercised there. Per patterns.md the mutation a precedence test must fail on is 'the arm moved', and this one does not fail on 3 of the 4 directions it could move.", + "recommendation": "Grow Test_selectReport_WALPrecedence into a small table that pins both boundaries of the arm's slot: `{opts: options{showActivity: true, showWAL: \"a\"}, want: \"activity\"}` (keep - upper bound vs the flag chain head), `{opts: options{showFunctions: true, showWAL: \"a\"}, want: \"functions\"}` (upper bound - the arm may not rise above showFunctions), `{opts: options{showWAL: \"a\", showBgwriter: true}, want: \"archiver\"}` (lower bound - the arm may not sink below showBgwriter), and `{opts: options{showDatabases: \"g\", showWAL: \"a\"}, want: \"databases_general\"}`. Keep the existing per-assert message style so a failure names which boundary moved.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "cmd/report/report_test.go:138 and :148-151 (Test_walFlagPflagFailureShapes)", + "issue": "The `-f` half of the mirror is inert and is presented as load-bearing. VERIFIED MUTATION (green): deleting the `fs.StringP(\"file\", \"f\", \"pgcenter.stat.tar\", ...)` line entirely leaves the test passing, because pflag makes a string flag consume the next token unconditionally - `*wal == \"-f\"` would hold even if -f were never defined. So the second failure shape demonstrates 'the -W value is the literal -f' but does NOT demonstrate the thing the comment claims, namely that pflag consumed -f as a value INSTEAD of parsing it as the file flag. Separately, Test_walFlagDefinition guards only the 'wal' flag, so nothing keeps the -f half of the mirror in sync with the real definition at report.go:79 - the honesty claim in the comment covers half the mirror.", + "recommendation": "Make the -f line load-bearing by binding and asserting it: `file := fs.StringP(\"file\", \"f\", \"pgcenter.stat.tar\", \"read stats from file\")`, then after the successful Parse add `assert.Equal(t, \"pgcenter.stat.tar\", *file, \"-f was swallowed as the value of -W, so the file flag keeps its default\")` and `assert.Equal(t, []string{\"dump.tar\"}, fs.Args(), \"dump.tar falls through as a positional argument - the input file is never set, which is why no 'file not found' masks the real cause\")`. Also extend Test_walFlagDefinition with a Lookup(\"file\") block asserting Value.Type()==\"string\", Shorthand==\"f\", DefValue==\"pgcenter.stat.tar\", so both halves of the mirror are guarded as the comment promises.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "cmd/report/report_test.go:120-122 (Test_walFlagDefinition)", + "issue": "`assert.NotNil(t, f)` records a failure but does not stop the test, so a nil Lookup result falls straight into `f.Value.Type()`. VERIFIED MUTATION: renaming the flag from \"wal\" to \"wal-report\" panics with a nil pointer dereference at report_test.go:122, and the panic aborts the whole test binary - Test_walFlagPflagFailureShapes, Test_setReportInterval, Test_parseTimestamp, Test_parseTimepart and Test_parseFilterString never ran. The suite still goes red (correctly), but a single flag rename hides the results of five unrelated tests and buries the real assertion output under a stack trace.", + "recommendation": "Halt on the nil: either import `github.com/stretchr/testify/require` and use `require.NotNil(t, f, \"flag 'wal' must be registered on CommandDefinition\")`, or keep the assert-only import style of the file with `if f == nil { t.Fatal(\"flag 'wal' is not registered on CommandDefinition\") }`. Everything after it stays unchanged.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "cmd/report/report_test.go:85-97 (Test_selectReport_WALWhitelistIsClosed table)", + "issue": "AC 3 forbids three kinds of normalisation - lowercasing, trimming, prefix matching - and the table pins two of them. The 'W' row kills a strings.ToLower widening and the 'wal'/'archiver' rows kill a prefix/HasPrefix widening, but no row is whitespace-padded. VERIFIED MUTATION (green): changing the inner switch to `switch strings.TrimSpace(opts.showWAL)` keeps the entire suite passing, because none of the eight values changes under trimming. A trimming widening is the most likely of the three to be added in good faith by a later maintainer ('be forgiving about shell quoting'), and it is precisely the class Decision 17 calls 'a new way to select a report the operator did not ask for'.", + "recommendation": "Add two rows to the existing table: `{value: \" w\", why: \"leading whitespace - no trimming is done on purpose; a trim would make a quoted ' w' select the wal report\"}` and `{value: \"a \", why: \"trailing whitespace - same widening from the other side\"}`. No other change; the loop and assert message already handle them.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "cmd/report/report.go:118 (`ReportType: r`) / cmd/report/report_test.go:12-35 (Test_options_validate)", + "issue": "Nothing asserts that selectReport's carefully-whitelisted result actually reaches report.Config.ReportType. VERIFIED MUTATION (green): hardcoding `ReportType: \"wal\"` in validate() leaves the suite passing. The cause is pre-existing and not this task's doing - Test_options_validate's positive branch does `assert.NotNil(t, got)` on a non-pointer report.Config (vacuously true for any struct value) and its declared `want report.Config` field is never read, so the same blind spot covers -J, -X and -P too. It matters here because the task's Dependencies section explicitly anticipates the wrong move it would hide: a temporary 'archiver -> wal' alias added in validate() while Task 5 is unlanded would pass every test in this diff, including the whole whitelist suite, and would ship exactly the silently-wrong-report Decision 17 exists to prevent. Note the TDD Anchor forbids restructuring the Test_options_validate table, so the fix must live outside it.", + "recommendation": "Add a small standalone test (no table restructure, per the anchor): `func Test_options_validate_WALReportTypeReachesConfig(t *testing.T)` asserting `cfg, err := options{showWAL: \"a\"}.validate(); assert.NoError(t, err); assert.Equal(t, \"archiver\", cfg.ReportType, \"the whitelisted value must reach report.Config unchanged - no aliasing in validate()\")`, and the same for `\"w\"` -> `\"wal\"`. Two calls, four assertions, and the aliasing mutation turns red.", + "litmusTestFailed": false + } + ], + "metrics": { + "filesReviewed": 2, + "litmusTest": { + "checked": 12, + "passed": 12, + "failed": 0 + }, + "coverageAssessment": "adequate", + "pyramidBalance": { + "unit": 12, + "integration": 0, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-04.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04.md similarity index 99% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-04.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04.md index bc517e09..e38ae987 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-04.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-04.md @@ -1,5 +1,5 @@ --- -status: planned # planned -> in_progress -> done +status: done # planned -> in_progress -> done depends_on: [] # ID задач-зависимостей (строки: ["01", "02"]) wave: 1 # волна параллельного выполнения skills: [code-writing] # МАССИВ скиллов для загрузки diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-code-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-code-reviewer-review-round2.json new file mode 100644 index 00000000..8f4ea4ab --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-code-reviewer-review-round2.json @@ -0,0 +1,64 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "Round 2 closes the one major from round 1 and both accepted minors, and the extra pins that came in from dev-test-reviewer (Name, QueryTmpl seed, full-equality Msg, wantArchiver membership) make TestNew_ArchiverView and Test_filterViews materially stronger than round 1. I re-ran six mutations against a scratch copy of the tree and every one reddened exactly the predicted tests, with the new msgAndArgs naming the offending row: dropping ColsWidth+Filters, renaming Name, swapping the QueryTmpl seed and a Msg prefix typo each redden TestNew_ArchiverView only; NotRecordable:true reddens Test_filterViews on exactly the three >=PG14 rows (both the count and the archiver-membership assertion); dropping MinRequiredVersion reddens Test_filterViews and TestView_VersionOK on exactly the four <=PG13 rows. gofmt is clean on all three files, record/record.go is untouched and case \"wal\": is unchanged. One residual minor: the new comment above the NotNil assertions names align.SetAlign as an in-place writer of ColsWidth, which it provably is not. No further round is required.", + "criticalIssues": [], + "suggestions": [ + { + "file": "internal/view/view_test.go", + "line": 121, + "severity": "minor", + "category": "readability", + "suggestion": "The comment above the two NotNil assertions says \"align.SetAlign and setFilter/clearAllFilters write into them in place\". align.SetAlign does not: internal/align/align.go:18 builds a fresh map (widthes := make(map[int]int)) and top/stat.go:788 assigns it wholesale over view.ColsWidth, so that path repairs a nil map rather than panicking on it. The real unguarded in-place writers are top/config_view.go:100 (increaseWidth, no bounds or nil guard at all) and :124 (decreaseWidth, guarded only on Cols) for ColsWidth, and top/config_view.go:166 (setFilter, view.Filters[view.OrderKey] = re) for Filters — clearAllFilters only calls delete, which is a no-op on a nil map, not a panic. Suggested wording: \"top/config_view.go:100/:124 write into ColsWidth and :166 writes into Filters in place, so a nil map is a runtime panic inside a gocui key handler that no count test would catch.\" Same paragraph, one line, cite the writers that actually panic. (The inaccuracy is inherited from the task's own Edge Cases section, which makes the same align.SetAlign claim — worth correcting there too if the task file is still live.) While in the neighbourhood: line 100 of the same file has a stray quote in `deleting case \"archiver:\" from Configure()` (should be `case \"archiver\":`).", + "benefit": "The assertions are correct and load-bearing — the panic is genuinely reachable, since printDbstat returns early on s.Error and never calls alignViewToResult, so a failing archiver query leaves ColsWidth as the seed and the widen hotkey writes into it unguarded. But the comment sends the next maintainer to a function that demonstrably copies rather than mutates; anyone verifying the claim will conclude the assertion is unnecessary and may drop it. Naming the two config_view.go lines makes the claim checkable in one grep.", + "optional": true + } + ], + "resolvedFromRound1": [ + { + "round1Severity": "major", + "file": "internal/view/view_test.go", + "finding": "TestNew_ArchiverView did not assert non-nil ColsWidth/Filters", + "resolution": "Fixed. assert.NotNil on both at view_test.go:123-124. Verified: deleting both fields from the New() entry reddens TestNew_ArchiverView and nothing else." + }, + { + "round1Severity": "minor", + "file": "internal/view/view.go", + "finding": "Configure() case comment over-promised a one-file future version branch", + "resolution": "Fixed. Narrowed to \"confined to query/archiver.go on the production side\" (view.go:409-410). Accurate as written." + }, + { + "round1Severity": "minor", + "file": "record/record_test.go", + "finding": "Test_filterViews and TestView_VersionOK gave no per-row identity on failure", + "resolution": "Fixed in both. Confirmed under mutation: failures now print version=130000 / archiver kept? version=140000 pgss=\"public\" instead of a bare count diff." + }, + { + "round1Severity": "minor", + "file": "record/record_test.go", + "finding": "Collapse the multi-layer block comment above the Test_filterViews table", + "resolution": "Rejected by the implementer; I withdraw it. The task explicitly instructs to extend that comment, and the per-feature layers each state which rows moved and why — that is exactly what the next view registration needs to recompute the numbers. The comment is now 26 lines for a 7-row table, which is long, but trimming it is a housekeeping change for whoever owns the file next, not a defect in this task." + }, + { + "round1Severity": "n/a", + "file": "internal/view/view_test.go", + "finding": "dev-test-reviewer's suggested registry-wide invariant test (key==Name, non-nil maps for all 28 views) was not added", + "resolution": "I agree with the rejection. Such a test asserts over views owned by other tasks and features; adding it here would make this task the owner of failures it cannot cause. Recording it as tech debt is the right call — the archiver-specific half of that invariant is already pinned by TestNew_ArchiverView." + } + ], + "mutationsVerified": [ + "drop ColsWidth + Filters from the New() entry -> RED: TestNew_ArchiverView only", + "Name: \"archiver\" -> \"archiver2\" -> RED: TestNew_ArchiverView only", + "QueryTmpl seed -> query.PgStatBgwriterPG14 -> RED: TestNew_ArchiverView only", + "Msg prefix typo (\"statistics\" -> \"stats\", archive_mode=on retained) -> RED: TestNew_ArchiverView only (substring assertion would have stayed green)", + "NotRecordable: true -> RED: Test_filterViews on exactly the three >=PG14 rows (count + wantArchiver), plus TestNew_ArchiverView; the four <=PG13 rows stay green as the task predicts", + "drop MinRequiredVersion -> RED: Test_filterViews and TestView_VersionOK on exactly the four <=PG13 rows" + ], + "metrics": { + "filesReviewed": 3, + "criticalIssuesCount": 0, + "majorIssuesCount": 0, + "minorIssuesCount": 1, + "testCoverageAssessment": "excellent" + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-code-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-code-reviewer-review.json new file mode 100644 index 00000000..1cf8372d --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-code-reviewer-review.json @@ -0,0 +1,51 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "The archiver view registration and the count-test updates are correct, minimal and match the task spec field for field: the New() entry, the Configure() case, the guard test, and all seven Test_filterViews rows reconcile arithmetically (wantN + wantV == 28 on every row), and case \"wal\": is byte-identical while record/record.go is untouched. I re-ran all five mutations named in the TDD Anchor and each reddened exactly the predicted tests and no others — including the honest negative claim that removing case \"archiver\": from Configure() leaves the suite green. One gap: TestNew_ArchiverView pins the fields listed in step 4 of the task but not the non-nil ColsWidth/Filters maps that the acceptance criteria also list, and those are the one field pair whose loss is an unrecoverable runtime panic rather than a wrong number.", + "criticalIssues": [], + "suggestions": [ + { + "file": "internal/view/view_test.go", + "line": 102, + "severity": "major", + "category": "testing", + "suggestion": "TestNew_ArchiverView does not assert that ColsWidth and Filters are non-nil, although the task's first acceptance criterion lists \"non-nil empty ColsWidth and Filters\" among the fields and the second says the test \"pins every field listed above\". (Step 4 of \"What to do\" enumerates a narrower field list, which is what was implemented — the task is internally inconsistent here, and the implementation followed the narrower half.) Add two lines: assert.NotNil(t, archiver.ColsWidth) and assert.NotNil(t, archiver.Filters). QueryTmpl is already covered indirectly by TestViews_Configure in both version arms, so it does not need duplicating.", + "benefit": "Every other pinned field fails loudly as a wrong number if it regresses; a nil ColsWidth or Filters instead panics at runtime on the first write — top/config_view.go:100 and :124 assign into ColsWidth[idx], and :166 assigns into Filters[view.OrderKey]. That is a hard crash the operator sees the moment they widen a column or set a filter on the archiver screen, and no count-based test in either package would catch it. The task's own Edge Cases section flags exactly this (\"A nil map here is a runtime panic on first use, and no count test would catch it\"). Two assertions close it. Note the sibling guard tests (TestNew_BgwriterView, TestNew_StatIOView) omit it too, so this is a convention gap rather than a regression introduced here.", + "optional": false + }, + { + "file": "internal/view/view.go", + "line": 405, + "severity": "minor", + "category": "readability", + "suggestion": "The comment on case \"archiver\": claims a future version branch \"stays a one-line change in query/archiver.go instead of two files\". That is true only of the production files — a real version branch would still change SelectStatArchiverQuery, the guard test, and the TestViews_Configure arms. Consider narrowing the wording to \"a future version branch stays confined to query/archiver.go on the production side\".", + "benefit": "The comment's job is to stop a reviewer reading a functionally no-op case as dead code, and it does that well. Tightening the second half keeps it from over-promising and going stale the day the branch is actually added.", + "optional": true + }, + { + "file": "record/record_test.go", + "line": 139, + "severity": "minor", + "category": "maintainability", + "suggestion": "Test_filterViews has no per-row identity on failure: the loop calls assert.Equal(t, tc.wantN, n) with no message, so a count drift reports only \"expected: 9, actual: 10\" with no version or pgssSchema. When I ran the NotRecordable:true mutation I had to count assertion pairs to work out which three rows had reddened. Either wrap the body in t.Run(fmt.Sprintf(\"pg%d_pgss=%q\", tc.version, tc.pgssSchema), ...) or pass the row as msgAndArgs: assert.Equal(t, tc.wantN, n, \"version=%d pgss=%q\", tc.version, tc.pgssSchema).", + "benefit": "This table is the tripwire every future view registration hits first, and this task moved all seven rows. Naming the row turns a future failure from an arithmetic puzzle into a one-line diagnosis. Pre-existing shape, but this task is the one that rewrote every number in it.", + "optional": true + }, + { + "file": "record/record_test.go", + "line": 116, + "severity": "minor", + "category": "maintainability", + "suggestion": "The reasoning block above the table now stacks four historical layers (the feature 008 NotRecordable baseline, the bgwriter/replslots move, the stat_io/statements_jit \"only the reason changed\" note, and now the feature 017 archiver paragraph), 23 lines for a 7-row table. The new paragraph is correct and well placed, but the block is accreting archaeology. Consider collapsing the 008-era paragraphs into one sentence stating the current invariant — wantN + wantV == len(view.New()) on every row, wantN being version-incompatible plus pgss-gated statements_* views — and keeping only the per-feature delta that is still load-bearing.", + "benefit": "The invariant is what a future maintainer needs to recompute the numbers; the change history is recoverable from git. A shorter comment is likelier to be updated correctly by the next view registration than a long one.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 3, + "criticalIssuesCount": 0, + "majorIssuesCount": 1, + "minorIssuesCount": 3, + "testCoverageAssessment": "excellent" + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-security-auditor-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-security-auditor-review.json new file mode 100644 index 00000000..84191f0f --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-security-auditor-review.json @@ -0,0 +1,31 @@ +{ + "status": "approved", + "summary": { + "totalFindings": 2, + "critical": 0, + "major": 0, + "minor": 2 + }, + "findings": [ + { + "severity": "minor", + "category": "A04: Insecure Design", + "title": "Recordable view backed by a privileged function makes an under-privileged `pgcenter record` run abort entirely", + "description": "The new entry registers `archiver` with `NotRecordable` left at its zero value `false`, so `record/record.go:filterViews()` keeps it on every PG >= 14 target and `tarRecorder.collect()` executes its query on every tick. The query (task 01, out of scope) calls `pg_ls_archive_statusdir()`, whose EXECUTE privilege is `{postgres, pg_monitor}`. In `record/recorder.go` the per-view loop does `res, err := stat.NewPGresultQuery(db, v.Query); if err != nil { return nil, err }`, and `record/record.go:172-175` propagates that error out of the sampling loop — so a single permission-denied view terminates the whole recording session, discarding the other 24+ screens the operator asked for. This is a documented design choice (tech-spec Decision 4: privilege failure takes down the whole screen) and it is not a regression: the already-recordable `wal` view calls `pg_ls_waldir()` unconditionally at the same PG14 floor and the same privilege class, so a role without `pg_monitor` already loses the whole recording today. Flagged as availability-by-design rather than a defect introduced here: this task widens the number of screens that can trigger the all-or-nothing abort by one.", + "location": "internal/view/view.go:141-152 (archiver entry, NotRecordable omitted); consequence at record/recorder.go:135-141 and record/record.go:172-175", + "impact": "An operator collecting a diagnostic capture with a role that has neither superuser nor pg_monitor gets no capture at all instead of a partial one — the failure mode is loss of troubleshooting data at the moment it is needed, not disclosure. No confidentiality or integrity impact; the privilege check itself is enforced correctly by PostgreSQL.", + "recommendation": "No change required in this task — `NotRecordable: false` is mandated by the tech-spec and matches `wal`. The durable fix belongs in the recorder, not the registry: make the per-view loop in `tarRecorder.collect()` tolerant of a single view's query error (record the view as empty/skipped and continue, printing an INFO line like the existing `pg_stat_statements not found` path) instead of returning the error. Worth raising as a tech-debt entry so the all-or-nothing behaviour stays a decision rather than an oversight.", + "cwe": "CWE-703" + }, + { + "severity": "minor", + "category": "A03: Injection", + "title": "Two server-supplied WAL-name columns reach the terminal unsanitised (pre-existing tech-debt [029], not widened)", + "description": "The registered view renders 9 columns, two of which — `last_archived_wal` (col 3) and `last_failed_wal` (col 6) — are strings supplied by the server and printed by `top/stat.go:printDataCell` without an SGR wrapper. Decision 16 argues no sanitisation is needed because PostgreSQL only records names that passed its own `VALID_XFN_CHARS` filter (hex digits plus `.history`/`.backup`/`.partial`), a set with no ESC and no control characters. That reasoning was verified and holds for a genuine PostgreSQL server; it does not hold if pgcenter is pointed at a hostile or spoofed endpoint that merely speaks the wire protocol and returns arbitrary text for those columns. This is exactly the trust boundary already recorded as tech-debt [029] (`docs/tech-debt.md:47`), which covers every text column on every existing screen; the archiver view neither widens nor closes it. Note also that `DiffIntvl{0,0}` on this entry is correct and load-bearing for robustness: it keeps the `'Archiver'` literal at column 0 and the four NULL-able columns out of `diff()`/`strconv.ParseInt`.", + "location": "internal/view/view.go:141-152 (registration); rendering sink at top/stat.go printDataCell; tech-debt docs/tech-debt.md:47", + "impact": "With a trusted PostgreSQL server: none — the server-side character filter makes escape injection unreachable. With an attacker-controlled endpoint the operator chose to connect to: terminal escape sequences in a row value could corrupt the display, and under the 016 pause feature the corrupted frame persists for the length of the freeze instead of self-healing on the next tick. No disk or log sink is involved.", + "recommendation": "No action in this task. If tech-debt [029] is ever addressed, fix it once in the shared write path (`printDataCell`), not per-view — a per-column sanitiser here would create an inconsistent precedent against every other screen, which is precisely what Decision 16 rejected.", + "cwe": "CWE-150" + } + ] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-test-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-test-reviewer-review-round2.json new file mode 100644 index 00000000..a22dd4de --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-test-reviewer-review-round2.json @@ -0,0 +1,38 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "passed", + "summary": "Every round-1 finding is closed, and I did not take the mutation report on trust — I copied the working tree into a sandbox and re-ran all six claimed mutations plus three of my own. All nine reproduce exactly as described. Removing ColsWidth and Filters from the archiver entry gives two NotNil failures at view_test.go:123-124; Name mutated to \"archive\" reddens TestNew_ArchiverView alone; the QueryTmpl seed swapped to query.PgStatWALPG14 reddens it with a full query diff; the prefix-only Msg typo \"Show archiver stats\" reddens it, which the old Contains would not have caught. NotRecordable:true reddens Test_filterViews on exactly the three >=PG14 rows — six count assertions plus the three new wantArchiver assertions naming archiver directly — and the four <=PG13 rows correctly stay green. Dropping MinRequiredVersion reddens exactly the four <=PG13 rows in both TestView_VersionOK and Test_filterViews, and the added msgAndArgs names the offending version on every one of them, which is the diagnostic round 1 asked for. My own additions: reverting task 02's PG 19 branch in SelectStatWALQuery reddens TestViews_Configure on QueryTmpl/Ncols/DiffIntvl, and so does deleting case \"wal\" from Configure — the wal half is a genuine wiring gate, not a restatement of the selector's own table test. Deleting case \"archiver\" from Configure leaves the package green, exactly as the now-documented caveat at the assertion site says. The map-lookup form of the archiver membership assertion is a better call than the assert.Contains I suggested: it produced clean one-line failures naming the version, where Contains would have dumped 28 View structs. I accept the TestNew_ViewMapInvariants deferral — see the finding below; I checked the invariant against all 28 registered views and it holds today with zero violations, so it is a pure generalization with no latent defect behind it, and this task's own risk is fully covered by the archiver-specific assertions. Only cosmetic issues remain.", + "findings": [ + { + "severity": "minor", + "category": "missing_coverage", + "location": "internal/view/view_test.go — absent test TestNew_ViewMapInvariants (deferred from round 1)", + "issue": "ACCEPTED, NOT BLOCKING — recorded here only so the tech-debt entry has a traceable source. The round-1 recommendation to add one registry-wide invariant (key == Name, non-nil ColsWidth, non-nil Filters over all views from New()) was declined as out of scope, since it would guard views owned by other tasks and features. The substance of the round-1 major findings is fully closed by the archiver-specific assertions, each mutation-verified above, so nothing this task introduces is left unpinned. I ran the proposed invariant against the current registry to check the deferral is not hiding a defect: all 28 views satisfy all three conditions — no key/Name mismatch, no nil ColsWidth, no nil Filters. So the test would be green the day it is written; it is a generalization that lowers the cost of the next view registration, not a repair. The residual risk is narrow and real but small: the next view added by a different task can still omit a non-nil map or misspell Name and no test will notice, because the guard tests are per-view by construction.", + "recommendation": "Keep the deferral. When the tech-debt item is picked up, the whole test is: `func TestNew_ViewMapInvariants(t *testing.T) { for k, v := range New() { assert.Equal(t, k, v.Name, \"map key %q must equal Name\", k); assert.NotNil(t, v.ColsWidth, \"%s.ColsWidth must be non-nil\", k); assert.NotNil(t, v.Filters, \"%s.Filters must be non-nil\", k) } }`. Verified green against the current 28-entry registry, so it lands without touching any other task's view. Worth attaching to patterns.md 'Adding a New View' rather than to a feature task, since that is where the next registration will look.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/view/view_test.go:230", + "issue": "Typo in the honesty caveat added this round: the comment reads `deleting case \"archiver:\"` with the closing quote inside the colon instead of `case \"archiver\":`. Cosmetic, but this particular comment is load-bearing — it is the only place a future reader is told that these three assertions cannot be reddened by a Configure() mutation, and it names the exact code construct they should try deleting to check that claim. A mis-quoted identifier in an instruction to go look at the source is worth one character of care. Behaviour is unaffected; I confirmed the claim itself is accurate by deleting the case block and watching internal/view stay green.", + "recommendation": "In view_test.go:230 change `deleting case \"archiver:\"` to `deleting case \"archiver\":`. No assertion change.", + "litmusTestFailed": false + } + ], + "metrics": { + "filesReviewed": 3, + "litmusTest": { + "checked": 16, + "passed": 15, + "failed": 1 + }, + "coverageAssessment": "excellent", + "pyramidBalance": { + "unit": 5, + "integration": 0, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-test-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-test-reviewer-review.json new file mode 100644 index 00000000..4198b626 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05-dev-test-reviewer-review.json @@ -0,0 +1,78 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "needs_improvement", + "summary": "The count-based tests are correct and genuinely sensitive: I re-derived all seven TestView_VersionOK rows and all seven Test_filterViews rows by hand against the 28-entry registry and every number matches, and the reported mutation runs (MinRequiredVersion dropped, Ncols/DiffIntvl perturbed, Msg substring removed, NotRecordable:true, task 02's PG 19 branch reverted) are consistent with what the code can actually produce — including the subtle claim that the four <=PG13 rows correctly stay green under NotRecordable:true, since filterViews increments the same counter in both drop branches. The wal half of TestViews_Configure is a real wiring gate. The weakness is not in what is asserted but in what TestNew_ArchiverView leaves out: three fields the task itself calls load-bearing (Name, non-nil ColsWidth, non-nil Filters) are pinned by nothing in the repo, and each fails the litmus test — mutating them leaves all five tests in this task green while producing a silent runtime failure. Two one-line assertions (or one registry-wide invariant test) close both. The acknowledged limitation of the archiver arms in TestViews_Configure is adequately handled in substance — those assertions are not vacuous, they redden on drift between SelectStatArchiverQuery and the static entry — but the limitation is documented only in the task file, not at the assertion site.", + "findings": [ + { + "severity": "major", + "category": "missing_coverage", + "location": "internal/view/view_test.go:102-117 (TestNew_ArchiverView), against internal/view/view.go:149,151", + "issue": "Neither TestNew_ArchiverView nor any other test asserts that the archiver entry's ColsWidth and Filters maps are non-nil. AC bullet 1 requires 'non-nil empty ColsWidth and Filters', and the task's own Edge cases section names this exact failure: 'A nil map here is a runtime panic on first use, and no count test would catch it'. align/SetAlign and top/config_view.go:100,124 assign into ColsWidth in place, and setFilter assigns into Filters in place — assignment to a nil map panics in Go. Litmus: delete 'ColsWidth: map[int]int{}' and 'Filters: map[int]*regexp.Regexp{}' from the archiver entry and all five tests in this task stay green (I checked every consumer: report_test.go:1045 asserts NotNil only for databases_general, and the top/ tests operate on activity/synthetic views), while the TUI panics the first time an operator widens a column or sets a filter on the archiver screen.", + "recommendation": "Add to TestNew_ArchiverView: `assert.NotNil(t, archiver.ColsWidth)` and `assert.NotNil(t, archiver.Filters)`. Better, since this gap is registry-wide (bgwriter, replslots, stat_io, stat_io_time and statements_jit guards omit it too) and every future view registration will repeat it, add one invariant test to the file this task owns: `func TestNew_ViewMapInvariants(t *testing.T) { for k, v := range New() { assert.Equal(t, k, v.Name, \"map key %q must equal Name\", k); assert.NotNil(t, v.ColsWidth, \"%s.ColsWidth must be non-nil\", k); assert.NotNil(t, v.Filters, \"%s.Filters must be non-nil\", k) } }`. That single test also closes the Name finding below and matches patterns.md 'Adding a New View'.", + "litmusTestFailed": true + }, + { + "severity": "major", + "category": "missing_coverage", + "location": "internal/view/view_test.go:102-117 (TestNew_ArchiverView), against internal/view/view.go:142", + "issue": "Nothing pins `New()[\"archiver\"].Name == \"archiver\"`. Step 1 of the task states the field is load-bearing ('must equal the map key; it is also the report type string and the tar entry prefix'), and top/config_view.go:348 and :398 write the working view back into the registry with `config.views[config.view.Name] = config.view`. If Name and the map key ever diverge, the TUI silently writes column widths, filters and sort state into a phantom key while the rendered view keeps the stale copy — no error, no panic, just settings that do not stick. Litmus: change Name to \"archive\" and TestNew, TestNew_ArchiverView, TestViews_Configure, TestView_VersionOK and Test_filterViews all stay green, because every one of them looks the view up by map key. No test in the repository asserts key==Name for any view.", + "recommendation": "Add `assert.Equal(t, \"archiver\", archiver.Name)` to TestNew_ArchiverView, or adopt the registry-wide TestNew_ViewMapInvariants above, which asserts `k == v.Name` for all 28 entries and prevents the same omission on the next view.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "internal/view/view_test.go:102-117 (TestNew_ArchiverView) and :219, :237 (TestViews_Configure)", + "issue": "The static QueryTmpl seed on the archiver entry is untested. AC bullet 1 lists `QueryTmpl: query.PgStatArchiverDefault` among the fields New() must return, but TestNew_ArchiverView does not assert it, and the two TestViews_Configure assertions that do check QueryTmpl run *after* Configure() has reassigned it from SelectStatArchiverQuery — so they pin the selector's output, not the seed. Litmus: set the seed to query.PgStatWALPG14 and every test stays green. Impact is limited (Configure is called on the top and record paths before any query runs), which is why this is minor rather than major, but the field is an AC item covered by nothing.", + "recommendation": "Add `assert.Equal(t, query.PgStatArchiverDefault, archiver.QueryTmpl)` to TestNew_ArchiverView, matching how TestNew_StatIOView-style guards pin the pre-Configure defaults for the other version-aware views.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/view/view_test.go:116", + "issue": "Msg is pinned by substring only (`assert.Contains(t, archiver.Msg, \"archive_mode=on\")`), while AC bullet 1 requires `Msg == \"Show archiver statistics (requires archive_mode=on)\"` verbatim (Decision 5). The prefix 'Show archiver statistics' is unpinned: a typo or a reworded prefix ships silently, even though this string is what the operator reads in the cmdline when switching to the screen. The sibling guards for the same class of view use full equality (TestNew_BgwriterView:96, TestNew_ReplslotsView:82); only TestNew_StatIOTimeView uses Contains, and there the task did not demand a verbatim Msg.", + "recommendation": "Replace with `assert.Equal(t, \"Show archiver statistics (requires archive_mode=on)\", archiver.Msg)`. Full equality is a strict superset of the substring guarantee, so it still satisfies the 'Msg substring' mutation check (dropping archive_mode=on still reddens it) while also covering the AC's verbatim requirement.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "record/record_test.go:139-145 (Test_filterViews) and internal/view/view_test.go:297-303 (TestView_VersionOK)", + "issue": "Both tables assert only aggregate integers; no assertion names the archiver view. The recordability claim ('kept on >=PG14, dropped below') is therefore proved arithmetically rather than directly, and a compensating change — archiver silently dropped while another view's gate loosens by one — passes both tables. This is inherent to count-based tests and is partly compensated by TestNew_ArchiverView pinning MinRequiredVersion, but the task builds its whole safety argument on these two tables, and patterns.md explicitly warns about count tests that pass for the wrong reason.", + "recommendation": "In Test_filterViews, add an expectation of membership per row, e.g. a `wantArchiver bool` column with `if tc.wantArchiver { assert.Contains(t, v, \"archiver\") } else { assert.NotContains(t, v, \"archiver\") }` — true on the three >=PG14 rows, false on the four <=PG13 rows. This turns 'the count moved by one' into 'this specific view survived', and it reddens under both the NotRecordable:true and the dropped-MinRequiredVersion mutations for the right reason.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "record/record_test.go:148-152 and internal/view/view_test.go:306-316", + "issue": "Both count tables loop without t.Run and pass no msgAndArgs, so all seven rows share a single assert line. When a count regresses, the failure reports 'expected 19, actual 18' with no indication of which version or pgssSchema produced it — precisely the diagnostic a future contributor needs when a newly registered view moves several rows at once. The task's own instruction ('work the arithmetic per row rather than applying a blanket +1') is exactly the reasoning this output does not support.", + "recommendation": "Wrap each row in a named subtest — `t.Run(fmt.Sprintf(\"version/%d/pgss/%q\", tc.version, tc.pgssSchema), func(t *testing.T) { ... })` in Test_filterViews and `t.Run(fmt.Sprintf(\"version/%d\", tc.version), ...)` in TestView_VersionOK — or at minimum pass the row as msgAndArgs: `assert.Equal(t, tc.wantN, n, \"version=%d pgss=%q\", tc.version, tc.pgssSchema)`. internal/query/archiver_test.go already uses the subtest form for its version tables.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "internal/view/view_test.go:218-221 and :236-239 (archiver assertions in TestViews_Configure)", + "issue": "The acknowledged limitation is handled adequately in substance but not at the assertion site. These assertions cannot be reddened by removing `case \"archiver\":` from Configure() — New() already sets the identical QueryTmpl/Ncols/DiffIntvl, so the selector call changes nothing observable — and the task explicitly accepts that. Worth recording that they are NOT vacuous, however: if SelectStatArchiverQuery ever drifts from the static entry (say it returns Ncols 8), Configure overwrites the view and these assertions go red, so they are a real drift guard between internal/query/archiver.go and internal/view/view.go. The problem is only that the honesty lives in the task file: the in-test comment reads 'pg_stat_archiver is schema-stable, so the archiver layout is the same on every version', which a future reader can easily take as a claim that Configure's wiring is gated here. patterns.md ('Extract the decision out of the unreachable closure') asks for that caveat to be visible where the assertion is.", + "recommendation": "Extend the existing one-line comment in both arms, e.g.: '// Regression guard only: New() already sets these, so removing case \"archiver\" from Configure() cannot redden this. What it does catch is drift between SelectStatArchiverQuery and the static entry. The wal assertions above are the real wiring gate.' No assertion change needed.", + "litmusTestFailed": true + } + ], + "metrics": { + "filesReviewed": 3, + "litmusTest": { + "checked": 16, + "passed": 12, + "failed": 4 + }, + "coverageAssessment": "adequate", + "pyramidBalance": { + "unit": 5, + "integration": 0, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-05.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05.md similarity index 96% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-05.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05.md index 55c655f7..cebdd429 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-05.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-05.md @@ -1,5 +1,5 @@ --- -status: planned # planned -> in_progress -> done +status: done # planned -> in_progress -> done depends_on: ["01", "02"] # ID задач-зависимостей (строки: ["01", "02"]) wave: 2 # волна параллельного выполнения skills: [code-writing] # МАССИВ скиллов для загрузки @@ -120,8 +120,14 @@ and observed failing **against the current tree** (no `archiver` key) before `vi existing `case 190000:` arm assert `views["wal"]` has `QueryTmpl == query.PgStatWALPG19`, `Ncols == 8`, `DiffIntvl == [2]int{2, 6}`; in `case 140000:` assert `query.PgStatWALPG14`, `11`, `[2]int{2, 9}`. Add `views["archiver"]` (`query.PgStatArchiverDefault`, `Ncols == 9`, - `DiffIntvl == [2]int{0,0}`) to both arms. Red today on two counts: no `archiver` key exists (zero - value view → `Ncols == 0`), and nothing pins the wal layout at all. + `DiffIntvl == [2]int{0,0}`) to both arms. + + **Be honest about what each half proves.** The `archiver` assertions are red *today* only because the + view is not registered yet; once step 1 lands they pass, and they can never be reddened by removing + `case "archiver":` from `Configure()` — `New()` already sets the same `Ncols`/`DiffIntvl`, so the + selector call changes nothing observable here. They are a regression guard, not a proof that + `Configure()` is wired. The `wal` half is the real gate, and its mutation (below) is what makes it + one; it is green from the moment it is written, because task 02 has already landed in Wave 1. - `internal/view/view_test.go::TestNew` — total view count `27` → **28**. Red until the entry is added. The trailing comment "27 is the total number of views have to be returned" moves with it. - `internal/view/view_test.go::TestView_VersionOK` — rows at version ≥ 140000 gain one: @@ -178,8 +184,9 @@ looking. - [ ] `TestViews_Configure` gained `wal` assertions in its `case 190000:` arm (`query.PgStatWALPG19`, `Ncols 8`, `DiffIntvl {2,6}`) and its `case 140000:` arm (`query.PgStatWALPG14`, `Ncols 11`, `DiffIntvl {2,9}`), plus `archiver` assertions - (`query.PgStatArchiverDefault`, `Ncols 9`, `DiffIntvl {0,0}`) in both. This closes the gap task - 02 pointed here. + (`query.PgStatArchiverDefault`, `Ncols 9`, `DiffIntvl {0,0}`) in both. The `wal` half closes the + gap task 02 pointed here; the `archiver` half is a regression guard that cannot be reddened by a + `Configure()` mutation, and the criterion below is the one that gates the wiring. - [ ] Mutation check: reverting task 02's PG 19 branch in `SelectStatWALQuery` (so it returns the PG 18 layout `7 / {2,5}` at 190000) turns `TestViews_Configure` red. If it stays green the wiring is still unpinned. diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-code-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-code-reviewer-review-round2.json new file mode 100644 index 00000000..352a3a58 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-code-reviewer-review-round2.json @@ -0,0 +1,106 @@ +{ + "reviewer": "dev-code-reviewer", + "round": 2, + "status": "approved_with_suggestions", + "summary": "All three round-1 minor findings are closed, and the keybindings() split that closes dev-test-reviewer's MAJOR finding is the minimum change that could have closed it: keybindings() is now eleven lines of registration, keybindingsList is a pure slice literal with no branching, no signature was threaded and no parameter added. I re-ran the round-1 mutation set plus five new mutations aimed specifically at the split; every one turns exactly the named test red, and the round-1 panic finding is verifiably fixed — deleting the case \"wal\" arm now reports `--- FAIL: Test_switchViewTo/26_` as a named row instead of taking the binary down. The single round-1 major (menuSelect at 102 lines) is carried forward unchanged as acknowledged, deliberately deferred tech debt; no new critical or major finding.", + "criticalIssues": [], + "suggestions": [ + { + "file": "top/menu.go", + "line": 150, + "severity": "major", + "category": "maintainability", + "suggestion": "CARRIED FORWARD FROM ROUND 1, NO ACTION EXPECTED IN THIS TASK. menuSelect still spans 102 lines (150-251) — six near-identical nested switches whose only per-branch content is a cursor-index -> view-name mapping plus one printCmdline. The collapse shape remains a package-level map[menuType][]string resolved as `names := menuViews[t]; if cy < 0 || cy >= len(names) { cy = 0 }; viewSwitchHandler(app.config, names[cy]); printCmdline(...)`, leaving menuConf and menuNone as the only explicit cases. I flagged it in round 1 as explicitly NOT a request to change this task (the task file mandates 'Copy, do not improvise' so the diff stays reviewable at a glance, and the repetition is pre-existing — this branch added the tenth to twelfth of it). It has since been recorded in the decisions log as tech debt, which is exactly the disposition round 1 asked for. Re-listing it only so the count in this report reconciles with round 1; it does not block this task and I am not re-escalating it on the >100-line rule, because the function was already ~90 lines before the diff touched it.", + "benefit": "Brings the dispatcher back under the project's function-length norm and turns six parallel switches into one table, so the next menu costs a data row rather than a branch.", + "optional": true, + "carriedForward": true, + "disposition": "accepted as tech debt by the team; verified present in the decisions log" + }, + { + "file": "top/keybindings_test.go", + "line": 30, + "severity": "minor", + "category": "testing", + "suggestion": "boundHandler runs a handler built by a FRESH keybindingsList(app) call, not the closure gocui actually holds — keybindingsList constructs ~60 new closures on every invocation. That is sound today because every row is a constructor that only captures app/config/db and has no per-instance state, so the closure under test is behaviourally identical to the registered one; I verified the whole table constructs cleanly against an app with a nil db and a zero Gui. What it means is that these tests rest on an unstated invariant — 'keybindings() registers exactly the rows keybindingsList returns'. Mutation M9 (truncating the registration loop to `[:1]`) does turn Test_keybindingsWAL red, so the invariant is guarded for 'W' specifically; it is not guarded generally, and gocui exposes no way to enumerate registered bindings to guard it generally. Nothing to change now — worth one sentence in keybindingsList's doc comment so the next person adding a stateful handler constructor knows the tests would not notice the divergence.", + "benefit": "Makes the assumption the new test seam rests on explicit at the seam, rather than something a reader has to reconstruct from two files.", + "optional": true + }, + { + "file": "top/keybindings.go", + "line": 18, + "severity": "minor", + "category": "testing", + "suggestion": "`app.ui.InputEsc = true` moved in this diff (it used to sit after the table literal, it now opens the function) and no test observes it — I deleted the line and the whole suite stayed green (mutation M8). The move is provably behaviour-neutral: it sits in the same function, before the same SetKeybinding loop, and nothing between the two reads InputEsc, so the ordering relative to every registration is unchanged. The gap is pre-existing, not introduced here, and Esc handling for the menu and dialog views depends on the flag. Given that this feature's discipline has been to pin every user-visible behaviour rather than rely on review attention, a one-line `assert.True(t, app.ui.InputEsc)` in newKeybindingsApp's caller would close it for the cost of the line. Out of this task's mandate — flagging because the diff moved the line and so put it briefly in scope.", + "benefit": "The one line in keybindings() that no test observes stops depending on nobody deleting it during a future refactor of the same function.", + "optional": true + }, + { + "file": "top/keybindings_test.go", + "line": 104, + "severity": "minor", + "category": "maintainability", + "suggestion": "The receive idiom — buffered `received` channel, a goroutine draining the unbuffered viewCh, a select with a one-second time.After fallback — is now written out twice, here and in Test_menuSelectWAL (top/menu_test.go:349-360), character for character apart from the failure message. Both copies are correct and both carry the same intentional-goroutine-leak note. A `receiveView(t, config) string` helper in one of the two files would remove the second copy and give the timeout a single place to live. Marginal at two copies; worth doing at three, which is roughly when the next screen group lands.", + "benefit": "One place to change the timeout and one place for the leak note, instead of two copies drifting.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 9, + "criticalIssuesCount": 0, + "majorIssuesCount": 1, + "newMajorIssuesCount": 0, + "minorIssuesCount": 3, + "testCoverageAssessment": "excellent" + }, + "round1FindingsDisposition": [ + { + "finding": "MAJOR — menuSelect 102 lines / six parallel switches", + "status": "deferred, as round 1 explicitly allowed", + "note": "Recorded in the decisions log as tech debt. Round 1 stated 'NOT a request to change this task'; the mandate 'Copy, do not improvise' stands. Carried forward, not re-escalated." + }, + { + "finding": "MINOR — wg.Wait() outside the t.Run closure in Test_switchViewTo", + "status": "FIXED and verified", + "note": "wg.Wait() is now the closure's last statement (top/config_view_test.go:647-651) with a comment naming the failure mode. Re-verified by deleting the case \"wal\" arm: output is now `--- FAIL: Test_switchViewTo (0.00s)` / ` --- FAIL: Test_switchViewTo/26_ (0.00s)` — the row is named and the binary does not panic. Round 1 reproduced a panic on the same mutation." + }, + { + "finding": "MINOR — menuWAL title pinned by nothing", + "status": "FIXED, and more strongly than asked", + "note": "Round 1 suggested a wantTitle column on Test_selectMenuStyle. The chosen fix pins the title through the real 'W' path instead (Test_keybindingsWALOpensMenu asserts app.config.menu.title, the items slice, AND the created menu view's Title), so the assertion covers menuOpen's window construction too, not just selectMenuStyle's return value. Rewording the title turns it red (mutation M3); swapping the two item strings turns it red (mutation M6) — the latter was not covered at all in round 1." + }, + { + "finding": "MINOR — lower-case 'w' binding had no guard", + "status": "FIXED, and more strongly than asked", + "note": "Round 1 suggested a symmetric DeleteKeybinding probe, which would have proven only that 'w' is still registered. Test_keybindingsWalCycles runs the bound handler from the wal screen and expects \"archiver\" on viewCh, so it proves 'w' is registered AND that it carries the cycle. Deleting the 'w' row turns it red (M2); so does deleting the switchViewTo case \"wal\" arm (M4), giving the dispatch arm a second independent guard." + } + ], + "verification": { + "commandsRun": [ + "gofmt -l top/ — clean", + "go vet ./top/... — clean", + "golangci-lint run ./top/... — 0 issues", + "go test ./top/ -run 'Test_walNextView|Test_switchViewTo|Test_selectMenuStyle|Test_menuSelectWAL|Test_keybindings|Test_helpTemplate' -count=1 -v — all PASS, including the three new subtests", + "go test ./top/ -run 'Test_keybindings|Test_menuSelectWAL|Test_switchViewTo' -count=3 -race — ok, so the new tests are repeatable and race-free" + ], + "mutationsReRunByReviewer": [ + "M1 'W' bound to menuOpen(menuStatIO,...) -> Test_keybindingsWALOpensMenu red (this is the exact hole dev-test-reviewer found; confirmed closed)", + "M2 'w' row deleted from the table -> Test_keybindingsWalCycles red (boundHandler's t.Fatalf names the missing key)", + "M3 menuWAL title reworded -> Test_keybindingsWALOpensMenu red", + "M4 case \"wal\" arm deleted from switchViewTo -> Test_switchViewTo/26_ red BY NAME plus Test_keybindingsWalCycles red; no panic (round-1 finding confirmed fixed)", + "M6 menuWAL item strings swapped -> Test_keybindingsWALOpensMenu red (new coverage, not present in round 1)", + "M7 'W' row moved from view \"sysstat\" to view \"menu\" -> Test_keybindingsWAL (both subtests) and Test_keybindingsWALOpensMenu red", + "M8 app.ui.InputEsc = true deleted -> NOTHING red (see minor finding; pre-existing gap, behaviour-neutral move)", + "M9 registration loop truncated to keybindingsList(app)[:1] -> Test_keybindingsWAL red (the table-to-registration invariant is guarded for 'W')", + "baseline for the same filtered set on the untouched copy — ok; the full-package run also trips Test_getQueryReport, which needs a live PostgreSQL fixture and is unrelated to this task" + ], + "crossFileChecks": [ + "key.key is declared `any` (top/keybindings.go:12) and boundHandler takes `k any` comparing with ==; rune literals box to int32 on both sides and gocui.Key values to gocui.Key, so lookups match. A type-mismatched probe would fail loudly via t.Fatalf, never silently pass", + "keybindingsList constructs every row against an app with nil db and a zero Gui without panicking — postgresProps is a value struct (top/top.go:46), not a pointer, so app.postgresProps.ExtPGSSSchema and .VersionNum are safe on a bare &app{}", + "keybindings() sets InputEsc before the loop; nothing between the assignment and SetKeybinding reads it, so the relocation is ordering-neutral", + "the two new tests build their own &app{} and never call keybindings(), so they register nothing into gocui and cannot perturb Test_keybindingsWAL's destructive DeleteKeybinding probes", + "config.viewCh is unbuffered (top/config.go:57); the drain goroutine is started before the handler runs in both new tests, so the handler's send has a receiver and cannot deadlock", + "app.config.views[\"wal\"] and the \"archiver\" view both resolve (internal/view/view.go), so Test_keybindingsWalCycles' start state and expectation are real view names", + "keybindingsList is 71 lines but is a single slice literal with cyclomatic complexity 1 and no control flow; the >50-line rule is not applied to it — the split REDUCED the largest function here (keybindings() was ~85 lines before, it is 11 now), and round 1 did not flag the pre-split function either. Recorded so the dismissal is visible rather than silent" + ] + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-code-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-code-reviewer-review.json new file mode 100644 index 00000000..af5bcefb --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-code-reviewer-review.json @@ -0,0 +1,82 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "The navigation layer reproduces the pg_stat_io (j/J) precedent faithfully across all five pieces: walNextView, the switchViewTo dispatch arm with the Decision 6 comment, menuWAL (iota placement inside the menu group, style branch, select branch), the 'W' keybinding and the three help lines — all matching the user-spec text word for word. I independently re-ran the mutation gate on a throwaway copy of the tree: nine of ten mutations turn exactly the named test red, and the tenth (deleting the case \"wal\" arm) fails the binary but through a pre-existing Test_switchViewTo harness flaw that panics instead of naming the row. No critical issues; one threshold-crossing maintainability finding in menuSelect and three optional test-pinning gaps.", + "criticalIssues": [], + "suggestions": [ + { + "file": "top/menu.go", + "line": 150, + "severity": "major", + "category": "maintainability", + "suggestion": "menuSelect now spans 102 lines (150-251) — the new menuWAL branch is what pushed it past the 100-line threshold. It is six near-identical nested switches whose only per-branch content is a cursor-index -> view-name mapping plus one printCmdline; only menuConf differs structurally (it calls editPgConfig and returns errors). The shape that would collapse it: a package-level map[menuType][]string of view names in menu order, resolved as `names := menuViews[t]; if cy < 0 || cy >= len(names) { cy = 0 }; viewSwitchHandler(app.config, names[cy]); printCmdline(...)`, leaving menuConf and menuNone as the only explicit cases. That also removes the copy-paste risk this branch was exposed to (a copied `menuType: menuStatIO` inside the menuWAL style would have produced a WAL-titled menu that dispatches to stat_io — Test_menuSelectWAL does catch it, but only because it builds the menu through selectMenuStyle). NOT a request to change this task: the task file mandates 'Copy, do not improvise' precisely so the diff stays reviewable at a glance, and the repetition is pre-existing. Worth a separate cleanup item, or an explicit decision to leave it.", + "benefit": "Brings the dispatcher back under the project's function-length norm and turns six parallel switches into one table, so the next menu costs a data row rather than a branch.", + "optional": true + }, + { + "file": "top/config_view_test.go", + "line": 631, + "severity": "minor", + "category": "testing", + "suggestion": "Pre-existing harness flaw, now load-bearing for this task's acceptance gate: the per-row assertion runs in a goroutine and `wg.Wait()` sits OUTSIDE the `t.Run` closure (line 631 vs the closure ending at 645). The unbuffered viewCh receive happens before switchViewTo returns, but the assert.Equal after it can run once the subtest has completed. On a green run nothing is called, so this is invisible; on a red row testify calls t.Errorf on a finished *testing.T and the binary dies with `panic: Fail in goroutine after Test_switchViewTo/26_ has completed`. I reproduced exactly that by deleting the `case \"wal\":` arm — the acceptance criterion 'that row turns red' is technically met (non-zero exit), but the output names no row and takes the rest of the run down with it. One-line fix: move `wg.Wait()` inside the closure, as its last statement. Out of this task's mandate; flagging so the next person to touch this table knows the failure mode is a panic, not a FAIL line.", + "benefit": "A failing row would report as `--- FAIL: Test_switchViewTo/NN` instead of a panic that hides which row broke.", + "optional": true + }, + { + "file": "top/menu_test.go", + "line": 23, + "severity": "minor", + "category": "testing", + "suggestion": "Test_selectMenuStyle asserts only len(items), so the menuWAL title (\" Choose WAL / archiver mode (Enter to choose, Esc to exit): \") is pinned by nothing — a reworded or truncated title ships silently, unlike every help-screen string in this feature, which is pinned word for word. The menu title is equally user-facing and is the only text the 'W' path shows before a selection. Cheap fix: extend the existing table with a `wantTitle` field, or add one assert.Equal on selectMenuStyle(menuWAL).title. (The item strings and the menuType field are already covered indirectly by Test_menuSelectWAL.)", + "benefit": "Closes the one user-visible string this task introduces that no test observes.", + "optional": true + }, + { + "file": "top/keybindings_test.go", + "line": 26, + "severity": "minor", + "category": "testing", + "suggestion": "Test_keybindingsWAL pins 'W' from both sides (exactly one binding, no other view claims it), but the acceptance criterion 'the existing 'w' binding line is byte-identical to before' is verified by diff review only. Since the whole point of this task is that 'w' changed meaning without changing its line, a symmetric `DeleteKeybinding(\"sysstat\", 'w', gocui.ModNone)` probe in the same style would make the lower-case half a regression guard too — the fresh-app-per-group idiom is already there and costs three lines.", + "benefit": "The 'w' hotkey — the primary entry point of the feature — stops depending on review attention to stay registered.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 9, + "criticalIssuesCount": 0, + "majorIssuesCount": 1, + "minorIssuesCount": 3, + "testCoverageAssessment": "excellent" + }, + "verification": { + "commandsRun": [ + "go vet ./top/... — clean", + "gofmt -l top/ — clean", + "golangci-lint run ./top/... — 0 issues", + "go test ./top/ -run 'Test_walNextView|Test_switchViewTo|Test_selectMenuStyle|Test_menuSelectWAL|Test_keybindingsWAL|Test_helpTemplate' — ok", + "go test -race -count=2 (same filter) — ok, so the new tests are repeatable and race-free" + ], + "mutationsReRunByReviewer": [ + "delete `case \"wal\":` from switchViewTo -> binary FAILs, but via panic in Test_switchViewTo (see minor finding), not a named row", + "walNextView `case \"wal\"` returns \"wal\" -> Test_walNextView + Test_switchViewTo red", + "swap menuWAL cursor 0/1 targets -> Test_menuSelectWAL red", + "delete the menu reset (selectMenuStyle(menuNone)) -> Test_menuSelectWAL red", + "drop one menuWAL item -> Test_selectMenuStyle red", + "remove the 'W' keybinding row -> Test_keybindingsWAL red", + "add a second 'W' keybinding row -> Test_keybindingsWAL red", + "restore `'w' WAL,` on the r line -> Test_helpTemplate_walEntry + Test_helpTemplate_replicationEntry red", + "reword the w,W description -> Test_helpTemplate_walEntry red", + "drop `archiver` from the Q caveat -> Test_helpTemplate_resetCaveat red" + ], + "crossFileChecks": [ + "internal/view/view.go:141 registers the `archiver` view with Msg 'Show archiver statistics (requires archive_mode=on)' — both names used by walNextView and the menuWAL branch resolve", + "menuOpen's pgssSchema guard only fires for menuPgss, so the empty third argument on the 'W' row is correct", + "the menuWAL iota entry sits above the blank line; moveUp/moveDown shift 6/7 -> 7/8, and `direction` values are compared only against themselves in moveCursor — no persistence or numeric literal depends on either type", + "the menu path reaches viewSwitchHandler (which calls liftPause), so the 016 pause contract holds for both new screens; no liftPause call was added to menu.go", + "exactly one printCmdline per path — the trailing one in switchViewTo, one at the end of the menuWAL branch", + "help.go's new Q caveat is factually right: resetStat runs query.ExecResetStats = `SELECT pg_stat_reset()`, which does not touch shared pg_stat_archiver", + "help text matches the user-spec (lines 180, 183) character for character", + "the corrected top/pause_test.go comment checks out: editPgConfig (top/pgconfig.go:70-74) does return early on !db.Local, so the narrowed claim is accurate and no new false claim replaced the old one" + ] + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-security-auditor-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-security-auditor-review.json new file mode 100644 index 00000000..717401e0 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-security-auditor-review.json @@ -0,0 +1,21 @@ +{ + "status": "approved", + "summary": { + "totalFindings": 1, + "critical": 0, + "major": 0, + "minor": 1 + }, + "findings": [ + { + "severity": "minor", + "category": "best-practice", + "title": "Deliberate per-sub-test goroutine leak in Test_menuSelectWAL (test-only, documented)", + "description": "Test_menuSelectWAL drives menuSelect over a zero-value &gocui.Gui{}. The branch ends with printCmdline -> g.Update, which on a zero-value Gui parks a goroutine forever on the nil userEvents channel; a second goroutine (`go func() { received <- <-app.config.viewCh }()`) also parks if the handler ever stops sending on viewCh. Three sub-tests leak up to six parked goroutines for the lifetime of the test binary. This is explicitly acknowledged in the test comment and matches the pre-existing accepted pattern of Test_showExtraCloseLifts (top/pause_test.go). No production code path is affected: printCmdline/writeCmdline in top/ui.go run against a live Gui with a real userEvents channel, and writeCmdline additionally returns early on a nil Gui.", + "location": "top/menu_test.go:70-112 (Test_menuSelectWAL)", + "impact": "None in production. In the test binary the leaked goroutines are bounded by the number of sub-tests and are freed at process exit; the only practical consequence is that a future goroutine-leak detector (goleak or similar) added to the top package would need an exemption for these tests rather than a real fix.", + "recommendation": "No change required. If a leak detector is ever introduced in this package, register the exemption alongside the existing Test_showExtraCloseLifts one instead of reshaping the test; the comment in menu_test.go already records the reason.", + "cwe": "CWE-404" + } + ] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-test-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-test-reviewer-review-round2.json new file mode 100644 index 00000000..98b902c8 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-test-reviewer-review-round2.json @@ -0,0 +1,46 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "needs_improvement", + "summary": "Round-1's major finding is genuinely closed, verified independently rather than taken on report: with 'W' rebound to menuOpen(menuStatIO, app.config, \"\"), Test_keybindingsWALOpensMenu now fails on three assertions at once (menuType 6 vs 5, the title, the item slice). The keybindingsList(app) extraction is the right shape — no parameter threading, keybindings() still owns InputEsc and the registration loop, and the table stays the single source. Both round-1 minors are closed with it: the menu title and both item strings are now pinned word for word by the test that drives menuOpen, and the lower-case 'w' has a behavioural guard. dev-code-reviewer's wg.Wait() relocation is correct — a failing row now names itself instead of panicking out of a finished subtest. Three minor items remain, each a one-liner, all confirmed by mutation on this working tree (every mutation reverted, tree clean, go test -race and go vet green). Ranked by what they actually cost: (1) the task's documented -run filter does not match Test_keybindingsWalCycles at all, so the guard the team believes it installed for the feature's primary hotkey is invisible to the exact command the Acceptance Criteria mutation protocol prescribes; (2) 'w' is guarded on which handler it carries but not on being registered — 'W' has both halves; (3) menuOpen's menuDraw call is still unasserted, one line short of closing. Round-1's redundant-row minor was declined with a valid reason (the row is mandated verbatim by the TDD Anchor) and is not re-raised.", + "findings": [ + { + "severity": "minor", + "category": "missing_coverage", + "location": "top/keybindings_test.go:100 (Test_keybindingsWalCycles) + 017-feat-wal-archiver-task-06.md frontmatter `verify:` and Verification Steps step 1", + "issue": "The new test is named Test_keybindingsWalCycles ('Wal'), but every documented verification command filters on 'Test_keybindingsWAL' ('WAL'). Go's -run is a case-sensitive regexp, so the test is silently excluded. Verified on this tree: `go test ./top/ -run 'Test_walNextView|Test_switchViewTo|Test_selectMenuStyle|Test_menuSelectWAL|Test_keybindingsWAL|Test_helpTemplate' -v` lists only Test_keybindingsWAL and Test_keybindingsWALOpensMenu — Test_keybindingsWalCycles never runs. This is not cosmetic: the Acceptance Criteria mutation protocol says \"apply the mutation, run the filtered command from step 1, confirm the named test fails\", and under that command deleting the {\"sysstat\", 'w', switchViewTo(app, \"wal\")} row is GREEN — nothing else in the filtered set observes the 'w' row (Test_keybindingsWAL probes only 'W'; Test_switchViewTo calls switchViewTo directly, never through the table). The red the team observed came from a broader filter than the one recorded. So an acceptance criterion (\"the host-side filtered run passes\") can be satisfied while the guard for the feature's primary entry point is not exercised, and it stays that way for every future contributor who runs the documented command.", + "recommendation": "Rename the test to Test_keybindingsWALCycles — that matches the existing 'Test_keybindingsWAL' filter, matches the capitalisation of its two neighbours in the same file, and needs no edit to the task file or to any recorded command. If the name is preferred as-is, widen the filter instead in BOTH places it appears (frontmatter `verify:` and Verification Steps step 1): replace `Test_keybindingsWAL` with `Test_keybindings`, which is what the whole file should be running anyway as the file grows. Then re-run the 'w'-row-deletion mutation under the corrected command and record that command in the decisions log verbatim, so the log matches what was actually run.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "top/keybindings_test.go:100 (Test_keybindingsWalCycles) + top/keybindings.go:20 (the registration loop)", + "issue": "Test_keybindingsWalCycles reads the table via boundHandler/keybindingsList and never touches the bindings gocui actually registered, so it covers WHICH handler the 'w' row carries but not that the row reaches SetKeybinding. Verified by mutation: adding `if k.viewname == \"sysstat\" && k.key == 'w' { continue }` to the loop in keybindings() leaves the whole filtered suite green while 'w' is dead in the running program. 'W' does not have this hole — Test_keybindingsWAL probes registration through DeleteKeybinding and Test_keybindingsWALOpensMenu probes the handler through the table, so both edges are covered. The asymmetry comes from round-1's finding offering the handler assertion as an alternative to the DeleteKeybinding probe (\"assert instead that...\"); with the keybindingsList refactor in place both are cheap and they cover different failures, so it should have been 'as well as', not 'instead of'. A whole-loop breakage is still caught (Test_keybindingsWAL would fail on its first delete) — what slips through is any per-row or conditional skip.", + "recommendation": "Add a third sub-test to Test_keybindingsWAL, mirroring the 'W' probe on a fresh app: `app := newKeybindingsApp(t)`; `assert.NoError(t, app.ui.DeleteKeybinding(\"sysstat\", 'w', gocui.ModNone))`; then `err := app.ui.DeleteKeybinding(\"sysstat\", 'w', gocui.ModNone)`; `require.Error(t, err)`; `assert.Equal(t, \"keybinding not found\", err.Error())`. Name it \"'w' registered on sysstat exactly once\". That closes the table-to-gocui edge for the lower-case key and makes the two halves symmetric; keep Test_keybindingsWalCycles unchanged, it covers the other half. Re-check by re-applying the `continue` mutation above and confirming the new sub-test is the one that goes red.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "top/keybindings_test.go:87-89 (Test_keybindingsWALOpensMenu) + top/menu.go:137 (menuOpen's menuDraw call)", + "issue": "Test_keybindingsWALOpensMenu is now the only test in the package that drives menuOpen, and its comment claims it pins \"the only text the 'W' path shows before a selection is made\". It does not, quite: the item assertion is on app.config.menu.items, which menuOpen copies from selectMenuStyle(m) — it never observes what was written into the menu window. Verified by mutation: replacing `err = menuDraw(v, s.items)` in menuOpen with a no-op leaves the whole filtered suite green, i.e. an empty menu window for all six menus is undetectable by any test in top/. menuDraw is pre-existing shared code this task did not touch, which is why this is minor rather than a gap the task created — but the test already holds `mv` two lines above and is one assertion short of closing it, and no other test in the package is positioned to.", + "recommendation": "Extend the existing assertion block, after `mv, err := app.ui.View(\"menu\")`: `buf := mv.Buffer()`; `assert.Contains(t, buf, \"pg_stat_wal\")`; `assert.Contains(t, buf, \"pg_stat_archiver\")`. Two lines, no new setup, and it turns the mutation above red. Then either trim the comment's claim about pinning what the 'W' path shows, or leave it — with these assertions it becomes accurate. Note for the author: menuDraw wraps the cursor row in \\033[30;47m…\\033[0m, so assert Contains on the bare item text rather than Equal on the whole buffer.", + "litmusTestFailed": true + } + ], + "metrics": { + "filesReviewed": 8, + "litmusTest": { + "checked": 20, + "passed": 17, + "failed": 3 + }, + "coverageAssessment": "adequate", + "pyramidBalance": { + "unit": 20, + "integration": 0, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-test-reviewer-review-round3.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-test-reviewer-review-round3.json new file mode 100644 index 00000000..12ad273a --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-test-reviewer-review-round3.json @@ -0,0 +1,38 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "passed", + "summary": "All three round-2 minors are closed, and closed for real — every claim was re-verified by mutation on this working tree rather than read off the report, with the tree restored byte-for-byte afterwards (git diff --stat matches the supplied diff, go vet green, filtered suite green). (1) The rename to Test_keybindingsWALCycles works: under the exact documented -run filter, `go test ./top/ -v` now lists Test_keybindingsWAL with all three sub-tests, Test_keybindingsWALOpensMenu and Test_keybindingsWALCycles. (2) Deleting the 'w' row turns BOTH Test_keybindingsWAL/lower-case_'w'_still_registered and Test_keybindingsWALCycles red under that filter — the acceptance-criteria hole from round 2 is gone. (3) The registration-loop skip mutation (`if k.viewname == \"sysstat\" && k.key == 'w' { continue }`) is now caught by the new sub-test alone, which is exactly the table-to-gocui edge it was written for. (4) The menuDraw no-op mutation fails Test_keybindingsWALOpensMenu on '\"\" does not contain \" pg_stat_wal\"' — the assertion reads the window, not the config, as intended. I also re-ran two mutations outside the round-2 set to check nothing regressed while these were being added: deleting `case \"wal\":` from switchViewTo goes red by name on Test_switchViewTo/26_ AND Test_keybindingsWALCycles, and swapping the menuWAL cursor rows goes red on two Test_menuSelectWAL sub-tests. dev-code-reviewer's boundHandler doc-comment addition is accurate — the freshly-constructed-closure caveat is the right thing to write down, and its 'equivalent as long as every row is a stateless constructor' condition holds for every row in the table today. Two minor items remain, both non-blocking and neither worth a fourth round: the 'w' probe implements only the first half of round 2's prescription (delete-succeeds) and not the second (second-delete-fails), so a DUPLICATE 'w' row is still invisible — and per gocui's execKeybindings, which runs every matching binding rather than the first, a duplicate would make 'w' advance the cycle twice per press, i.e. a visibly dead key; and the doc comment above Test_keybindingsWALCycles still opens with the old 'Test_keybindingsWalCycles' spelling. Both are one-line fixes that can ride along with any later edit to this file. The task's test layer is in good shape: every production line this task adds is individually mutation-guarded.", + "findings": [ + { + "severity": "minor", + "category": "missing_coverage", + "location": "top/keybindings_test.go:74-78 (Test_keybindingsWAL/\"lower-case 'w' still registered\") + top/keybindings.go:54", + "issue": "The new sub-test is half of round 2's prescription: it asserts the first DeleteKeybinding succeeds (proving the row reaches SetKeybinding — the edge that mattered, now genuinely closed) but omits the follow-up second-delete-must-fail assertion, so it proves 'at least once' rather than 'exactly once'. 'W' has both halves. Verified by mutation: duplicating the row into `{\"sysstat\", 'w', switchViewTo(app, \"wal\")}, {\"sysstat\", 'w', switchViewTo(app, \"wal\")}` leaves the entire documented filtered run GREEN. This is not a theoretical hole. gocui's execKeybindings (jroimartin/gocui@v0.5.0 gui.go:622) iterates all bindings and calls every handler whose key and view match — it does not stop at the first — so two rows means two viewSwitchHandler sends per keypress, wal -> archiver -> wal, and 'w' looks like a dead key to the user while every test stays green. Test_keybindingsWALCycles cannot see it either: boundHandler returns the first matching row from the table and runs it once. Probability is low (it needs a copy-paste of the row, which the file's shape makes unlikely) — hence minor, not major — but the guard costs three lines and the asymmetry with 'W' is now the only uneven thing left in this file.", + "recommendation": "Extend the existing sub-test rather than add a fourth, so the fresh app is reused: after the existing `assert.NoError(t, app.ui.DeleteKeybinding(\"sysstat\", 'w', gocui.ModNone))` add `err := app.ui.DeleteKeybinding(\"sysstat\", 'w', gocui.ModNone)`; `require.Error(t, err)`; `assert.Equal(t, \"keybinding not found\", err.Error())` — byte-identical to the 'W' sub-test above it with the rune swapped. Rename it to \"lower-case 'w' registered exactly once\" to match. Re-check by duplicating the 'w' row and confirming this sub-test is the one that goes red. No other file changes; the documented -run filter already matches.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "top/keybindings_test.go:112", + "issue": "The rename fixed the function name but not the doc comment that introduces it: line 112 still opens `// Test_keybindingsWalCycles is the lower-case half.` while the function on line 120 is Test_keybindingsWALCycles. Go's convention that a doc comment starts with the identifier it documents is what makes `grep Test_keybindingsWALCycles top/` find the explanation along with the code — right now it finds only the func line, and the paragraph explaining WHY this test exists (the 'w' row is byte-identical to before the feature, only its meaning changed) is what a future contributor most needs when deciding whether the test still earns its place. It is also the last live occurrence of the old spelling anywhere in the tree outside the round-2 review JSONs; the cross-reference on line 71 already uses the corrected name.", + "recommendation": "Change line 112 to `// Test_keybindingsWALCycles is the lower-case half.` — one word, nothing else in the comment needs touching.", + "litmusTestFailed": false + } + ], + "metrics": { + "filesReviewed": 9, + "litmusTest": { + "checked": 22, + "passed": 22, + "failed": 0 + }, + "coverageAssessment": "excellent", + "pyramidBalance": { + "unit": 22, + "integration": 0, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-test-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-test-reviewer-review.json new file mode 100644 index 00000000..9e05b6fc --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06-dev-test-reviewer-review.json @@ -0,0 +1,54 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "needs_improvement", + "summary": "Strong, behaviour-driven test set: Test_walNextView, Test_menuSelectWAL and the three help tests all assert on real results (view name arriving on viewCh, exact help description strings, menu reset), and the zero-value &gocui.Gui{} seam is used correctly rather than mocked around. One real hole remains: nothing in the suite ties the 'W' key to menuWAL. Test_keybindingsWAL probes only key uniqueness via DeleteKeybinding, and menuOpen is deliberately left unexercised, so rebinding 'W' to menuOpen(menuStatIO, ...) leaves the whole filtered suite green — verified empirically during this review. That is a user-spec acceptance criterion with no regression guard, and menuOpen turns out to be fully drivable with the same zero-value Gui idiom the task already established. Three minor items: menu title/items text unpinned, two redundant Test_switchViewTo rows, and the surviving 'w' binding itself untested.", + "findings": [ + { + "severity": "major", + "category": "missing_coverage", + "location": "top/keybindings_test.go:28 (Test_keybindingsWAL) + top/menu_test.go:52 (Test_menuSelectWAL)", + "issue": "No test connects the 'W' key to the WAL menu. Test_keybindingsWAL asserts only that exactly one binding claims 'W' on \"sysstat\" and that no other view claims it — the handler behind the binding is never exercised. Test_menuSelectWAL bypasses the binding entirely by hand-building app.config.menu = selectMenuStyle(menuWAL). Verified by mutation during this review: replacing the row with {\"sysstat\", 'W', menuOpen(menuStatIO, app.config, \"\")} keeps the entire filtered suite green (`go test ./top/ -run 'Test_walNextView|Test_switchViewTo|Test_selectMenuStyle|Test_menuSelectWAL|Test_keybindingsWAL|Test_helpTemplate'` → ok). The user-spec AC \"'W' opens the WAL statistics menu\" therefore has no regression guard. The task file's justification (\"menuOpen is still not exercised … covered by Test_selectMenuStyle plus the stand run\") does not hold: menuOpen IS drivable with the very zero-value Gui idiom this task introduced — a throwaway probe run during this review passed, asserting config.menu.menuType == menuWAL, the exact items slice, the title, the created \"menu\" view's Title and its rendered buffer.", + "recommendation": "Preferred (closes both the binding edge and menuOpen in one test): apply the project's own 'extract the decision out of the unreachable closure' pattern — split the keys table out of keybindings() into `func keybindingsList(app *app) []key`, with keybindings() iterating the returned slice (no behaviour change). Then in Test_keybindingsWAL: locate the single entry with viewname \"sysstat\" and key 'W', register a \"sysstat\" view on a zero-value &gocui.Gui{}, call `require.NoError(t, entry.handler(g, nil))`, and assert `app.config.menu.menuType == menuWAL`, `app.config.menu.items == []string{\" pg_stat_wal\", \" pg_stat_archiver\"}`, `app.config.menu.title == \" Choose WAL / archiver mode (Enter to choose, Esc to exit): \"`, plus `v, err := g.View(\"menu\")` → `v.Title` equals the same title and `v.Buffer()` contains \"pg_stat_archiver\". Cheaper half-fix if the production refactor is unwanted: add Test_menuOpenWAL driving `menuOpen(menuWAL, config, \"\")(&gocui.Gui{}, nil)` with the same assertions — that pins the menu window and its text, leaving only the key→handler edge unguarded, and it also removes the now-inaccurate claim in Test_menuSelectWAL's comment.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "top/menu_test.go:24 (Test_selectMenuStyle, {menu: menuWAL, want: 2})", + "issue": "The only assertion on selectMenuStyle(menuWAL) is len(items) == 2. The menu title and the two item strings are user-facing text, and a typo in \" pg_stat_archiver\" or in the title ships silently. This is inconsistent with the standard the task sets for the other user-facing text in the same change, where help lines are pinned word for word. Test_menuSelectWAL catches only a wrong menuType field (it would route into the menuStatIO branch), not the strings.", + "recommendation": "Either fold the string assertions into the menuOpen test recommended above, or extend Test_selectMenuStyle with a dedicated case for menuWAL: `s := selectMenuStyle(menuWAL)`; `assert.Equal(t, menuWAL, s.menuType)`; `assert.Equal(t, \" Choose WAL / archiver mode (Enter to choose, Esc to exit): \", s.title)`; `assert.Equal(t, []string{\" pg_stat_wal\", \" pg_stat_archiver\"}, s.items)`. Keep the existing count row — it is what catches an item being dropped.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "redundant_testing", + "location": "top/config_view_test.go:628-629", + "issue": "Two of the three new Test_switchViewTo rows catch nothing that is not already caught elsewhere. {current: \"activity\", to: \"wal\", want: \"wal\"} is behaviourally identical to the pre-existing row :604 {current: \"sizes\", to: \"wal\", want: \"wal\"} — same default arm, same dispatch, only a different irrelevant starting screen. {current: \"archiver\", to: \"wal\", want: \"wal\"} passes unchanged when the `case \"wal\":` arm is deleted (walNextView's default returns \"wal\"), so relative to Test_walNextView's archiver→wal case it adds no failure mode. The in-file comment states this honestly, which is good practice, but a test row that documents its own vacuity is still a row to maintain.", + "recommendation": "Drop {current: \"activity\", to: \"wal\", want: \"wal\"} — :604 already covers the default arm through switchViewTo. Keep {current: \"wal\", to: \"wal\", want: \"archiver\"} (the only row that proves the dispatch case exists) and, if the cycle-back is wanted at integration level, keep the archiver row but shorten the comment to say it is table symmetry rather than a guard.", + "litmusTestFailed": true + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "top/keybindings_test.go:28 (Test_keybindingsWAL)", + "issue": "The task's acceptance criteria require the existing {\"sysstat\", 'w', switchViewTo(app, \"wal\")} row to stay byte-identical, but no test observes it. Deleting that row makes the WAL screen unreachable by hotkey and every test in the package still passes — Test_switchViewTo calls switchViewTo directly, never through the binding. The new keybindings_test.go is the first place in the package where this is cheap to assert.", + "recommendation": "Add a sub-test to Test_keybindingsWAL, mirroring the existing uniqueness probe on a fresh app: `assert.NoError(t, app.ui.DeleteKeybinding(\"sysstat\", 'w', gocui.ModNone))`, then a second delete returning the \"keybinding not found\" error — the lowercase 'w' cycle entry point is registered exactly once. If the keybindingsList refactor from the major finding is applied, assert instead that the 'w' entry's handler sends \"archiver\" on viewCh when app.config.view is the wal view.", + "litmusTestFailed": false + } + ], + "metrics": { + "filesReviewed": 5, + "litmusTest": { + "checked": 15, + "passed": 13, + "failed": 2 + }, + "coverageAssessment": "adequate", + "pyramidBalance": { + "unit": 15, + "integration": 0, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-06.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06.md similarity index 95% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-06.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06.md index f69a867a..aa690823 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-06.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-06.md @@ -1,5 +1,5 @@ --- -status: planned +status: done depends_on: ["05"] wave: 3 skills: [code-writing] @@ -110,8 +110,12 @@ from a zero-value one; `top/ui_test.go:406-425` already uses the zero-value idio 8. **Amend the stale prohibition in `top/pause_test.go`** (`:561-568`): the sentences claiming that *`menuSelect` itself is unreachable from a unit test* and that a Gui *"comes only from `gocui.NewGui`"* are wrong — a zero-value `&gocui.Gui{}` reaches every branch (see Description). - Correct exactly those sentences to say that the blocker is specific to the `menuConf` branch, whose - terminal call `editPgConfig` needs a live `*postgres.DB`. Nothing else in that comment block moves, + Correct exactly those sentences to say that the blocker is narrower than stated — and do **not** + replace one false claim with another: `editPgConfig` is *not* out of reach for want of a live + `*postgres.DB`, because `top/pgconfig.go` returns early on `!db.Local`, so the `menuConf` branch is + drivable with `&gocui.Gui{}` plus `&postgres.DB{Local: false}` — an idiom this package already uses + in `top/config_view_test.go`. What genuinely needs a real environment is only the local-DB editor + path beyond that early return. Nothing else in that comment block moves, and the deleted `Test_menuConfPathDoesNotLift` stays deleted — its problem was that it asserted on a config the callee never receives, which is untouched by any of this. @@ -233,7 +237,7 @@ test observed failing, and the mutation reverted. - [ ] The host-side filtered run `go test ./top/ -run 'Test_walNextView|Test_switchViewTo|Test_selectMenuStyle|Test_menuSelectWAL|Test_keybindingsWAL|Test_helpTemplate'` passes -- [ ] `go build ./cmd` (that is the main package — `cmd/pgcenter.go`; the Makefile builds it as +- [ ] `make build` (note: `go build ./cmd` fails — Go refuses to write an executable named `cmd` next to the `cmd/` directory; use `make build` or `go build -o /dev/null ./cmd` as a compile check) (that is the main package — `cmd/pgcenter.go`; the Makefile builds it as `go build … -o bin/pgcenter ./cmd`) and `go vet ./top/...` are clean - [ ] `make lint` reports no new findings in `top/` @@ -320,7 +324,7 @@ host, one full run in the image. `./internal/view/...` and `./record/...` ride along unchanged by this task — a failure there means accidental coupling was introduced. (The `GOROOT` path tracks the local toolchain; check `go env GOROOT` if the image errors out on it.) -4. Run `go build ./cmd` — exits 0. (`./cmd` is the main package, `cmd/pgcenter.go`; there is no +4. Run `make build` (note: `go build ./cmd` fails — Go refuses to write an executable named `cmd` next to the `cmd/` directory; use `make build` or `go build -o /dev/null ./cmd` as a compile check) — exits 0. (`./cmd` is the main package, `cmd/pgcenter.go`; there is no `./cmd/pgcenter` package.) 5. Run `go vet ./top/...` and `make lint` — no new findings. `golangci-lint` lives in `$(go env GOPATH)/bin`, which is not on the default PATH; without it `make lint` exits 127. @@ -388,8 +392,9 @@ what makes it work is that `gocui.SetKeybinding` (`gui.go:249-259`) only appends what the user actually sees is verified on the stand (Task 10). - **The zero-value Gui is the seam — do not invent another one.** `menuSelect` and `keybindings` are both driven directly, as described in the Description and the TDD Anchor. What remains genuinely - out of reach is narrower and unrelated to this task: the `menuConf` branch, whose `editPgConfig` - needs a live `*postgres.DB` and an editor. Nothing here justifies threading new parameters through + out of reach is narrower and unrelated to this task: only the local-DB editor path inside + `editPgConfig`, past its `!db.Local` early return — the branch itself is drivable with + `&gocui.Gui{}` and `&postgres.DB{Local: false}`. Nothing here justifies threading new parameters through production signatures to make something observable, and nothing here justifies restoring the deleted `Test_menuConfPathDoesNotLift`, which asserted on a config its callee never received. - **`menuOpen` is still not exercised.** The test drives `menuSelect` with a hand-built menu view and diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-code-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-code-reviewer-review-round2.json new file mode 100644 index 00000000..6db73d56 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-code-reviewer-review-round2.json @@ -0,0 +1,42 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "Nothing regressed. report/describe.go and report/report.go are byte-identical to round 1 (git diff --stat: 18 added lines, 1 added line, one hunk each), and the only new work is 131 lines in report/report_test.go. The two helpers are sound and the three new tests are real gates, not decoration: I ran five fresh mutations against the round-2 assertions - wrong origin on the `last_failed` row, a duplicated `failed` row, a removed `(PG 19+)` annotation, a line appended after the Details URL, and the blank-cell clause stripped from `archived_age` alone - and each reddens exactly the intended test with the intended message, then restores clean (md5 verified). Cross-checked the pinned lists against the merged queries: the archiver column list matches query.PgStatArchiverDefault and archiverColumns (internal/query/archiver_test.go:23) name for name and order for order, and the origins match the query's source fields; the wal list matches the constant, and `wal_fpi_bytes AS \"fpi,KiB\"` sits right after `wal_fpi AS fpi` in PgStatWALPG19 as the row claims. go vet clean, gofmt clean on describe.go and report_test.go (report.go's only gofmt hit is the pre-existing :304 whitespace, out of scope), full `go test ./report/` green. Round-1 minors #1 and #3: I accept both non-applications - see the note in suggestions, no action wanted.", + "criticalIssues": [], + "suggestions": [ + { + "file": "report/report_test.go", + "line": 1318, + "severity": "minor", + "category": "maintainability", + "suggestion": "The origin check reads `strings.Fields(row)[2]`, which collapses every run of whitespace, so the tests pin exactly one tab per row - the one the marker `\"\\n- \"+name+\"\\t\"` demands right after the column name. Everything after that (the alignment padding between name/origin/description) is unverified: converting those separators to spaces keeps all five describe tests green while the printed table misaligns. The task treats the tabs as load-bearing and verifies them only by hand (`cat -A`, Verification Steps), which is exactly the kind of check that stops being run. If you ever want it pinned, one `assert.NotContains(t, row, \" \")` per row inside assertDescribeColumns would do it - though it would first need the two existing wal rows' trailing spaces (:145, :152) accounted for. Not worth doing now: gofmt does not touch raw string contents, so the realistic trigger is an editor setting, and the marker's tab already catches the common case.", + "benefit": "Names the one property of these constants that the new tests do not cover, so nobody assumes green means the table still lines up.", + "optional": true + }, + { + "file": "report/report_test.go", + "line": 1325, + "severity": "minor", + "category": "maintainability", + "suggestion": "The exact-count assertion `len(columns) == strings.Count(text, \"\\n- \")` is what caught my duplicated-row mutation, so it earns its place - but it also assumes the constant contains no prose line beginning with \"- \". That holds for both constants it is applied to and does not hold generally, which is incidentally the reason Test_describeActivityColumnOrder cannot simply be moved onto this helper (pgStatActivityDescription carries a caveats block, and its rows have no origin column worth listing for 17 columns). Round-1 minor #2 is therefore as closed as it usefully gets: the duplication is down to one inline copy, and folding that last one in would cost more than it saves. If the count ever does fire on a prose bullet, note the message says \"documents a row that is not in the column list\", which would point at the wrong thing.", + "benefit": "Records why the last inline copy stays inline, so a future consolidation pass does not rediscover the obstacle the hard way.", + "optional": true + }, + { + "file": "report/describe.go", + "line": 165, + "severity": "minor", + "category": "maintainability", + "suggestion": "Round-1 minors #1 (a pg_monitor clause on the `ready` row) and #3 (inline `(PG 19+)` vs the file's trailing `Note:` convention) were not applied, and I do not think that reasoning is wrong. On #1 the decisive argument is one I did not weigh in round 1: the wal screen's `waldir_size` row calls pg_ls_waldir(), the same privilege class with the same screen-wide failure mode, and its describe row carries no privilege clause either. Adding one to `ready` alone would make the file inconsistent in the other direction, and the fix that would actually help an operator is a runtime error message, not describe prose. On #3 the inline form is now pinned by Test_describeWALFPIVersionNote's HasSuffix assertion, which makes the split a tested decision rather than drift. Both closed; no action.", + "benefit": "Closes the two carried-over minors explicitly so they are not re-raised in a later round.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 3, + "criticalIssuesCount": 0, + "majorIssuesCount": 0, + "minorIssuesCount": 2, + "testCoverageAssessment": "excellent" + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-code-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-code-reviewer-review.json new file mode 100644 index 00000000..4b081673 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-code-reviewer-review.json @@ -0,0 +1,42 @@ +{ + "reviewer": "dev-code-reviewer", + "status": "approved_with_suggestions", + "summary": "The change does exactly what task 07 asks and nothing more: nine archiver rows in the query's emitted order, one map entry, one fpi,KiB row after fpi, and three test additions. I verified the column names/order against internal/query/archiver.go and internal/query/wal.go, verified tab alignment by rendering the output through `cat -A` (origin at column 16, description at column 40 on every row of both constants), verified the map key matches cmd/report/report.go:159 and internal/view/view.go:141, and re-ran all five mutations from the Acceptance Criteria plus one extra row-swap - each reddens the named test and only that test. Note for the record: Test_describeReport compares describeReport's output against the same constant, so it is tautological for row text - the two new order tests are what actually pin anything, and they pin names and order but not the origin-column spelling.", + "criticalIssues": [], + "suggestions": [ + { + "file": "report/describe.go", + "line": 165, + "severity": "minor", + "category": "maintainability", + "suggestion": "The `ready` row names pg_ls_archive_statusdir in the origin column but does not say the function is superuser/pg_monitor-only. Per Decision 4 (documented at internal/query/archiver.go:20-25) a role without pg_monitor loses the whole archiver screen, not just this cell - and describe is the only user-facing documentation of these columns (Decision 13 keeps the README out). A short clause on that row, e.g. '... (*.ready files); requires superuser or pg_monitor', would tell the operator why the screen errors out under a plain role. The file already carries privilege caveats of this kind for the activity screen (pgStatActivityDescription, pinned by Test_describeActivityCaveats).", + "benefit": "The one column in the feature with a privilege precondition documents it in the only place a user can read about it.", + "optional": true + }, + { + "file": "report/report_test.go", + "line": 1284, + "severity": "minor", + "category": "maintainability", + "suggestion": "Test_describeActivityColumnOrder, Test_describeArchiverColumnOrder and Test_describeWALColumnOrder are now three copies of the same 12-line loop (marker build, require.NotEqual presence, assert.Greater ordering), differing only in the constant and the column list. A shared helper - assertColumnOrder(t *testing.T, text string, columns []string) - would remove roughly 30 duplicated lines while keeping the three separately named tests the task asks for (a failure would still name the screen through the calling test's name; add t.Helper()). Flagged as future consolidation only: the task explicitly directed copying the shape of Test_describeActivityColumnOrder, so the duplication here is compliance, not drift.", + "benefit": "One place to fix if the marker convention ever changes, instead of three that can silently diverge.", + "optional": true + }, + { + "file": "report/describe.go", + "line": 149, + "severity": "minor", + "category": "readability", + "suggestion": "The version marker on the fpi,KiB row is an inline '(PG 19+)' parenthetical, while every other version note in this file is a trailing sentence after the table ('Note: started_by and mode are available since PG19.' at :262, :219, :328, :350, :491, :542, :583). No action is required - the tech-spec and the task text both settled on the inline annotation, and inline keeps the marker attached to the row it qualifies, which the trailing-Note form cannot do for a single column in a 12-row table. Recorded only so the next person editing this file knows the two styles coexist deliberately.", + "benefit": "Awareness of a deliberate style split; prevents a future 'cleanup' from moving the marker away from its row.", + "optional": true + } + ], + "metrics": { + "filesReviewed": 3, + "criticalIssuesCount": 0, + "majorIssuesCount": 0, + "minorIssuesCount": 3, + "testCoverageAssessment": "adequate" + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-security-auditor-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-security-auditor-review.json new file mode 100644 index 00000000..85567512 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-security-auditor-review.json @@ -0,0 +1,10 @@ +{ + "status": "approved", + "summary": { + "totalFindings": 0, + "critical": 0, + "major": 0, + "minor": 0 + }, + "findings": [] +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-test-reviewer-review-round2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-test-reviewer-review-round2.json new file mode 100644 index 00000000..eeabbab1 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-test-reviewer-review-round2.json @@ -0,0 +1,30 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "passed", + "summary": "All five round-1 findings are closed, and every closure was re-verified independently by mutation on an isolated copy of the tree (tar copy at scratchpad/mutcopy, baseline green, each mutation applied and reverted, final baseline green again). The seven mutations named in the handoff all redden the named test, and the sub-test isolation claim holds exactly: deleting the blank-cell clause from the last_archived row only reddens Test_describeArchiverBlankCells/last_archived and nothing else. The round-1 battery still reddens (map entry removed, map entry mis-wired to pgStatWALDescription, fpi,KiB row deleted, fpi,KiB moved before fpi, an archiver row deleted). Five further mutations I designed myself also redden: swapping the archived/failed origins, swapping the last_failed/failed_age rows, renaming ready to pending, deleting all four blank-cell clauses (four subtests red, one per clause), corrupting the WAL stats_age origin, and swapping two whole archiver rows at equal row count. The origins asserted in the tables are truthful rather than transcribed from the describe text: I cross-checked all nine archiver origins against the AS aliases in query.PgStatArchiverDefault (internal/query/archiver.go:38-47) and they match. Finding 5's cheap option is honest: archiverColumns does exist in internal/query/archiver_test.go:23-26 and its nine names are identical, in order, to the describe test's list, so the comment's grep-able pointer resolves to something real. report_test.go changed by addition only - no round-1 assertion was weakened or removed. gofmt clean, go vet clean, full report package green. The one residual gap is a deliberate boundary rather than a defect and does not warrant a third round.", + "findings": [ + { + "severity": "minor", + "category": "missing_coverage", + "location": "report/report_test.go:1309-1327 (assertDescribeColumns) / report/describe.go:164, 169", + "issue": "The description prose - the fourth column of each row - remains unguarded except where a test names it explicitly (the four blank-cell clauses and the (PG 19+) note). Two mutations I ran stay green: (a) truncating a row to name plus origin, `- failed\\tfailed_count\\t\\tTotal number of failed attempts to archive a WAL file` -> `- failed\\tfailed_count`, passes because `require.GreaterOrEqual(t, len(fields), 3)` admits exactly three fields, so a row that documents nothing satisfies the malformed-row guard; (b) rewriting the source row from \"Always has 'Archiver' value\" to \"Always has 'WAL' value\" passes, even though the literal is what gives the row its identity across samples (the query comment at internal/query/archiver.go:6 calls source \"the stable row identity\", and the SQL emits `SELECT 'Archiver' AS source`). Neither is a live defect - the text is correct today - and both are visible the moment anyone runs `report -d -W a`, which is why this is drift risk rather than a bug. It is also the same boundary the pre-existing pgStatWALDescription rows sit behind, so closing it here would leave the package inconsistent unless done for both.", + "recommendation": "Optional, and only if the cost is judged worth it - this does not need another round. Two one-line options, either independently: (1) tighten the malformed-row guard from `GreaterOrEqual(t, len(fields), 3)` to `Greater(t, len(fields), 3)` in assertDescribeColumns, which makes a row with no prose fail as malformed and costs nothing, since every row in both constants has prose today (verified: the suite stays green with the tightened bound); (2) pin the source literal the way the blank-cell clauses are pinned, `_, row := describeRow(t, pgStatArchiverDescription, \"source\"); assert.Contains(t, row, \"'Archiver'\", \"the source row must document the literal the query emits\")` - the mutation that must then redden is 'Archiver' -> 'WAL' in the source row. Do not go further than this: asserting whole prose strings per row would turn the test into a copy of the constant and would break on every wording fix without catching a behaviour change.", + "litmusTestFailed": false + } + ], + "metrics": { + "filesReviewed": 6, + "litmusTest": { + "checked": 7, + "passed": 7, + "failed": 0 + }, + "coverageAssessment": "excellent", + "pyramidBalance": { + "unit": 6, + "integration": 0, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-test-reviewer-review.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-test-reviewer-review.json new file mode 100644 index 00000000..ea19f58f --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07-dev-test-reviewer-review.json @@ -0,0 +1,62 @@ +{ + "reviewer": "dev-test-reviewer", + "status": "needs_improvement", + "summary": "No assertion added by task 07 is vacuous. All three tests were re-verified by mutation on an isolated copy of the tree: the map entry (mis-wire and deletion), row presence, row order, and tab separators each redden a named test. The gap is one dimension down: the tests pin WHICH rows exist and in WHAT order, but nothing pins WHAT a row says. Replacing the fpi,KiB origin wal_fpi_bytes with a bogus value, replacing pg_ls_archive_statusdir with pg_ls_waldir, dropping the (PG 19+) annotation, breaking the Details URL, or deleting the 'empty if nothing has been archived yet' wording all leave the whole describe suite green - and each of those is an explicit acceptance criterion of this task. The column set is also guarded in one direction only: a missing row reddens, an extra or duplicated row does not.", + "findings": [ + { + "severity": "major", + "category": "missing_coverage", + "location": "report/report_test.go:1284-1329 (Test_describeArchiverColumnOrder, Test_describeWALColumnOrder)", + "issue": "The origin column - the field name each described column comes from - is never asserted, and neither is the (PG 19+) annotation. Verified by mutation on a copy of the tree: (a) `wal_fpi_bytes` -> `wal_fpi_BOGUS` in the fpi,KiB row -> `go test ./report/ -run Test_describe` green; (b) `pg_ls_archive_statusdir` -> `pg_ls_waldir` in the ready row -> green; (c) removing `(PG 19+)` from the fpi,KiB row -> green. Two of these are literal acceptance criteria of task 07 (AC: \"an fpi,KiB row with origin wal_fpi_bytes ... annotated (PG 19+)\"), and the task's own rationale calls the pg_ls_archive_statusdir origin load-bearing: it is the only place a user learns why that one cell fails on a role without pg_monitor (Decision 4). The marker `\"\\n- \"+c+\"\\t\"` stops at the column name, so everything to the right of the first tab is unguarded text.", + "recommendation": "Turn the flat `columns []string` into a table of {name, origin} and assert the origin on the row the marker already located. Concretely, inside the existing loop: `line := pgStatArchiverDescription[pos+1:]; line = line[:strings.Index(line, \"\\n\")]; f := strings.Fields(line); require.Len(t, f, ...); assert.Equal(t, tc.origin, f[2], \"row %q documents the wrong origin\", tc.name)` - `strings.Fields` collapses the alignment tabs and f[0]==\"-\", f[1]==name, f[2]==origin for every row in both constants (no name or origin contains a space). Expected origins for archiver: source=\"-\", ready=\"pg_ls_archive_statusdir\", archived=\"archived_count\", last_archived=\"last_archived_wal\", archived_age=\"last_archived_time\", failed=\"failed_count\", last_failed=\"last_failed_wal\", failed_age=\"last_failed_time\", stats_age=\"stats_reset\"; for wal, at minimum fpi=\"wal_fpi\" and \"fpi,KiB\"=\"wal_fpi_bytes\". Add one line for the version annotation, in the style of the existing Test_describeActivityCaveats: `assert.Contains(t, pgStatWALDescription, \"wal_fpi_bytes\\tAmount of WAL generated by full page images, in KiB (PG 19+)\")` - or assert `strings.HasSuffix(line, \"(PG 19+)\")` for that row. Mutations that must then redden: the three listed in the issue.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "report/report_test.go:1294-1305, 1318-1328", + "issue": "The column set is bounded from below only. The loop proves every expected row is present and correctly ordered, but says nothing about rows that should not be there. Verified: appending `- bogus\\tbogus_col\\t\\tA column that does not exist in the query` to pgStatArchiverDescription -> suite green; appending a second `- failed` row -> green (strings.Index returns the first occurrence, so duplicates are invisible by construction). A describe text listing a column the query does not emit is exactly the kind of drift this test exists to catch, and it is the likelier direction of error for pgStatWALDescription, where the deliberate no-version-awareness contract invites future per-version rows.", + "recommendation": "Add one assertion per constant, after the loop: `assert.Equal(t, len(columns), strings.Count(pgStatArchiverDescription, \"\\n- \"), \"description documents a row that is not in the query\")` (9 for archiver, 12 for wal - both counts match the constants as they stand today). This also closes the duplicate-row hole. Mutations that must then redden: appending a bogus row, and duplicating any existing row.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "report/describe.go:174 / report/report_test.go:1284-1306", + "issue": "The trailing `Details:` URL is untested. Verified: rewriting the anchor to `#PG-STAT-BGWRITER-VIEW` leaves the suite green. The acceptance criteria name this URL explicitly (\"ends with the PG-STAT-ARCHIVER-VIEW docs URL\"), and since Decision 13 keeps the README out of scope, this line is the only pointer a user gets to upstream documentation for the screen.", + "recommendation": "One line in Test_describeArchiverColumnOrder (or a small Test_describeArchiverDetails): `assert.True(t, strings.HasSuffix(strings.TrimRight(pgStatArchiverDescription, \"\\n\"), \"https://www.postgresql.org/docs/current/monitoring-stats.html#PG-STAT-ARCHIVER-VIEW\"), \"description must end with the pg_stat_archiver docs URL\")`. HasSuffix rather than Contains, so a URL that survives but is no longer the closing line is also caught.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "missing_coverage", + "location": "report/describe.go:167-171 (last_archived / archived_age / last_failed / failed_age rows)", + "issue": "The blank-cell wording is untested. Verified: deleting every occurrence of \", empty if nothing has been archived yet\" leaves the suite green. Task 07 asks for this wording specifically (\"describe is where a user finds out a blank is not an error\"), and it is the user-facing half of Decision 3 - the four NULL-able columns render blank by design and there is no other place a user can learn that. The repo already has the idiom for pinning exactly this kind of prose: Test_describeActivityCaveats (report_test.go:1331+) asserts each caveat separately so that deleting any one of them reddens a named subtest.", + "recommendation": "Add a Test_describeArchiverCaveats modelled on Test_describeActivityCaveats: a subtest per column asserting the row carries its blank-cell clause, e.g. `assert.Contains(t, pgStatArchiverDescription, \"empty if nothing has been archived yet\")` for last_archived/archived_age and `\"empty if there were no failures\"` for last_failed/failed_age - scoped to the row, e.g. by extracting the located line as in finding 1 and asserting Contains on that line, so deleting the clause from one row only still reddens.", + "litmusTestFailed": false + }, + { + "severity": "minor", + "category": "anti_pattern", + "location": "report/report_test.go:1289-1292 (comment claims the list is query.PgStatArchiverDefault's column order)", + "issue": "The test comment says the list 'is the column order of query.PgStatArchiverDefault (internal/query/archiver.go)', but nothing in the test references that constant - it is a hand-copied snapshot. The invariant actually guarded is 'the description matches a list written in this file', not 'the description matches the query'. Today the two agree (I diffed them: internal/query/archiver_test.go:23-26 archiverColumns is identical, in order, to the describe test's list), so this is drift risk rather than a live defect: a future reorder of the aliases in archiver.go would redden internal/query's own test, be fixed there, and leave the describe text silently documenting a layout that no longer ships. The report package already imports internal/query (report/report.go:8), so the coupling is available if wanted.", + "recommendation": "Cheapest honest fix: in the comment, state that the list is a copy that must be kept in sync with internal/query/archiver_test.go's archiverColumns, and name that file - so an editor of the query has one grep-able pointer. Stronger fix, if the coupling is judged worth the SQL parsing: derive the expected order in the test from query.PgStatArchiverDefault with `regexp.MustCompile(` + \"`\" + ` AS \\\"?([a-zA-Z_,]+)\\\"?`\" + `).FindAllStringSubmatch(...)` and assert the describe rows appear in exactly that order; the mutation that must then redden is swapping two aliases in internal/query/archiver.go. Do not apply the stronger fix to pgStatWALDescription - that constant is deliberately a cross-version superset and has no single query to derive from.", + "litmusTestFailed": false + } + ], + "metrics": { + "filesReviewed": 6, + "litmusTest": { + "checked": 5, + "passed": 5, + "failed": 0 + }, + "coverageAssessment": "adequate", + "pyramidBalance": { + "unit": 3, + "integration": 0, + "e2e": 0, + "assessment": "healthy" + } + } +} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-07.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07.md similarity index 99% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-07.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07.md index aa36458a..182f6559 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-07.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-07.md @@ -1,5 +1,5 @@ --- -status: planned # planned -> in_progress -> done +status: done # planned -> in_progress -> done depends_on: ["04", "05"] # ID задач-зависимостей (строки: ["01", "02"]) wave: 3 # волна параллельного выполнения skills: [code-writing] # МАССИВ скиллов для загрузки diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-08.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-08.md similarity index 98% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-08.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-08.md index bd9e2166..0e37bfde 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-08.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-08.md @@ -1,5 +1,5 @@ --- -status: planned # planned -> in_progress -> done +status: done # planned -> in_progress -> done depends_on: ["05"] # ID задач-зависимостей (строки: ["01", "02"]) wave: 3 # волна параллельного выполнения skills: [code-writing] # МАССИВ скиллов для загрузки @@ -108,9 +108,6 @@ that claims otherwise — see the "no A5" item below. ## TDD Anchor - - -Тесты, которые нужно написать ДО реализации. Пишем → запускаем → убеждаемся что падают → пишем код → убеждаемся что проходят. This task *is* tests, so "red first" means something specific: write each subcase with its assertions **before** generating its golden, run it, and confirm it fails on the missing golden file and on the @@ -333,7 +330,7 @@ those two. - Step 6 — `make lint` on the host: clean. - Step 7 — confirm the pre-existing `Test_app_doReport` wal case - (`report/report_test.go:73-77`, driven by the legacy PG13-era tar) is still green and its golden + (`report/report_test.go:73-77`, driven by the legacy PG 14beta1 tar) is still green and its golden unchanged — the PG 19 branch must not reach a ~PG13 recording. ## Details diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-09.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-09.md similarity index 92% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-09.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-09.md index e91de10c..e7e9ae26 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-09.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-09.md @@ -1,5 +1,5 @@ --- -status: planned # planned -> in_progress -> done +status: done # planned -> in_progress -> done depends_on: ["04"] # ID задач-зависимостей (строки: ["01", "02"]) wave: 3 # волна параллельного выполнения skills: [documentation-writing] # МАССИВ скиллов для загрузки @@ -233,3 +233,22 @@ inventing one, which is exactly why the new file must look like its neighbours. спеки пишут новое значение backlog как `0 B`, а программа печатает `0` — в release notes формулировка прозой, литерал не цитируется - [ ] Обновить user-spec/tech-spec если что-то изменилось + + +## Scope added during Wave 1 (2026-08-06) + +Task 04 discovered that **the flag help users actually see is not cobra's.** `printReportHelp()` in +`cmd/help.go` is installed via `SetHelpTemplate`/`SetUsageTemplate` and fully overrides cobra's flag +usage, so the `StringVarP` description task 04 updated never reaches `pgcenter report --help`. +`cmd/help.go:170` still reads `-W, --wal show pg_stat_wal statistics` — describing a boolean flag +with no selectors. No task in this feature covered `cmd/help.go`: every `help.go` reference in the +tech-spec and the other tasks means `top/help.go`, the TUI screen. + +**This task now also updates `cmd/help.go:170`**, following the `SELECTOR` pattern the neighbouring +`-D`, `-X` and `-P` lines already use. Without it, the release notes this task writes would point at +help text nobody sees. + +**Also required in the release notes, measured during Wave 1:** the failure exits with **code 0**. +`pgcenter report -W -f dump.tar` prints `report type is not specified, quit` and returns success, so a +wrapper using `|| alert` will not fire and will keep an empty output file. Say so explicitly — this is +the difference between a loud failure and a silent one for anything scripted. diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-10.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-10.md similarity index 96% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-10.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-10.md index 864896da..186dd918 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-10.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-10.md @@ -67,7 +67,7 @@ measurement (Decision 9), which this task *decides* but does not implement. `Test_selectMenuStyle`, `Test_switchViewTo` — and confirm each carries a new **correct** number, not a deleted assertion, a loosened comparison or a removed row. - Confirm the stand address with the project owner at the start of the run. The recorded address - (`pgpro@10.128.31.96`) has a 24h TTL from 2026-08-05 and is very likely stale. **If the stand has + (`pgpro@10.128.28.194`) has a 24h TTL from 2026-08-05 and is very likely stale. **If the stand has expired, say so and ask for a new one — do not silently skip the manual gate.** There is no fallback: the archiving, navigation, narrow-terminal and cost criteria cannot be satisfied any other way, and marking them PASS without a capture is the failure mode this task exists to prevent. @@ -312,7 +312,7 @@ per-feature copy, mirror it as `017-feat-wal-archiver-qa-report.json` in the fea the CI image — a missing cluster is an environment blocker, and calling it a criterion FAIL would be wrong. - A stand is required and its address must be re-confirmed with the project owner. The recorded - `pgpro@10.128.31.96` has a 24h TTL from 2026-08-05. + `pgpro@10.128.28.194` has a 24h TTL from 2026-08-05. **Stand regimen (from `patterns.md` → «Driving the TUI on a remote test stand»):** 1. Confirm the address with the owner at the start of the run; do not record it afterwards. @@ -329,8 +329,7 @@ per-feature copy, mirror it as `017-feat-wal-archiver-qa-report.json` in the fea 6. Leave the stand as found — see the cleanup order in Edge cases. **Edge cases:** -- **The stand has expired.** Its TTL is 24h from 2026-08-05, and manual QA comes last. Ask for a new - one. Do **not** mark the manual criteria PASS by inference and do **not** quietly drop them — the +- **The stand has expired.** Its one. Do **not** mark the manual criteria PASS by inference and do **not** quietly drop them — the tech-spec Risks table already names this as the scenario in which the pipeline must not silently skip the gate. NOT VERIFIABLE with the reason is the honest outcome if no stand can be had. - **`archive_mode` needs a restart, `archive_command` only a reload.** Plan the scenario order around @@ -416,15 +415,7 @@ accepted by the project owner at feature acceptance. - [ ] Обновить user-spec/tech-spec если что-то изменилось -## Correction applied during task validation (2026-08-06) -**The verbose backlog renders `0`, not `0 B`.** Both specs and an earlier draft of this task quoted -`0 B` as the value shown when the `archive_status` directory is missing. The panel formats that field -through the project's size formatter, whose zero case returns a bare `0`. A QA gate checking for the -string `0 B` would report a false FAIL on correct behaviour. - -**The stand's TTL has lapsed.** The stand named in the specs (`pgpro@10.128.31.96`) was issued on -2026-08-05 with a 24-hour TTL, so it is gone by the time this task runs. Do NOT quietly skip the manual -half: ask the project owner for a fresh stand at the start of this task, and record in the QA report -which scenarios were executed and which were blocked waiting for one. The automated half — the full -suite inside the CI image, lint and vuln — does not depend on the stand and runs regardless. +> **Stand address updated 2026-08-06:** `pgpro@10.128.28.194`. The earlier one +> (`10.128.31.96`) expired. Re-confirm it is alive at the start of the run; if it is gone again, ask +> for a new one rather than skipping the manual half. diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch1.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch1.json new file mode 100644 index 00000000..bc7b547e --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch1.json @@ -0,0 +1,89 @@ +{ + "validator": "dev-reality-checker", + "feature_base": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver", + "tasks_checked": ["01", "03", "06"], + "status": "changes_required", + "round": 2, + "note": "Round 2. Re-validates the rewritten Tasks 01, 03, 06 against the tree. Claims already confirmed correct in round 1 are not restated; round-1 findings that the rewrite did NOT address are carried forward (marked carry-forward) so nothing is lost by overwriting the round-1 report.", + "findings": [ + { + "severity": "critical", + "category": "feasibility", + "task": "03", + "issue": "Task 03 and Task 01 now contradict each other about who owns the test-role NAMES. Task 03 says the roles belong to the helper — TDD Anchor: 'using the shared helper's **deny** role (granted nothing)'; Details: 'The SQL role names come from the helper too; do not invent parallel ones.' Task 01 specifies the opposite and is the owner of the file: the signature is `func SetupTestRole(db *DB, name string, pgMonitor bool) error` with a caller-supplied name, and Task 01 states 'The name is caller-supplied, so this task's roles (pgcenter_test_archiver_monitor, pgcenter_test_archiver_norole) and Task 03's roles stay distinct while sharing one implementation' plus 'Task 03 creates its own roles on the same clusters through the same helper; distinct names keep the two suites from mutating each other's grants.' The helper therefore exports no role names at all. An executor of Task 03 who follows its own text will look in internal/postgres/testing.go for role constants, find none, and be blocked: inventing names violates Task 03's Details, and adding constants to testing.go violates Task 03's own AC ('internal/postgres/testing.go is not modified here'). The two mutation ACs of Task 03 ('remove the GRANT pg_monitor from the role setup', 'add GRANT pg_monitor to the deny role') inherit the same ambiguity — role setup is now a call, not code this task owns.", + "fix": "Align Task 03 with Task 01: replace 'The SQL role names come from the helper too; do not invent parallel ones' with 'The helper takes the role name as a parameter (see its signature). This task passes its own names — pgcenter_test_overview_monitor and pgcenter_test_overview_norole — so its grants can never collide with Task 01's pgcenter_test_archiver_* roles.' Change 'using the shared helper's deny role' to 'calling the shared helper with pgMonitor=false and this task's deny-role name'. Restate the two role mutations as 'pass pgMonitor=false in the call' / 'pass pgMonitor=true for the deny role' so they name something this task actually owns." + }, + { + "severity": "major", + "category": "tdd", + "task": "01", + "issue": "The replacement mutation names a line that, by this task's own rules, does not exist. AC: 'Mutation — delete the `SET ROLE` line from `Test_StatArchiverQuery_PgMonitorRoleSucceeds`, so the test runs on the fixture superuser connection.' But another AC of the same task says 'both privilege tests call it — neither test open-codes CREATE ROLE / SET ROLE', and What-to-do says 'They must not open-code their own CREATE ROLE / SET ROLE SQL'. SET ROLE lives inside postgres.SetupTestRole (internal/postgres/testing.go), so the mutation's only applicable site is the helper. The mutation DOES discriminate when applied there — with SET ROLE removed the session stays `postgres`, so `current_user` != 'pgcenter_test_archiver_monitor' and pg_roles.rolsuper is true, and the guard reddens before the query runs (verified: pg_roles is world-readable, so the guard query itself succeeds under any role; pgx scans the `name`-typed current_user into a string) — but as written an executor either cannot find the line or inlines a SET ROLE into the test, breaking Decision 19.", + "fix": "Reword: 'Mutation — delete the `SET ROLE` statement from `postgres.SetupTestRole` (internal/postgres/testing.go): `Test_StatArchiverQuery_PgMonitorRoleSucceeds` and `Test_StatArchiverQuery_WithoutPgMonitorFails` both turn red on their own current_user / rolsuper guard, before either reaches the query. Task 03's privilege tests redden too — that is expected; revert immediately after observing it.' Optionally add the cheaper variant: delete the `SetupTestRole` call from the test body." + }, + { + "severity": "major", + "category": "tdd", + "task": "03", + "issue": "Same misplaced mutation as Task 01, plus a self-collision: 'Mutation: delete the `SET ROLE` statement from `Test_ArchivingBacklogQuery_PgMonitorRole`'. This task's tests contain no SET ROLE statement (AC: 'no inline CREATE ROLE/GRANT SQL in either package'; Details: 'this task's job is to call it and to RESET ROLE in a defer'). The statement lives in internal/postgres/testing.go, which this task's AC declares 'is not modified here'. The mutation is executable only as a temporary edit to a file the task says it does not touch, or by deleting the SetupTestRole call from the test — and neither is stated.", + "fix": "Reword to: 'Mutation: delete the `postgres.SetupTestRole(...)` call from `Test_ArchivingBacklogQuery_PgMonitorRole`, leaving the session as the fixture superuser -> the test must go RED on its own current_user / rolsuper guard.' If the helper-side variant is wanted, say explicitly that a mutation is a temporary edit reverted immediately and does not count as modifying testing.go." + }, + { + "severity": "major", + "category": "feasibility", + "task": "03", + "issue": "Carry-forward, not fixed by the rewrite: the Verification gate and the Implementation hint still cannot both be satisfied. Verification Steps: 'Grep the tree for the stale claims — no hit may remain: `grep -rn \"pg_ls_dir\" internal/`'. Implementation hints: the rewritten doc comment should say the function is pg_ls_archive_statusdir(), 'which superuser and pg_monitor can execute (unlike pg_ls_dir, which is superuser-only — worth one clause, since that is the bug being fixed and Decision 8 supersedes ADR [010])'. Confirmed against the tree: internal/ currently has 5 `pg_ls_dir` hits (internal/query/overview.go:95, :97, :102; internal/stat/postgres.go:288; internal/query/overview_test.go:124), all in files this task owns — so the bare-token grep is reachable, but only by dropping the one clause the hints require. The AC one screen earlier already states the correct, narrower rule ('no comment in the tree still CLAIMS pg_ls_dir requires pg_monitor/superuser').", + "fix": "Narrow the grep to the stale claim, matching the AC wording: `grep -rniE \"pg_ls_dir (requires|needs)|pg_ls_dir, which requires\" internal/` and `grep -rn \"has pg_monitor\" internal/`. Keep the explanatory clause." + }, + { + "severity": "major", + "category": "hallucination", + "task": "06", + "issue": "Step 8 replaces one over-general untestability claim with another one that is false in the same way. The two sentences it targets do exist verbatim at top/pause_test.go:561-568 ('menuSelect itself is unreachable from a unit test...', '...a live one comes only from gocui.NewGui'), and amending them does not conflict with what that comment block is for (the block explains why the menuConf pause-preservation has no test and is protected structurally by editPgConfig's signature — that argument stands). But the prescribed replacement — 'the blocker is specific to the menuConf branch, whose terminal call editPgConfig needs a live *postgres.DB' — is not true. top/pgconfig.go:70-74 opens with `if !db.Local { printCmdline(...); return nil }`, so `editPgConfig(&gocui.Gui{}, &postgres.DB{Local: false}, ...)` returns nil without a query, an editor, or a terminal; printCmdline on a zero-value Gui only parks a goroutine (top/ui.go:541-545 + gocui Update). The whole menuConf branch is therefore drivable today with `&gocui.Gui{}` + `&postgres.DB{Local: false}` + a registered 'sysstat' view — and that exact remote-DB idiom is already used in this package (top/config_view_test.go:256-259, :294-297). Writing this into the tree repeats the failure mode Task 06's own hint (last bullet of Implementation hints) warns about.", + "fix": "Make the replacement precise and verifiable: the reachable part is the branch itself (zero-value Gui + `&postgres.DB{Local:false}` early-returns); what stays out of reach is the local-DB path — a real `query.GetSetting` lookup plus an exec'd $EDITOR. And keep the reason no test is written: editPgConfig receives no *config, so a pause assertion around it observes state the callee cannot touch (the same defect the deleted Test_menuConfPathDoesNotLift had). If the executor is unsure, the hint applies to them too — run the throwaway test before writing the claim." + }, + { + "severity": "minor", + "category": "tdd", + "task": "06", + "issue": "The goroutine-leak note in Test_menuSelectWAL undercounts. It says 'printCmdline calls g.Update, which spawns a goroutine that parks forever on the zero Gui's nil userEvents channel — one per case'. writeCmdline (top/ui.go:541-...) also arms a 2s timer whenever `arm && msg != \"\"`, and the archiver/wal views have a non-empty Msg, so each case leaves the Update goroutine plus a timer goroutine that fires 2s later and spawns a second parked Update goroutine. Harmless (no panic, nothing is dereferenced), but the comment should describe what a leak detector would actually see.", + "fix": "Say 'each case leaves the parked Update goroutine and, because the Msg is non-empty, the 2-second cmdline-clear timer and its own parked Update — the same intentional class as Test_showExtraCloseLifts'." + }, + { + "severity": "minor", + "category": "tdd", + "task": "03", + "issue": "Carry-forward, not fixed: the Acceptance Criterion 'A cluster whose archive_status directory is absent now reports 0 B instead of n/a' still sits in a list whose every other item names a mutation and the test it reddens, but nothing this task builds can check it — the fixtures always have the directory and the task is forbidden from touching them. The same fact is already stated in Details -> Edge cases.", + "fix": "Move it to Details -> Edge cases (where it duplicates an existing bullet), or mark it 'documented, not test-gated — user-facing half carried by Task 09'." + }, + { + "severity": "minor", + "category": "hallucination", + "task": "01", + "issue": "Carry-forward, none of these round-1 items were fixed in the rewrite. (a) Edge cases still claims 'every printCmdline call in the tree passes an explicit \"%s\" verb' — false (top/extra.go:44, :64, :71; top/dialog.go:144, :282; top/pglog.go:16, :22; top/ui.go:304 pass bare literals); the conclusion that '%.ready' is safe holds for the other reason already given in the same bullet (query.Format is text/template, only {{ }} is meaningful). (b) 'testing/prepare-test-environment.sh:17-33' is still cited for 'archive_mode=off, no archive_command' — the file mentions neither setting anywhere; the claim is true by absence, not by that block. (c) internal/postgres/postgres.go Exec/Query/QueryRow cited as ':119-133'; the three wrappers span 118-131 (:132-133 is Close). (d) internal/stat/postgres.go calculateDelta short-circuit cited as ':589-597'; the `interval != [2]int{0,0}` guard is at :590.", + "fix": "(a) drop the printCmdline half of the sentence. (b) 'the fixture setup script sets neither archive_mode nor archive_command, so both keep their defaults (archive_mode=off)'. (c)/(d) tighten or drop the ranges — no functional impact." + }, + { + "severity": "minor", + "category": "hallucination", + "task": "06", + "issue": "Two cosmetic line-range drifts among otherwise exact references: 'newConfig() (top/config.go:52-59)' — the function is at :51-59 (:51 is the func line, :50 its doc comment); 'Test_switchViewTo table (:592-624)' — the table literal ends at :611 and the loop/assertions run to :623. Everything else checks out exactly: switchViewTo :232-257, statioNextView :275-287 (:274 is its doc comment), menu.go iota :14-26, selectMenuStyle :36-103 with menuStatIO at :87-95, menuSelect menuStatIO at :194-203, menuOpen :116-122, menuClose :234-243, moveCursor :268-312, keybindings.go keys table :18 with 'w' at :38 and 'J' at :49 and no 'W' anywhere, help.go helpTemplate :10-50 with r,w at :14, j,J at :19, Q caveat at :45, help_test.go helpEntryLine :14-30 / descColumn :35-48 / the <=80 rule at :77-79, Test_selectMenuStyle :8-25 with exactly the six listed values, Test_statioNextView :670-683, rows :604/:605 as quoted, ui_test.go :406-425, pause_test.go :552-577 and the leak note at :588-591, report_test.go — connect at :11, assert.NoError at :12, the nil-conn deref at :14.", + "fix": "Optional: adjust the two ranges." + } + ], + "verified_ok": [ + "Task 01: the helper's specified semantics are internally consistent with how the tests use it — idempotent creation + no revoke is exactly what makes the two role-state cautions (drop the role before any grant-related mutation; do NOT add teardown, because the re-runnability AC depends on idempotency) necessary and sufficient; caller-side `defer` RESET ROLE matches the 'placed immediately after a successful call' rule and the 'leaked SET ROLE poisons later tests' edge case. The name being a caller parameter is consistent within Task 01 (distinct pgcenter_test_archiver_* names, literal constants only, fmt.Sprintf into DDL, never user input) — the inconsistency is on Task 03's side, see the critical finding.", + "Task 01: the round-1 critical is fixed — the helper is now a named deliverable in Description, What-to-do, the first AC, the Files block and Dependencies, with the signature pinned and a no-`testing`-import AC. internal/postgres/testing.go is 47 lines, imports only fmt, has no build tag; DB.Exec/QueryRow/Query exist; internal/postgres imports nothing from internal/query, so no import cycle. The 'do not substitute the pg_ls_dir mutation' caution is correct (pg_ls_dir(text) yields a column named pg_ls_dir, so an unaliased FROM fails 42703 for every role).", + "Task 03: no competing helper is defined anywhere in the file — the round-1 'Coordination risk' paragraph is gone and replaced by three consistent prohibitions (What-to-do/TDD Anchor, AC, Details 'Test helper — reuse, do not define'). Wave 2 / depends_on [\"01\"] is now consistent in frontmatter, header note, Details -> Dependencies and the Context Files tech-spec line (Decisions 8, 11, 18, 19). The corrected `AS name` bullet matches the round-1 live measurement exactly, including the explicit ban on pinning anything to a `column \"name\" does not exist` failure. Line refs check out: overview.go doc comment :92-99 and constant :100-102 verbatim as quoted, overviewVersions :13, Test_ArchivingBacklogQuery_Degrades :123-146 with its comment :124-125, stat/postgres.go :288-295 verbatim, postgres_test.go :206-236 with the 'fixtures role has pg_monitor' comment at :224, and ADR [010] at docs/decisions-log.md:654-666 (round 1 read this one off by one — the task's range is right).", + "Task 06: the zero-value &gocui.Gui{} plan is executable as written against jroimartin/gocui v0.5.0. SetView(name,x0,y0,x1,y1) builds a real *View with no terminal and returns ErrUnknownView as its created signal (gui.go:130-152, the same signal menu.go:117-122 keys off); View.Size() = (x1-x0-1, y1-y0-1) and SetCursor rejects out-of-range points (view.go:114-116, :164-172), so production geometry SetView(\"menu\",0,5,72,6+2) gives maxY=2 and cursor 5 is indeed unreachable — the taller 0,5,72,20 (maxY=14) is required and correct; DeleteView/SetCurrentView scan a slice and return ErrUnknownView, so menuClose does need the registered \"sysstat\" view, as the snippet says; printCmdline on a non-nil zero Gui does not panic (Update just sends on the nil userEvents channel from a goroutine). DeleteKeybinding(viewname,key,mod) is exported in v0.5.0, matches on viewName+ch+key+mod and returns errors.New(\"keybinding not found\") — so the uniqueness probe works exactly as described, and SetKeybinding only appends, so keybindings(app) with a zero Gui returns nil (no field of app is dereferenced during table construction). All three help mutations genuinely fire: helpEntryLine fails on an ambiguous marker (help_test.go:21), so restoring `'w' WAL,` to the r line makes the \"'w' \" marker match two lines; the ` r ` prefix breaks on `r,w`; the exact-description equality breaks on a reword; the archiver token is asserted on one line. The three pre-existing help tests survive: the insertion is above the Space entry so all relative offsets shift uniformly, and the new text contains no '%'. descColumn arithmetic lines up at 22 for the j,J, w,W, r and s,t,i lines. menuWAL after menuStatIO shifts moveUp/moveDown 6/7 -> 7/8, and no test in top/ asserts numeric menu or direction values.", + "Task 06: the `go test ./top/...` gate is now stated correctly — Test_getQueryReport connects at report_test.go:11, asserts at :12 and dereferences the nil conn at :14, so the host run does take the binary down; the CI-image run is named as the only whole-package gate and the host loop is the -run filter. The filter is safe: it also matches Test_switchViewToProcPidStatResetsAutoScrollFlag/ResetsScrollOffset (config_view_test.go:256, :294), which run clusterlessly via &postgres.DB{Local:false}. `go build ./cmd` is right (cmd/pgcenter.go is package main; the repo root has no Go files).", + "Task 01: the host/CI split is accurate for internal/query — all 37 live-test connect sites in internal/query/*_test.go go through t.Skipf, so a host run skips rather than panics, unlike internal/stat (Task 03's stronger warning)." + ], + "stats": { + "tasks_checked": 3, + "claims_verified": 71, + "issues_found": 9, + "critical": 1, + "major": 4, + "minor": 4 + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch2.json new file mode 100644 index 00000000..041e6cfa --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch2.json @@ -0,0 +1,65 @@ +{ + "validator": "dev-reality-checker", + "feature_base": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver", + "tasks_checked": ["02", "04", "05", "08"], + "status": "approved", + "round": 2, + "note": "Round-2 re-validation of revised tasks. All round-1 criticals and majors in scope are resolved and re-verified against the tree (see 'resolved' below). Remaining findings are minor: two are round-1 minors that the revision did not pick up, two are new imprecisions introduced or left by the revision.", + "resolved": [ + "task 02 — the Task 05 handoff is now coherent: Task 02 says Task 05 OWNS the TestViews_Configure wal assertion, Task 05 step 4b adds it. Verified `grep -in 'wal|archiver' internal/view/view_test.go` returns nothing, so the gap is real and is closed exactly once, by Task 05.", + "task 04 — the impossible 'grep returns four hits' AC is gone. The containment form is satisfiable: today `grep -rn showWAL --include=*.go .` returns exactly the four named sites (cmd/report/report.go:25, :70, :151; cmd/report/report_test.go:46) and nothing outside cmd/report/; after the change all hits stay inside those two files. All four anchors verified exact, as are :157-163 (the showStatIO shape to copy), :132 (selectReport), :91/:95 (validate and the literal), report_test.go :34-73 / :46 / :65-66 / :10-32.", + "task 04 — `go run ./cmd report -W` is the working form: verified today it reaches report.RunMain (prints 'open pgcenter.stat.tar: no such file or directory' under the current bool flag). The repository root has no Go files, so `go run .` would fail.", + "task 04 — the gofmt-churn hint and the Test_options_validate message-field instruction are both corrected and now match the tree (the options struct already carries string fields; the validate table has only valid/opts/want).", + "task 05 (a) — the NotRecordable mutation claim is mechanically correct. record/record.go filterViews: the NotRecordable branch and the version-gate branch both `delete(views, k)` and `filtered++`, so on the four <=PG13 rows swapping the reason leaves wantN and wantV identical (green), while the three >=PG14 rows move wantV -1 / wantN +1 (red). Current tree values match every before-number the task quotes: wantV 27/18/24 and wantN 8/11/13/13.", + "task 05 (b) — the host commands are correctly scoped. `go test ./internal/view/...` and `go test ./record/... -run Test_filterViews` both run green on this host (executed); the frontmatter verify:, the AC and the Verification Steps all carry the scoped form plus the Test_tarRecorder panic caveat.", + "task 05 (c) — the TestViews_Configure arms accommodate the assertions as described. `case 190000:` is at view_test.go:183 and `case 140000:` at :193, each already a flat list of assert.Equal calls on views[...] after `views.Configure(opts)`; adding wal/archiver asserts needs no restructuring. Configure does not filter by version, so views['wal'] and views['archiver'] exist in both arms. Constants and values are consistent across tasks: PgStatWALPG14/11/{2,9} is what SelectStatWALQuery returns today; PgStatWALPG19/8/{2,6} is exactly what Task 02 adds; PgStatArchiverDefault/9/{0,0} is exactly what Task 01 defines. The `case \"wal\":` delegation is at view.go:389-391 and the map-key switch at :372-373, as cited.", + "task 08 — dropping A5 is correct and the argument checks out line-for-line: processData builds `views := view.Views{config.ReportType: v}` at report/report.go:282-284, Views.Configure switches on the map key at internal/view/view.go:372-373, newApp returns a zero-value view.View without error at report/report.go:83-85, and formatStatSample REPLACES view.ColsWidth/Cols wholesale (report.go:528-538) so the nil maps of a zero-value View never panic. PGresult.sort returns early / is stable on a single row, so OrderKey/OrderDesc/UniqueKey really are inert. Removing the registry entry would indeed leave the output byte-identical.", + "task 08 — B4 can genuinely fire: with source='Archiver', ready/archived/failed='0', four NULL cells and stats_age='02:00:00', strings.Fields of the ANSI-stripped data line yields exactly 5 tokens; replacing the four NULLs with '0' yields 9. printStatSample prints the 'ts, rate: ...' header on its own line (report.go:607-613), so the data row is never mixed into the count.", + "task 08 — B5 can genuinely fire: readTar only emits a sample when both meta and stat entries are present, so a meta+sysinfo-only tar prints nothing; adding the two archiver.* ticks yields one printed row (first sample discarded, Decision 12), making the buffer non-empty.", + "task 08 — the unfalsifiable assertions are gone and correctly justified. doReport really does end in an unconditional `return nil` after wg.Wait() (report/report.go:109-151, return at :151) and prints processData's error with fmt.Println, not to app.writer. naLiteral='n/a' is at top/stat.go:430 in package top and appears nowhere in report/, so the NotContains would have been vacuous.", + "task 08 — the markdown links now resolve from the task file's own directory: ../../decisions-log.md -> docs/decisions-log.md and ../../../ -> repo root all exist (checked on disk). The bgwriter anchors (:141-151 meta, :155-161 mkRow, :188-202 tar, :204-216 drive, :218-228 invariants, :230-237 -update) and report_test.go:24 / :73-77 are all exact.", + "task 08 — the A1-A4 mutations were re-checked against the diff/render code and each does redden its named subcase: A1 turns archived into the delta 3, A2/A3 push buffers_full out of the diffed range on pg19, A4 pulls the pg18 stats_age string into DiffIntvl{2,9} so diffPair -> ParseInt fails and the buffer stays empty (assert.NotEmpty fails). Ncols on the view is not consulted by diff(), which loops on curr.Ncols, so none of these panic instead of failing cleanly." + ], + "findings": [ + { + "severity": "minor", + "category": "hints", + "task": "02", + "issue": "Round-1 minor not picked up by the revision: Details still says '`internal/query/wal.go` (33 lines today)'. The file is 32 lines (`wc -l internal/query/wal.go`). Every other anchor in Task 02 re-verified exact this round: PgStatWALPG14 :5-11, PgStatWALDefault :15-21, SelectStatWALQuery :25-32, wal_test.go 56 lines with Test_SelectStatWALQuery :10-31 and the 190000 row at :21 reading `{version: 190000, wantNcols: 7, wantDiffIntvl: [2]int{2, 5}}`, Test_StatWALQueries :34-56 with the version list at :35, SelectStatBgwriterQuery bgwriter.go:41-52, PostgresV19 query.go:22, view.go `case \"wal\":` :389-391.", + "fix": "Change '33 lines today' to '32 lines today', or drop the count." + }, + { + "severity": "minor", + "category": "missing_file", + "task": "04", + "issue": "Task 04's markdown links are repo-root-relative but the file lives at docs/features/017-feat-wal-archiver/, so none of them resolve. Verified from the task's own directory: docs/features/017-feat-wal-archiver/017-feat-wal-archiver.md, cmd/report/report.go, .claude/skills/project-knowledge/overview.md and docs/decisions-log.md are all MISSING as written; the same targets resolve fine as `017-feat-wal-archiver.md`, `../../../cmd/report/report.go`, `../../../.claude/...`, `../../decisions-log.md` — which is precisely the form Tasks 02, 05 and 08 use, and the form Task 08 was fixed to this round. This affects the whole Context Files section plus the Post-completion decisions-log link.", + "fix": "Rewrite Task 04's links in the same relative form the other three tasks use: bare filenames for feature artifacts, `../../decisions-log.md` for the ADR log, `../../../` for code and project-knowledge files." + }, + { + "severity": "minor", + "category": "tdd", + "task": "05", + "issue": "The new step 4b brings a red-first claim that cannot hold for its wal half. The TDD Anchor says of TestViews_Configure: 'Red today on two counts: no `archiver` key exists ... and nothing pins the wal layout at all.' The second count is only true against a tree without Task 02. Task 05 declares depends_on ['01','02'] and sits in Wave 2, so by the time it runs SelectStatWALQuery already returns PgStatWALPG19/8/{2,6} at 190000 and PgStatWALPG14/11/{2,9} at 140000 — both new wal assertions pass the moment they are written, and the executor following the section's own instruction ('the tests must be written and observed failing against the current tree') will see green and may conclude the assertion is wrong or the wiring is missing. Only the archiver assertions can be observed red. The task does supply the correct compensation (the AC mutation 'reverting task 02's PG 19 branch ... turns TestViews_Configure red'), so the guard is real — the framing is what is off.", + "fix": "Say it explicitly in the TDD Anchor: 'the archiver assertions are red today; the wal assertions are green on first write because Task 02 has already landed — their red-first evidence is the AC mutation that reverts Task 02's PG 19 branch, which must be run and observed.'" + }, + { + "severity": "minor", + "category": "hints", + "task": "08", + "issue": "Round-1 minor (the 'two internal/source inconsistencies' item) was not picked up by the revision, and both halves are still wrong. (a) Verification Step 7 still calls the legacy fixture 'the legacy PG13-era tar' and says 'the PG 19 branch must not reach a ~PG13 recording', contradicting the Description's 'a 2021 PG14beta1 recording'. The Description is right: report/testdata/pgcenter.stat.golden.tar carries meta version_num '140000' / version '14beta1 (Ubuntu 14~beta1-1.pgdg20.04+1)' (read out of the tar this round). (b) The report_record_statio_test.go anchors still drift: statIOAnsiRE/statIOStripANSI are at :42-47 (var at :44, func at :47), not ':43-46'; the version-independent single-golden precedent Test_app_doReport_StatIOTime starts at :185 with its doc comment at :176-184, not ':177-206'. The bgwriter anchors, by contrast, are all exact.", + "fix": "Say 'the legacy 2021 PG14-beta fixture' in Step 7 as well as in the Description, and refresh the two statio anchors (or name the symbols without line numbers — they do not drift)." + }, + { + "severity": "minor", + "category": "hints", + "task": "08", + "issue": "New imprecision in the text that replaced A5: 'the empty-archive subcase never reaches `processData` at all'. processData is always started (report/report.go:130-148 launches it as a goroutine before any tar entry is read); on an archive with no archiver.* entries it simply never enters the data branch and exits through `case <-doneCh`. What the argument needs — that the subcase never reaches the `views.Configure` call inside the first-sample branch — is true, but as written the sentence is a claim about the code that is false, in a paragraph whose whole purpose is to model the report path accurately. The A5 conclusion itself is unaffected and correct.", + "fix": "Reword to 'the empty-archive subcase never reaches the Configure call inside processData's first-sample branch — no sample is ever delivered, so processData exits through its doneCh arm'." + } + ], + "stats": { + "tasks_checked": 4, + "claims_verified": 78, + "issues_found": 5 + } +} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch3.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch3.json similarity index 100% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch3.json rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-reality-batch3.json diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch1.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch1.json new file mode 100644 index 00000000..41adc386 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch1.json @@ -0,0 +1,147 @@ +{ + "validator": "dev-task-validator", + "feature_base": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver", + "iteration": 2, + "tasks_checked": ["01", "02", "03", "04", "05"], + "status": "approved", + "round1_resolved": [ + "01 frontmatter — all five files now parse with yaml.safe_load; `verify` is quoted in 01/03 and comment-shifted in 02. Verified by parsing each block.", + "01 helper ownership — `internal/postgres/testing.go` now has a What-to-do step, a signature (`SetupTestRole(db *DB, name string, pgMonitor bool) error`), three acceptance criteria and a production-build guard (`grep -n '\"testing\"'`). The three 'touches nothing that exists' statements are corrected to name the three files, matching tech-spec Task 1's Files to modify. The no-*testing.T rationale is correct: internal/postgres/testing.go carries no build tag (verified — 47 lines, imports only fmt) and ships in ./cmd.", + "01 pg_ls_dir mutation — replaced by 'delete the SET ROLE' + an explicit note that the old mutation reddens with 42703 under every role. The replacement discriminates correctly.", + "02 — no longer claims Task 05 adds wal assertions on its behalf; the hand-off is now stated as Task 05's ownership and Task 05 accepts it in step 4b. `grep wal internal/view/view_test.go` returns nothing, so Task 02's premise is accurate.", + "03 — Wave 2 / depends_on [\"01\"] is consistent in frontmatter, blockquote, Context Files annotation and Details → Dependencies. The competing-helper paragraph is replaced by 'Test helper — reuse, do not define'. The invented 'AS name is required' rationale is corrected and the task now forbids pinning anything to a 42703 failure. Both role mutations carry the not-revertible caution plus a drop-role/fresh-container instruction.", + "04 — the grep-count criterion is now a containment check (verified: `grep -rn showWAL --include=*.go .` currently returns exactly the four named sites, all under cmd/report/); `go run ./cmd` is used throughout (cmd/pgcenter.go is package main).", + "05 — the NotRecordable mutation is now scoped to the three >=PG14 rows and the arithmetic is correct against the real filterViews (both drop branches increment the same `filtered`, so the four <=PG13 rows keep wantN 8/11/13/13 -> 9/12/14/14 either way). Every count in the task matches the tree: TestNew 27, TestView_VersionOK 27/27/24/19/16/14/14, Test_filterViews wantV 27/18/24 and wantN 8/11/13/13. record-package panic and the `-run Test_filterViews` scoping are stated. Task 05 now owns the TestViews_Configure wal+archiver assertions (arms verified at :183-192 and :193-202)." + ], + "findings": [ + { + "severity": "major", + "category": "consistency", + "task": "03", + "section": "Acceptance Criteria (mutations 2, 3, 4)", + "issue": "New contradiction introduced by the Decision-19 consolidation. Three of Task 03's five mutations are phrased as edits to role-setup code that Task 03 does not own and is forbidden to touch. (a) 'delete the SET ROLE statement from Test_ArchivingBacklogQuery_PgMonitorRole' — there is no SET ROLE statement in the test: Task 01's helper does the SET ROLE internally ('idempotently creates ... and SET ROLEs the given connection into it'), and Task 03's own Details say its job is 'to call it and to RESET ROLE in a defer at the call site'. (b) 'remove the GRANT pg_monitor from the role setup' and (c) 'add GRANT pg_monitor to the deny role' read as edits to the GRANT inside internal/postgres/testing.go — the file the task's own acceptance criterion says 'is not modified here'. An executor following the words literally edits Task 01's shipped helper mid-feature; an executor following the criterion skips the mutations. Task 01 words its equivalent correctly ('pass pgMonitor: false in the ... SetupTestRole call').", + "fix": "Restate all three against the call site, which is the only code Task 03 owns: (a) 'delete the postgres.SetupTestRole(...) call so the session stays the fixture superuser -> the test must go RED on its own current_user/rolsuper guard'; (b) 'pass pgMonitor: false at the call site, having first dropped the role on the target cluster'; (c) 'pass pgMonitor: true for the deny role'. Add one sentence: internal/postgres/testing.go is never edited, not even temporarily — a mutation there would also change what Task 01's privilege tests see in the same `go test ./internal/query/...` run." + }, + { + "severity": "major", + "category": "content", + "task": "03", + "section": "Verification Steps", + "issue": "A verification gate that cannot pass, and contradicts the task's own implementation hint. Verification Steps require `grep -rn \"pg_ls_dir\" internal/` to return no hit ('Grep the tree for the stale claims — no hit may remain'). But Implementation hints instruct the rewritten doc comment to keep a pg_ls_dir clause: 'which superuser and pg_monitor can execute (unlike pg_ls_dir, which is superuser-only — worth one clause, since that is the bug being fixed and Decision 8 supersedes ADR [010])'. The corrected comments in overview_test.go:124-125 and stat/postgres_test.go:224 will also most naturally name the old function. This is the same shape as the round-1 Task 04 grep-count finding: an unsatisfiable check gets quietly dropped, or the executor deletes the clause the task asked for.", + "fix": "Replace the blanket grep with the property actually being checked: `grep -rn \"pg_ls_dir\" internal/` must return no line that *asserts a privilege requirement* — i.e. no remaining hit of `grep -rniE 'pg_ls_dir.*(pg_monitor|superuser)' internal/` other than the one clause in OverviewArchivingBacklog's doc comment that says pg_ls_dir is superuser-only. Keep `grep -rn \"has pg_monitor\" internal/` as-is (it is genuinely expected to be empty), and add `grep -rn \"pg_ls_dir\" internal/query/overview.go internal/stat/postgres.go` returning only the intended clause." + }, + { + "severity": "major", + "category": "consistency", + "task": "03", + "section": "Details → Test helper / Edge cases", + "issue": "Role-name ownership is stated two incompatible ways across the pair. Task 03 says 'The SQL role names come from the helper too; do not invent parallel ones' and 'Read internal/postgres/testing.go for the helper's actual signature and call it'. But the helper Task 01 will write takes the name from the caller — `SetupTestRole(db *DB, name string, pgMonitor bool) error` — and supplies no names at all; Task 01 pins only its own two (`pgcenter_test_archiver_monitor`, `pgcenter_test_archiver_norole`) and states the opposite expectation in its Edge cases: 'Task 03 creates its own roles on the same clusters through the same helper; distinct names keep the two suites from mutating each other's grants.' Task 03's executor will look for role-name constants in the helper, find none, and is explicitly forbidden from choosing any. This is a new form of the round-1 clashing-roles finding: the round-1 fix suggested pinning names in the helper, and the implemented fix pinned them in Task 01's test file instead — but only Task 01 was updated to match.", + "fix": "Pick one and make both tasks say it. Simplest: Task 03 pins its own two role names in its TDD Anchor exactly as Task 01 does (e.g. `pgcenter_test_overview_monitor` / `pgcenter_test_overview_norole`), and the sentence becomes 'the helper takes the role name as an argument; use the two names pinned in this task's TDD Anchor and do not reuse Task 01's, so the two suites cannot mutate each other's grants'. Alternative: export the four names as constants from internal/postgres/testing.go in Task 01 and have Task 03 reference them — but then Task 01 must add that to its deliverable and its acceptance criteria." + }, + { + "severity": "major", + "category": "content", + "task": "05", + "section": "Acceptance Criteria / step 4b", + "issue": "Half of the inherited wiring assertion is provably inert, while the acceptance criterion presents all of it as closing a gap. The `wal` half is real: reverting Task 02's PG 19 branch changes what Configure() delivers, and the task names that mutation. The `archiver` half cannot fail on what it claims to guard. `New()` registers the archiver entry with QueryTmpl PgStatArchiverDefault, Ncols 9, DiffIntvl {0,0}, and `SelectStatArchiverQuery` returns exactly those three values at every version — so `case \"archiver\":` in Configure() re-assigns identical values (the task says so itself: 'this case re-assigns the same three values the static entry already carries'). Deleting the case entirely leaves views[\"archiver\"].QueryTmpl/Ncols/DiffIntvl unchanged, TestViews_Configure green, TestNew_ArchiverView green, and the trailing `assert.NotEqual(t, \"\", v.Query)` loop green (Configure's second loop Formats every view's QueryTmpl regardless). The criterion 'Configure() has a case \"archiver\": that assigns from SelectStatArchiverQuery' therefore has no possible red-proof, and none of the four listed mutations can produce one.", + "fix": "Say it out loud instead of implying coverage. Reword the AC to: 'Configure() has a case \"archiver\": ... — this case is deliberately unguardable by test while the selector is version-independent (it re-assigns identical values); it is kept so a future version branch is a one-line change, and it is verified by review, not by a mutation.' Keep the archiver assertions in TestViews_Configure only as characterization of the New() values (state that), and restrict the 'This closes the gap task 02 pointed here' claim to the wal assertions, which is the gap that actually existed." + }, + { + "severity": "minor", + "category": "content", + "task": "01", + "section": "Acceptance Criteria", + "issue": "Same wording problem as Task 03(a), but harmless-by-reinterpretation. The load-bearing mutation is 'delete the SET ROLE line from Test_StatArchiverQuery_PgMonitorRoleSucceeds' — the test contains no SET ROLE line, because this task's own What-to-do puts SET ROLE inside `SetupTestRole` and leaves only RESET ROLE at the call site. The equivalent that works is deleting the `postgres.SetupTestRole(...)` call, which does leave the session as the fixture superuser and does redden the current_user/rolsuper guard.", + "fix": "Restate as: 'Mutation — delete the postgres.SetupTestRole(...) call from Test_StatArchiverQuery_PgMonitorRoleSucceeds (the helper is what issues SET ROLE), so the test runs on the fixture superuser connection: the test turns red on its own current_user/rolsuper guard.' Apply the same wording to the negative test if it is checked the same way." + }, + { + "severity": "minor", + "category": "content", + "task": "01", + "section": "Acceptance Criteria", + "issue": "Round-1 minor not applied. The mutation 'replace the ready sub-select with the literal 0 (no privileged call at all) -> Test_StatArchiverQuery_WithoutPgMonitorFails turns red' still rests on an unverified premise: that `pg_stat_archiver` itself is selectable by a role holding no grants. It is (pg_catalog views carry SELECT to PUBLIC and pg_stat_archiver has no column-level restriction), so the mutation almost certainly works — but if any fixture version disagreed, the mutated query would still error and the test would stay green, proving nothing.", + "fix": "Add half a sentence to the criterion: '...turns red — i.e. the mutated, unprivileged query succeeds under the deny role, confirming pg_ls_archive_statusdir() is the only thing that makes the real query fail. Note the observation in the decisions-log entry.'" + }, + { + "severity": "minor", + "category": "content", + "task": "01", + "section": "Details → Implementation hints / Acceptance Criteria", + "issue": "New consequence of the Decision-19 consolidation, not covered by any criterion. `SetupTestRole` lands in internal/postgres/testing.go, which has no build tag and ships in the released binary — so the release now exports a function that issues `CREATE ROLE` / `GRANT pg_monitor` DDL built with fmt.Sprintf from a caller-supplied identifier. The task recognises the risk in prose ('keep the helper's callers to literal constants, never user input') but no acceptance criterion pins it, while the criterion right next to it pins the much smaller `testing` import concern. dev-security-auditor is a listed reviewer and will raise it.", + "fix": "Add one criterion: 'the role name reaching the DO block is either a compile-time constant at every call site or sanitised as an identifier (pgx exposes pgx.Identifier{name}.Sanitize()), and the helper's doc comment says the function is test-only despite shipping in the binary.' Cheap, and it turns a review argument into a checked item." + }, + { + "severity": "minor", + "category": "content", + "task": "02", + "section": "Acceptance Criteria (M1)", + "issue": "Round-1 minor not applied. M1 returns DiffIntvl {2,7} on a PG 19 result with 8 live columns (indices 0-7), so the TDD Anchor's assertion 'the header at DiffIntvl[1]+1 is stats_age' indexes element 8 of an 8-element slice: Test_StatWALQueries fails by index-out-of-range panic, not by assertion. It is red either way, but a panic inside a t.Run table also kills the surrounding subtests and hides which version failed.", + "fix": "Bound the boundary assertion in the TDD Anchor — assert `DiffIntvl[1]+1 < len(fields)` first (or look the header up by name) — so M1 produces a readable failure. Keep M1 itself; its Test_SelectStatWALQuery half is clean." + }, + { + "severity": "minor", + "category": "content", + "task": "04", + "section": "Acceptance Criteria", + "issue": "Round-1 minor not applied. The criterion 'Flag precedence is unchanged: the showWAL arm sits between showFunctions and showBgwriter, and -A still beats -W' states an invariant with no named mutation, while every other criterion in this task carries one. Test_selectReport_WALPrecedence can fail (showActivity is arm 1 at report.go:134, showWAL arm 5 at :151 — moving the new arm above showActivity makes options{showActivity: true, showWAL: \"a\"} yield \"archiver\"), so the guard is sound; only the red-proof is missing, which is the shape patterns.md rules out.", + "fix": "Add 'Mutation 4 — move the `case opts.showWAL != \"\":` arm above `case opts.showActivity:` -> Test_selectReport_WALPrecedence turns red. Run it, observe, revert.'" + }, + { + "severity": "minor", + "category": "content", + "task": "05", + "section": "TDD Anchor", + "issue": "Two inaccuracies in the newly added step-4b block. (a) 'Red today on two counts: no archiver key exists ... and nothing pins the wal layout at all' — the wal half is not red at any point. Task 05 is Wave 2 and depends on Task 02, so by the time the assertion is written SelectStatWALQuery already returns PgStatWALPG19/8/{2,6} and Configure already delivers it: the wal assertions are green on arrival. Only the archiver assertions are red-first. (b) The compensating mutation 'reverting task 02's PG 19 branch in SelectStatWALQuery' must remove only the branch and keep the PgStatWALPG19 constant — a literal revert of Task 02 deletes the constant too, and the package then fails to compile instead of failing the test, which is not a red the criterion can read.", + "fix": "Rewrite as: 'the archiver assertions are red today (no archiver key -> zero-value view, Ncols 0). The wal assertions are green from the moment they are written, because Task 02 has already landed — they are characterization, and their red-proof is the mutation below: delete only the `if version >= PostgresV19` branch from SelectStatWALQuery, keeping the PgStatWALPG19 constant so the package still compiles, and observe TestViews_Configure red at 190000. Revert.'" + }, + { + "severity": "minor", + "category": "carry-forward", + "task": "05", + "section": "What to do / Post-completion", + "issue": "Round-1 minor not applied. `OrderDesc: true` is pinned in step 1 and asserted by TestNew_ArchiverView, but the tech-spec's Data Models block specifies only `Ncols: 9`, `DiffIntvl: [2]int{0,0}`, `OrderKey: 0`, `UniqueKey: 0`, `NotRecordable: false` (re-checked — the line is verbatim and still has no OrderDesc). It matches the wal/bgwriter precedent, so it is a reasonable addition, but it is an addition to the spec and nothing records it.", + "fix": "Add a Post-completion item: 'record in the decisions log that OrderDesc: true was added beyond the tech-spec Data Models block (following the wal/bgwriter precedent), and add the field to that block if the tech-spec is touched anyway.'" + }, + { + "severity": "minor", + "category": "structure", + "task": "01", + "section": "TDD Anchor / Verification Steps / Details", + "issue": "Round-1 minor not applied. Template scaffolding survives: task 01 keeps `` (:111), the Russian boilerplate 'Тесты, которые нужно написать ДО реализации ...' (:113), `` (:273) and `` (:312); task 03 keeps the boilerplate line (:85) and the Details comment (:222); task 02 keeps the Details comment (:189). Tasks 04 and 05 are clean.", + "fix": "Delete the four HTML comments and the two boilerplate sentences — every one of those sections carries real content and the instruction is already restated in the acceptance criteria." + }, + { + "severity": "minor", + "category": "structure", + "task": "01", + "section": "Context Files", + "issue": "Round-1 minor not applied. The template requires `project.md` under Project knowledge; this repo's PK has overview.md / architecture.md / patterns.md / deployment.md and no project.md. Tasks 02 and 05 document the substitution inline ('there is no project.md in this repo's PK; overview.md plays that role'); tasks 01, 03 and 04 list overview.md with no explanation, so a reader checking template compliance sees a missing mandatory file.", + "fix": "Copy the one-line note from tasks 02/05 into tasks 01, 03 and 04." + }, + { + "severity": "minor", + "category": "structure", + "task": "05", + "section": "Context Files", + "issue": "Round-1 minor not applied, and it is broader than round 1 recorded: tasks 01, 03 and 04 use repo-root-relative link paths (`docs/features/...`, `internal/query/...`, `.claude/skills/...`), while tasks 02 and 05 use file-relative paths (`../../../internal/view/view.go`, bare `017-feat-wal-archiver.md`). Executors or tools resolving links one way hit dead paths in the other half of the batch.", + "fix": "Normalise tasks 02 and 05 to the repo-root-relative style used by 01/03/04 (or the reverse — but pick one for the batch)." + }, + { + "severity": "minor", + "category": "content", + "task": "02", + "section": "Details → Files", + "issue": "Round-1 minor not applied. `internal/query/wal.go` is still described as '33 lines today'; the file is 32 lines. Everything else in that block re-checks clean: PgStatWALDefault at 15-21, SelectStatWALQuery at 25-32, wal_test.go 56 lines, Test_SelectStatWALQuery at 10-31 with the 190000 row at :21 reading `{version: 190000, wantNcols: 7, wantDiffIntvl: [2]int{2, 5}}`, version list at :35 including 190000 — and, worth noting for step 3, the table has no 160000 row, which matches the task's 'rows 140000/150000/170000/180000 stay untouched'.", + "fix": "Correct to 32, or drop the line count — the structural anchors are what matter and they are right." + } + ], + "stats": { + "tasks_checked": 5, + "issues_found": 15, + "critical": 0, + "major": 4, + "minor": 11, + "round1_findings_resolved": 12, + "round1_findings_still_open": 9 + } +} diff --git a/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch2.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch2.json new file mode 100644 index 00000000..94e28889 --- /dev/null +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-task-validation-batch2.json @@ -0,0 +1,116 @@ +{ + "validator": "dev-task-validator", + "feature_base": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver", + "tasks_checked": ["06", "07", "08", "09", "10"], + "iteration": 2, + "status": "approved", + "findings": [ + { + "severity": "minor", + "category": "structure", + "task": "10", + "section": "Correction applied during task validation (2026-08-06)", + "issue": "The round-1 fix was appended as a twelfth section AFTER Post-completion, which the template does not have — the file must end at Post-completion. Both of its paragraphs also duplicate content that the same fix already put into the body: the `0` (not `0 B`) correction is Edge case «Three accepted behaviours must not be filed as defects» (`:362-363`), and the stand-TTL instruction is What-to-do bullet `:70-73` plus Edge case `:332-335`. A QA executor now reads the same two rules twice, in two places, with no statement of which is authoritative.", + "fix": "Delete the appended section. Both facts are already stated in the body; if the provenance matters, add one clause to the existing Edge case («corrected during task validation — the specs' `0 B` is a prose artefact») rather than a new section." + }, + { + "severity": "minor", + "category": "structure", + "task": "08", + "section": "TDD Anchor", + "issue": "Round-1 finding not applied: template residue survives in filled sections. Task 08 still carries `` (`:111`) inside a fully written TDD Anchor, the generic Russian preamble «Тесты, которые нужно написать ДО реализации…» (`:113`) immediately above the task-specific one, and the `` / `` comments (`:307`, `:341`). Tasks 09 (`:171`) and 10 keep the Details comment as well. Task 06 shows the intended end state — every instructional comment replaced by real prose. Not blocking (no `[Task Name]`, `{PK path}`, TODO/TBD anywhere in the batch).", + "fix": "Strip the instructional HTML comments from the filled sections of tasks 08, 09 and 10, and delete the generic Russian TDD preamble in task 08 — the task-specific paragraph two lines below already says it better." + }, + { + "severity": "minor", + "category": "consistency", + "task": "08", + "section": "Verification Steps", + "issue": "Round-1 finding not applied, and it is now settled: the Description calls the shared legacy fixture «a 2021 PG14beta1 recording» (`:24`) while Verification Step 7 calls the same file «the legacy PG13-era tar» (`:336`). Read directly from `report/testdata/pgcenter.stat.golden.tar`, its meta entry carries `version: 14beta1 (Ubuntu 14~beta1-1.pgdg20.04+1)` and `version_num: 140000`, with samples dated 2021-06-14. The Description is right; Step 7 is wrong.", + "fix": "In Verification Step 7 replace «the legacy PG13-era tar» with «the legacy PG 14beta1 tar (version_num 140000)», matching the Description." + }, + { + "severity": "minor", + "category": "content", + "task": "07", + "section": "Details", + "issue": "Round-1 finding not applied: the trailing-whitespace note still cites `:146`, `:151`. Verified byte-for-byte with `cat -A`: the rows that actually end with a trailing space are `- waldir_size` (`:145`) and `- write,ms` (`:151`); `:146` is `- wal,KiB`, which has none. Separately, the constant's own extent is now cited two ways in the same file — `:141-157` in the Description (`:29`) versus `:140-157` in What-to-do (`:54`) and Context Files (`:177`). The true span is 141-157, with the doc comment on 140.", + "fix": "Change the trailing-whitespace reference to `:145`, `:151` (or drop the numbers and keep the instruction «two existing wal rows end with a trailing space — do not clean them up»), and settle on one form for the constant's extent." + }, + { + "severity": "minor", + "category": "carry-forward", + "task": "06", + "section": "Context Files", + "issue": "The round-1 fix added two files the tech-spec's Task 6 entry does not list under «Files to modify»: `top/keybindings_test.go` (a brand-new file, holding `Test_keybindingsWAL`) and `top/pause_test.go` (step 8 rewrites two sentences of its comment block at `:561-568`). The tech-spec lists only `top/config_view.go`, `menu.go`, `keybindings.go`, `help.go`, `config_view_test.go`, `menu_test.go`, `help_test.go`. Both additions are well-argued and correct — this is a spec-drift bookkeeping gap, not a scope problem.", + "fix": "Add `top/keybindings_test.go` and `top/pause_test.go` to the tech-spec's Task 6 «Files to modify», or record the addition in the decisions log when the task completes." + }, + { + "severity": "minor", + "category": "carry-forward", + "task": "06", + "section": "Verification Steps", + "issue": "The round-1 fix made task 06 state — correctly — that `go test ./top/...` cannot be run whole on the host, because `Test_getQueryReport` (`top/report_test.go:9-17`) calls `postgres.NewTestConnect()`, asserts (does not require) NoError and then dereferences the nil connection, taking the binary down. That directly contradicts the tech-spec, which is not updated by any task in this batch: Task 6's «Verify:» line reads `bash — go test ./top/... (runs without PostgreSQL)` and the Agent Verification Plan table repeats it for row 6. Confirmed empirically: the filtered host run (`go test ./top/ -run 'Test_helpTemplate|Test_statioNextView|Test_selectMenuStyle|Test_switchViewTo'`) is green, so the task's replacement gate works and the spec's claim is the wrong one.", + "fix": "Correct the tech-spec's Task 6 «Verify:» line and the row-6 cell of the Agent Verification Plan to the two gates task 06 actually uses (host `-run` filter + full package inside the CI image), so Task 10 does not walk a verification plan that cannot be executed as written." + }, + { + "severity": "minor", + "category": "carry-forward", + "task": "09", + "section": "Description", + "issue": "The `0 B` correction is now applied downstream in three places (task 09 Description `:47-48`, Implementation hints `:207-214`, Post-completion `:232-234`; task 10 Edge cases `:362-363` and its appended section), but the source documents still carry the wrong literal: tech-spec `:210` and its Task 9 entry `:636` («note that … the verbose panel now reports `0 B` instead of `n/a`»), user-spec `:148` and `:305`. User-spec `:305` is one of the 23 criteria Task 10 must walk by name, so the QA gate reads a criterion whose literal the program never prints — task 10 covers it in Edge cases, but only because someone remembered to.", + "fix": "Fix the wording once at the source — tech-spec `:210`/`:636` and user-spec `:148`/`:305` — to «a backlog of zero (rendered `0`)», then let the task files simply follow it instead of each carrying its own correction." + }, + { + "severity": "minor", + "category": "content", + "task": "08", + "section": "Acceptance Criteria", + "issue": "The A5 removal is correct and its argument is verified in full (`report/report.go:282-284` builds `views := view.Views{config.ReportType: v}`; `internal/view/view.go:372-373` switches on the map key; `newApp` at `:83-85` returns a zero-value `view.View` with no error) — but it is filed as a checkbox in a list whose stated rule is «apply each one, see red, revert». «There is deliberately no A5» is the one item in that list that cannot be run, cannot fail and cannot be checked off by an observation; it is rationale, and the same argument already appears verbatim in the Description (`:71-74`).", + "fix": "Move the A5 paragraph out of Acceptance Criteria into Details → Dependencies (next to the existing «which half of Task 5» note, where it belongs) or leave it in the Description, and keep the criteria list to items that can be run and observed." + }, + { + "severity": "minor", + "category": "content", + "task": "07", + "section": "Acceptance Criteria", + "issue": "The three mutations gate exactly one dimension of the new constant — the presence and order of the nine column NAMES (the `\"\\n- \" + name + \"\\t\"` markers, which incidentally also gate the literal tabs, since a tab-to-space expansion breaks every marker). Nothing gates the other things the first criterion demands: the origin column (`pg_ls_archive_statusdir` for `ready`, the seven `pg_stat_archiver` field names), the «empty when never archived» wording on the four NULL-able rows, the `PG-STAT-ARCHIVER-VIEW` docs URL, and the `(PG 19+)` annotation on `fpi,KiB`. `Test_describeReport`'s row is `{report: \"archiver\", want: pgStatArchiverDescription}` — it compares the output to the same constant, so it pins the map entry and nothing about the text. Those criteria are review-only, which is defensible, but the task does not say so, unlike task 06 which explicitly labels the `printCmdline` write count a diff-review item.", + "fix": "Either add one assertion to `Test_describeArchiverColumnOrder` on the origin column of a couple of rows (e.g. that the `ready` row contains `pg_ls_archive_statusdir` and the `last_archived` row contains `last_archived_wal`) and one on the docs-URL suffix, or state plainly next to those criteria that origin text, row wording, the URL and the `(PG 19+)` annotation are diff-review items with no test behind them." + }, + { + "severity": "minor", + "category": "consistency", + "task": "07", + "section": "Reviewers", + "issue": "Report paths are written as bare filenames (`017-feat-wal-archiver-task-07-dev-code-reviewer-review.json`), while tasks 06, 08 and 09 use the template's feature_base-prefixed form (`docs/features/017-feat-wal-archiver/017-feat-wal-archiver-task-NN-…-review.json`). These are output paths a reviewer writes to, resolved from the repo root, not markdown links resolved from the task file — so the bare form is the outlier and risks a JSON landing in the working directory. (Round 1 asked task 08 to match the sibling style for its Context Files links, which it correctly did; its Reviewers paths were right to stay full.)", + "fix": "Prefix task 07's three Reviewers paths with `docs/features/017-feat-wal-archiver/`, matching tasks 06, 08 and 09 and the template's `{feature_base}-task-{ID}-{name}-review.json`." + } + ], + "verified_ok": [ + "ROUND-1 CRITICAL (task 08, A5) — RESOLVED. A5 is gone, replaced by an explicit no-A5 rationale whose every claim was re-verified against the tree: report/report.go:282-284 (`views := view.Views{config.ReportType: v}`), internal/view/view.go:372-373 (`for k, view := range v { switch k {`), report/report.go:83-85 (newApp returns views[config.ReportType], no error). The registry genuinely is unobservable in this replay, and the task now says so and points at Task 5's unit tests as the right layer.", + "ROUND-1 MAJOR (task 06, help mutation) — RESOLVED, and the replacement holds. helpEntryLine (top/help_test.go:14-30) asserts `assert.Equal(t, -1, idx)` on a second match, so an ambiguous marker fails the test. `'w' ` (trailing space) occurs exactly once in the post-edit helpTemplate, so restoring `'w' WAL,` to the r line makes it ambiguous and reddens Test_helpTemplate_walEntry. The new Test_helpTemplate_replicationEntry closes the other half: `'r' replication,` is unique, the ` r ` prefix fails under a restored `r,w` token, and the no-`'w'` assertion fires under the restored clause. descColumn arithmetic checks out — the r, s,t,i, j,J and new w,W lines all land on column 22.", + "ROUND-1 MAJOR (task 08, ungated subcases) — RESOLVED. B4 and B5 are real gates. B4: the never-archived row is source=Archiver, ready=0, archived=0, failed=0, stats_age=02:00:00 with four NULLs → exactly 5 strings.Fields; replacing the NULLs with sql.NullString{String:\"0\",Valid:true} yields 9. B5: two archiver.* entries in the previously empty tar produce one printed row (two ticks minus the discarded first), so the empty-buffer assertion reddens. The coverage rule now names every subcase: populated A1/B1/B2, pg18 A4, pg19 A2/A3/B3, never_archived B4, no_archiver_entries B5.", + "ROUND-1's FALSE PREMISE ABOUT top/ TESTABILITY — CORRECTED, AND I RE-PROVED IT MYSELF. A throwaway probe in package top (written, run, deleted) confirms every claim task 06 now makes: menuSelect(app)(&gocui.Gui{}, mv) over the menuStatIO branch returned stat_io / stat_io_time / stat_io for cursor 0 / 1 / 5, each with a nil error and app.config.menu.menuType reset to menuNone; keybindings(app) with ui=&gocui.Gui{} returned nil; DeleteKeybinding(\"sysstat\",'J',ModNone) succeeded once and reported `keybinding not found` on the second call, while 'W' reported not-found on \"sysstat\", \"\", \"menu\", \"dialog\" and \"help\". gocui v0.5.0 exports DeleteKeybinding (gui.go:262-276); SetView (gui.go:130-152) builds a *View from coordinates and returns ErrUnknownView as its created signal; DeleteView/SetCurrentView scan a slice and return ErrUnknownView rather than panicking. The pause_test.go sentences task 06 amends are at :561-566, exactly where step 8 says.", + "ROUND-1 (task 06, build command) — RESOLVED. `go build -o /dev/null ./cmd` exits 0; cmd/pgcenter.go is the main package and there is no ./cmd/pgcenter package.", + "ROUND-1 (task 07, depends_on) — RESOLVED. depends_on is now [\"04\", \"05\"], covering the `-W a` smoke check; Task 4 is Wave 1 and Task 5 Wave 2, so the wave-3 assignment is unchanged and Details → Dependencies now matches the declaration (Tasks 1 and 2 reachable transitively through 05).", + "ROUND-1 (tasks 09/10, `0 B`) — RESOLVED in the task files. Task 09 carries the correction in three places with the right code anchors (internal/pretty/pretty.go:11-12 returns the bare \"0\"), task 10 restates it as accepted behaviour, and task 10 additionally records the lapsed stand TTL with an explicit instruction to request a fresh stand rather than skip the manual gate. Only the upstream specs still say `0 B` — see the carry-forward finding.", + "ROUND-1 (task 08, markdown links) — RESOLVED for Context Files and Post-completion: feature artifacts are bare filenames, repo files carry `../../../`, and `[docs/decisions-log.md](../../decisions-log.md)` resolves correctly from docs/features/017-feat-wal-archiver/.", + "ALL MUTATION CRITERIA IN THE CODE TASKS CAN FAIL, and each names the test it reddens. Task 06: all ten mutations re-checked against top/config_view.go:232-257 (switchViewTo's default arm is what makes the deleted-case mutation red), top/menu.go:125 (the menuNone reset), gocui's first-match DeleteKeybinding, and helpEntryLine's ambiguity assertion. Task 07: mutation 1 works because describeReport (report/report.go:665-698) prints `unknown description requested` and returns nil for an unknown type, so only the named test catches a missing map entry; mutations 2-3 fire on the require.NotEqual(-1, pos) presence check, and the trailing-tab anchors keep `fpi` off `fpi,KiB` and `write` off `write,ms`. Task 08: A1 verified against calculateDelta's `interval != [2]int{0,0}` short-circuit (internal/stat/postgres.go:589-597) — {2,2} puts archived at index 2 inside the diffed range, turning 100003 into 3; A2/A3 both land on DiffIntvl {2,5} for a pg19 fixture and move buffers_full out of the diff; A4 drives stats_age '02:00:00' into diffPair and produces the wrapped `diff failed`; B3 is sound because itv is computed as `int(d.ts.Sub(prevTs) / time.Second)` (report/report.go:334-343), so two-second spacing halves every integer delta.", + "Frontmatter: all five tasks carry status/depends_on/wave/skills/verify/reviewers/teammate_name and nothing beyond the template; status is `planned` everywhere; depends_on is an array of ID strings; skills and reviewers are arrays. Waves and dependencies: 06→05, 07→[04,05], 08→05 (wave 3 over waves 1-2), 09→04 (wave 3 over wave 1), 10→01..09 (wave 4). No cycles, every referenced ID exists, every dependency sits in an earlier wave.", + "Skill↔reviewer mapping is unchanged and correct: code-writing → three reviewers (06, 07, 08), documentation-writing → dev-code-reviewer (09), pre-deploy-qa → [] (10). Frontmatter skills/reviewers match the Required Skills and Reviewers sections in all five files.", + "TDD Anchor: present and substantive for 06, 07, 08; correctly ABSENT from 09 and 10, which are non-code work and carry Verification Steps instead — as prescribed, not an omission.", + "Section presence and order match the template in tasks 06-09; Details carries Files / Dependencies / Edge cases / Implementation hints in all five. Task 10's ordering is correct up to Post-completion, then the appended twelfth section — see the structure finding.", + "Context Files: every mandatory artifact is present in all five tasks (user-spec, tech-spec, decisions log, architecture.md, patterns.md where relevant). `project.md` does not exist in this repo — .claude/skills/project-knowledge/ holds architecture.md, deployment.md, overview.md, patterns.md — and every task states that overview.md is its equivalent rather than linking a phantom file.", + "Carry-forward from the tech-spec: Tasks 7, 8 and 9 modify exactly the files their Implementation Tasks entries list, with none dropped. The Testing Strategy items land where expected (walNextView cycle + count-based test updates → 06; the describe map entry → 07; archiver + wal PG 18/PG 19 goldens → 08). Task 06 extends the switchViewTo table with three rows where the spec asks for two — a superset, which is allowed. Tech-spec Acceptance Criteria hold exactly 11 checkboxes and the user-spec «Критерии приёмки» exactly 23, the counts Task 10 walks by name.", + "Code anchors re-checked in this round and correct: pgStatWALDescription spans report/describe.go:141-157 with the fpi row at :148 and the wal,KiB row at :146; describeReport's map at report/report.go:666-694 with `\"wal\"` at :674; Test_describeReport's table at :1172+ with the `\"wal\"` row at :1184 and the shared `update` flag at report/report_test.go:24; statIOStripANSI/statIOAnsiRE at report/report_record_statio_test.go:43-47; top/keybindings.go 'w' at :38, 'J' at :49, no existing 'W' anywhere in :18-85; top/menu.go iota block :14-26, menuStatIO style :87-95, menuSelect branch :194-203, menuClose focusing sysstat :234-243; top/help.go helpTemplate :10-50 with the r,w line at :14, j,J at :19 and the Q caveat at :45, exactly one `%` verb; doc/release-notes/ holds v0.8.0.md and v0.9.0.md, both opening with a `Release date:` line — the format task 09 follows and the reason it forbids inventing one." + ], + "stats": { + "tasks_checked": 5, + "issues_found": 10, + "critical": 0, + "major": 0, + "minor": 10, + "round_1_findings_resolved": 7, + "round_1_findings_still_open": 3 + } +} diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-tech-spec.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-tech-spec.md similarity index 92% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-tech-spec.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-tech-spec.md index 556ba502..0ded6c14 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-tech-spec.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-tech-spec.md @@ -207,7 +207,10 @@ test below PG 14, so it would promise a version nobody verifies. **What changes:** `pg_ls_dir('pg_wal/archive_status')` is `missing_ok=false`, while `pg_ls_archive_statusdir()` is `missing_ok=true`. Proven by moving `$PGDATA/pg_wal/archive_status` aside on a live PG 18.4: the old query errors, the new one returns `0`. So on a cluster whose -`archive_status` directory is gone, the verbose panel flips from `n/a` to a confident `0 B`. +`archive_status` directory is gone, the verbose panel flips from `n/a` to a confident zero — and +the rendering is a bare `0`, not `0 B`: the field goes through the project's size formatter, whose +zero case returns the digit alone. A check written against the string `0 B` would fail on correct +behaviour. **Rationale:** a missing `archive_status` means a damaged or hand-edited data directory — a state in which the backlog number is the least of the operator's problems, and one that no supported PostgreSQL configuration produces on its own. Weighed against the gain (the most common monitoring @@ -330,6 +333,38 @@ creation must be idempotent and the session must not leak an assumed role into l bump and change what every existing test sees; skipping the privilege tests and relying on the stand — rejected, that is exactly the gap that let the wrong `pg_ls_dir` privilege assumption survive. +### Decision 20: the backlog walk was measured, and accepted without throttling + +**Decision:** no throttling. The verbose panel keeps walking `archive_status` every tick. +**The measurement** (stand run 2026-08-06, under Decision 9's exact conditions — a `pg_monitor`-only +role, verbose on, concurrent `pgcenter record`, 200 005 `.ready` files): + +| | | +|---|---| +| Backlog query wall time | ~1108 ms mean (0.9 ms on an empty directory) | +| View-switch latency | 70–240 ms | +| Effective refresh, feature, verbose on | 1.9 s/tick | +| Effective refresh, master, verbose on | 1.0 s/tick | + +So at 200 005 files the feature halves the refresh rate on every screen, and the panel rides every +screen, so the cost is not confined to the archiver view. Superusers are affected too, not only +`pg_monitor` roles: the old `pg_ls_dir` returned names, the new call stats every file. + +**Why that is accepted anyway.** The cost is linear — ~5.5 µs per file. A five-thousand-segment +backlog costs ~28 ms and a twenty-thousand one ~110 ms, both inside the noise. The doubling needs +200 000 segments, which is **3.1 TB of unarchived WAL** — a state that is barely reachable in +practice, would be noticed long before, and in which a screen refreshing every two seconds instead of +every second is nowhere near the operator's biggest problem. + +**Not recorded as tech debt, deliberately.** The roadmap owner declined: a debt entry is a commitment +to fix, and there is no intention to fix this. The numbers live here instead, so the next person +asking "how expensive is that directory walk" has an answer without re-measuring. + +**Alternatives considered:** applying [010]'s existing `latencyGuardThreshold` + +`dbSizeThrottled` machinery to the aggregate — implementable and briefly started, then stopped once +the per-file cost was put against realistic backlog sizes; reverting the panel to `pg_ls_dir` — +rejected, it would restore the `n/a` that this piece of the feature exists to remove. + ## Data Models No database schema, no Go types added. Two SQL shapes: @@ -432,7 +467,7 @@ verified by driving the TUI over ssh/tmux on the stand. | 3 | bash | `go test ./internal/query/... ./internal/stat/...` — verbose backlog aggregate runs under a `pg_monitor` role | | 4 | bash | `go test ./cmd/report/...` — `-W w`, `-W a`, `-W x` map as specified | | 5 | bash | `go test ./internal/view/... ./record/...` — registration, availability and filterViews counts | -| 6 | bash | `go test ./top/...` — cycle, menu, help; runs without PostgreSQL | +| 6 | bash | `go test ./top/...` in the CI image — cycle, menu, help, keybinding registration (the package panics on a bare host) | | 7 | bash | `go test ./report/...` — describe text for archiver and the wal FPI row | | 8 | bash | `go test ./report/...` — golden replay for archiver and for wal at PG 18 and PG 19 | | 9 | bash | grep the release notes for both literal messages (`report type is not specified, quit`, `diff failed`) | @@ -455,8 +490,14 @@ existing invocations: - `pgcenter report -W -f dump.tar` (the common legacy shape) → pflag consumes `-f` as the flag's value, `selectReport` returns `""`, and the command exits with `report type is not specified, quit`. -Both exit non-zero; neither silently changes meaning. The release notes must describe the second -shape, because that is what users will actually hit. +**Neither exits non-zero** — measured, not assumed: `main()` prints the message and returns without +`os.Exit(1)`, so both shapes exit 0. This is pre-existing and repo-wide (`-J q` behaves identically), +but this feature is what makes it bite: a legacy wrapper like +`pgcenter report -W -f dump.tar > out.txt || alert` now writes an empty file and reports success. So +the failure is loud on the terminal and SILENT to a script. The release notes must say exactly that, +and must describe the second shape, because that is the one real invocations take. Fixing the exit +code is out of scope here — it would change behaviour for every report type — and goes to the +tech-debt register at finalization. **Migration strategy:** none beyond documentation — the roadmap owner rejected both a deprecation period and a `NoOptDefVal` compatibility shim. The flag's help string and a new @@ -600,9 +641,11 @@ Technical criteria, complementing the user-facing ones in the user-spec: same way, following the existing entry tests. - **Skill:** code-writing - **Reviewers:** dev-code-reviewer, dev-security-auditor, dev-test-reviewer -- **Verify:** bash — `go test ./top/...` (runs without PostgreSQL) +- **Verify:** bash — `go test ./top/...` in the CI image; on a bare host the package panics + (`Test_getQueryReport` dereferences a nil connection), so a host-side run must be `-run` scoped - **Files to modify:** `top/config_view.go`, `top/menu.go`, `top/keybindings.go`, `top/help.go`, - `top/config_view_test.go`, `top/menu_test.go`, `top/help_test.go` + `top/config_view_test.go`, `top/menu_test.go`, `top/help_test.go`, `top/keybindings_test.go`, + `top/pause_test.go` - **Files to read:** `internal/view/view.go` #### Task 7: report describe text for archiver and the wal FPI row @@ -633,7 +676,7 @@ Technical criteria, complementing the user-facing ones in the user-spec: - **Description:** Add the 0.12.0 release-notes entries for the breaking `-W` change and the PG 19 legacy-archive limitation, quoting the literal messages users will see (`report type is not specified, quit` and `diff failed`), and note that on a cluster whose `archive_status` directory is - missing the verbose panel now reports `0 B` instead of `n/a`. Target, scope and what is deliberately + missing the verbose panel now reports a bare `0` instead of `n/a`. Target, scope and what is deliberately left out are fixed by Decision 13. - **Skill:** documentation-writing - **Reviewers:** dev-code-reviewer diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver-techspec-validation.json b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-techspec-validation.json similarity index 100% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver-techspec-validation.json rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver-techspec-validation.json diff --git a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver.md b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver.md similarity index 90% rename from docs/features/017-feat-wal-archiver/017-feat-wal-archiver.md rename to docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver.md index bed860e0..ea581919 100644 --- a/docs/features/017-feat-wal-archiver/017-feat-wal-archiver.md +++ b/docs/features/archive/017-feat-wal-archiver/017-feat-wal-archiver.md @@ -145,7 +145,7 @@ Archiver 1423 84512 00000001000000A300000012 00:47:12 3781 0000 **Одно поведение всё же меняется.** `pg_ls_dir` падает, если каталога нет, а `pg_ls_archive_statusdir()` в этом случае возвращает пустой результат (проверено переносом `$PGDATA/pg_wal/archive_status` на живом PG 18.4). То есть на кластере с отсутствующим каталогом - `archive_status` панель покажет уверенный `0 B` вместо `n/a`. Принимается: отсутствие этого + `archive_status` панель покажет уверенный ноль вместо `n/a`. Принимается: отсутствие этого каталога означает повреждённый или руками поправленный data directory — состояние, в котором размер backlog далеко не главная проблема. @@ -277,7 +277,11 @@ Archiver 1423 84512 00000001000000A300000012 00:47:12 3781 0000 отчёт по `pg_stat_wal`; `pgcenter report -d -W a` печатает описание колонок. - [ ] `pgcenter report -W` последним токеном падает с ошибкой `flag needs an argument: 'W' in -W`; `pgcenter report -W -f dump.tar` и `pgcenter report -W x` завершаются сообщением - `report type is not specified, quit` и ненулевым кодом возврата. + `report type is not specified, quit`. **Код возврата при этом 0** (проверено): сообщение + печатается, но `os.Exit(1)` не вызывается. Поведение существующее и общее для всех типов отчёта, + но именно эта фича делает его болезненным — обёртка вида + `pgcenter report -W -f dump.tar > out.txt || alert` запишет пустой файл и отрапортует успех. + Чинить код возврата в этой фиче не будем (это меняет поведение всех отчётов) — пойдёт в техдолг. - [ ] Роль без прав `pg_monitor`: экран `archiver` вместо таблицы показывает текст ошибки Postgres (`permission denied for function pg_ls_archive_statusdir`), pgcenter не падает, а на следующем тике повторяет попытку; экран `wal` в тех же условиях ведёт себя так же, как до фичи. @@ -302,7 +306,8 @@ Archiver 1423 84512 00000001000000A300000012 00:47:12 3781 0000 - [ ] Под ролью с правами `pg_monitor` (без суперпользователя) verbose-панель [010] показывает числовое значение backlog архивации, а не `n/a`. Под суперпользователем на кластере со штатным каталогом `archive_status` значение не изменилось; единственное расхождение — отсутствующий - каталог, где теперь показывается `0 B` вместо `n/a` (см. «Дизайн и интерфейс»). + каталог, где теперь показывается ноль вместо `n/a` — именно `0`, а не `0 B` + (см. «Дизайн и интерфейс»). - [ ] `pgcenter report -d -W w` описывает в том числе колонку `fpi,KiB`. Описание колонок в проекте **не версионно-зависимое** — это одна статическая константа на тип отчёта (она и сегодня описывает `write`/`sync`, убранные в PG 18), поэтому новая строка печатается для архивов любой @@ -514,6 +519,46 @@ Archiver 1423 84512 00000001000000A300000012 00:47:12 3781 0000 ## Post-implementation - +Updated: 2026-08-06 + +Все 34 критерия приёмки пройдены: автоматическая половина — в CI-образе проекта на фикстурах +PG 14–19 (полный `make test` зелёный, ноль гонок, `make lint` и `make vuln` чисто), ручная — на живом +стенде. Ниже — только то, чем результат отличается от текста этого спека. + +### Divergences from original spec + +- **Замер стоимости и его заранее согласованный исход.** Спек фиксировал правило: «если замер покажет + время, сопоставимое с интервалом обновления… троттлинг возвращается отдельным решением». Замер это + условие **выполнил** (≈1108 мс против интервала 1 с; при включённом verbose частота обновления + падает с 1.0 до 1.9 с/тик на всех экранах при 200 005 файлах `.ready`) — но владелец роадмапа, + увидев числа, пересмотрел не решение о троттлинге, а его основание: стоимость линейна (~5.5 мкс на + файл), а удвоение требует 200 000 сегментов ≈ 3.1 ТБ неархивированного WAL. Итог: **троттлинга нет, + и в регистр техдолга он не заводится** — запись в регистре была бы обязательством починить, + которого нет. Числа живут в `docs/decisions-log.md` (Decision 20), чтобы следующий спрашивающий не + мерил заново. +- **Строка помощи флага живёт не там, где предполагал спек.** Критерий требовал обновить «строку + помощи самого флага — то, что печатает `pgcenter report --help`». Фактически `--help` для `report` + полностью перекрыт рукописным `printReportHelp()` (`cmd/help.go`), а `Usage` cobra-флага + пользователю не показывается вообще. Критерий выполнен по смыслу, но правкой **второго** файла + (`cmd/help.go`) сверх пары `cmd/report/report.go` + release notes. Само расхождение «описание флага + в cobra vs `printReportHelp()`» системное и ничем не проверяется. + +### Added during implementation + +- **Общий тестовый хелпер `postgres.SetupTestRole`** (`internal/postgres/testing.go`) — спек требовал + проверять привилегии автотестом через `SET ROLE`, но не говорил, где живёт создание ролей. Хелпер + один на два пакета и **не принимает `*testing.T`**: файл без build-тега попадает в релизный бинарь. +- **Правка `cmd/help.go`** — см. выше. +- **Пятый исправленный комментарий** сверх четырёх названных: комментарий поля + `ArchivingBacklogValid` утверждал, что поле деградирует до `n/a` при `archive_mode=off`. Это + неверно — каталог статусов создаётся `initdb`, агрегат возвращает настоящий `0`. + +### Descoped / Deferred + +- Ничего из спека не выброшено. Всё, что спек сознательно оставлял за рамками (README, код возврата + `report`, троттлинг), таким и осталось. +- В регистр техдолга по итогам фичи заведены ровно три пункта: [034] `report` завершается кодом 0 на + **всех** путях отказа (спек предсказывал это для `-W`; замер показал, что свойство общее), [035] + обрезание имён WAL-сегментов до ширины заголовка при входе на экран до появления значений + (предсуществующее, подтверждено A/B с бинарём из `master`), [036] `// indirect` у `pflag` при + прямом импорте в тесте. diff --git a/docs/metrics/017-feat-wal-archiver-metrics-summary.md b/docs/metrics/017-feat-wal-archiver-metrics-summary.md new file mode 100644 index 00000000..677fe28e --- /dev/null +++ b/docs/metrics/017-feat-wal-archiver-metrics-summary.md @@ -0,0 +1,85 @@ +# Metrics Summary: 017-feat-wal-archiver + +## Context + +| Dimension | Value | +|-----------|-------| +| Model | Opus 5 (1M context) | +| Feature size | M | +| Started | 2026-08-05 | +| Completed | 2026-08-06 | + +**Caveat:** phase timestamps were recorded manually by the orchestrator and are approximate to ~5 +minutes. Human wait was instrumented only in the `user_spec` phase (the interview); the approval +gates of the later phases are counted as touch time, so flow efficiency below is an upper bound. + +## Timeline + +| Phase | Duration (min) | Touch (min) | Human wait (min) | +|-------|---------------|-------------|------------------| +| User Spec | 70 | 48 | 22 | +| Tech Spec | 50 | 50 | 0 | +| Task Decomposition | 295 | 295 | 0 | +| Feature Execution | 450 | 450 | 0 | +| Done | 17 | 17 | 0 | +| **Sum of phases** | **882** | **860** | **22** | +| Lead time (first start → last end) | 1130 | | | +| Idle between phases | 248 | | | + +**Flow efficiency: 76.1%** (860 touch / 1130 lead). The 248 idle minutes are two overnight gaps +between decomposition, execution and finalization, not a queue. + +Task decomposition took as long as it did for a reason worth keeping: two validation rounds found +tasks whose mutations targeted files those same tasks were forbidden to touch, which is a defect that +only shows up when a validator actually tries to run the plan. + +## Quality + +| Metric | Value | +|--------|-------| +| Validation rounds | user_spec: 2, tech_spec: 3, task_decomposition: 2 | +| Validation findings (crit/major/minor) | 1 / 18 / 74 (12 reports) | +| Review rounds by task | 01:2, 02:1, 03:2, 04:2, 05:2, 06:3, 07:2, 08:—, 09:—, 10:— | +| Review findings (crit/major/minor) | 0 / 15 / 96 (36 reports) | +| First pass rate | 0% (0 of 7 reviewed tasks cleared round 1 without a major) | + +**A 0% first-pass rate here is a signal about the reviewers, not about broken code.** Every task +reached round 1 with a green suite; what the majors found was almost uniformly the same class — a +test that passes and cannot fail. Task 02: none of the four TDD-anchor asserts pinned the `/1024` +conversion, so replacing `round(wal_fpi_bytes / 1024, 2)` with the bare column left the suite green +while the screen would have shown bytes under a KiB header. Task 06: binding `W` to the wrong menu +left the entire filtered run green, because gocui keeps registered handlers unexported — which is +what forced `keybindings()` to be split so the table row itself became callable. Tasks 01 and 03: the +fixtures run with an empty `archive_status` directory, so a live check could not tell +`count(*) FILTER (WHERE name LIKE '%.ready')` from a bare `count(*)`, and the predicate had to be +pinned by a server-free structural test instead. Three tasks (08, 09, 10) had no reviewer cycle — +golden tests, documentation and QA — so the rate is computed over seven. + +Tasks 06 needed a third round; every other reviewed task closed in two. + +## Volume + +| Metric | Value | +|--------|-------| +| Interview questions | 12 | +| Tasks | 10 (in 4 waves) | +| Agents spawned | ~90 (implementers, reviewers, validators) | +| Commits | 29 (on the feature branch) | + +## Verification + +| Gate | Result | +|------|--------| +| `make test` (`-race -p 1`, PG 14–19 fixtures in the CI image) | pass — 1085 PASS / 86 SKIP / 0 FAIL, 0 data races | +| `make lint` (golangci-lint + gosec) | pass, 0 issues | +| `make vuln` (govulncheck) | pass | +| Acceptance criteria, automated half | 24 of 34 | +| Acceptance criteria, stand run | 10 of 10, 0 FAIL | + +All 86 skips are EOL-cluster subtests (PG 9.4–13) absent from the test image — debt [019] — with no +skip anywhere in the PG 14–19 range and none in the feature's own tests. + +The manual gate ran twice: the first attempt found the stand unreachable and was reported as a +blocker rather than waved through, and the run was repeated on 2026-08-06 once it came back. That is +also where the `archive_status` cost measurement was taken (200 005 `.ready` files) and where the +truncation defect now registered as debt [035] was found by A/B against a `master`-built binary. diff --git a/docs/metrics/017-feat-wal-archiver-metrics.json b/docs/metrics/017-feat-wal-archiver-metrics.json new file mode 100644 index 00000000..7af09f48 --- /dev/null +++ b/docs/metrics/017-feat-wal-archiver-metrics.json @@ -0,0 +1,98 @@ +{ + "meta": { + "schema_version": "1.0", + "feature_id": "017-feat-wal-archiver", + "feature_name": "wal-archiver", + "feature_base": "docs/features/017-feat-wal-archiver/017-feat-wal-archiver", + "project": "pgcenter", + "feature_size": "M", + "model": "Opus 5 (1M context)", + "date_started": "2026-08-05T16:59:47Z", + "date_completed": "2026-08-06T11:50:00Z" + }, + "phases": { + "user_spec": { + "start_time": "2026-08-05T16:59:47Z", + "end_time": "2026-08-05T18:09:29Z", + "duration_min": 70, + "touch_time_min": 48, + "human_wait_time_min": 22, + "steps": {} + }, + "tech_spec": { + "start_time": "2026-08-05T18:15:00Z", + "end_time": "2026-08-05T19:05:00Z", + "duration_min": 50, + "touch_time_min": 50, + "human_wait_time_min": 0, + "steps": {} + }, + "task_decomposition": { + "start_time": "2026-08-05T19:10:00Z", + "end_time": "2026-08-06T00:05:00Z", + "duration_min": 295, + "touch_time_min": 295, + "human_wait_time_min": 0, + "steps": {} + }, + "feature_execution": { + "start_time": "2026-08-06T02:00:00Z", + "end_time": "2026-08-06T09:30:00Z", + "duration_min": 450, + "touch_time_min": 450, + "human_wait_time_min": 0, + "steps": {} + }, + "done": { + "start_time": "2026-08-06T11:33:00Z", + "end_time": "2026-08-06T11:50:00Z", + "duration_min": 17, + "touch_time_min": 17, + "human_wait_time_min": 0, + "steps": {} + } + }, + "quality": { + "validation_rounds": { + "user_spec": 2, + "tech_spec": 3, + "task_decomposition": 2 + }, + "validation_findings": { + "critical": 1, + "major": 18, + "minor": 74 + }, + "review_rounds": { + "task_01": 2, + "task_02": 1, + "task_03": 2, + "task_04": 2, + "task_05": 2, + "task_06": 3, + "task_07": 2 + }, + "review_findings": { + "critical": 0, + "major": 15, + "minor": 96 + }, + "first_pass_rate_pct": 0 + }, + "volume": { + "interview_questions": 12, + "tasks_count": 10, + "waves_count": 4, + "agents_spawned": 90, + "commits_count": 29 + }, + "summary": { + "total_lead_time_min": 1130, + "sum_of_phase_durations_min": 882, + "idle_between_phases_min": 248, + "total_touch_time_min": 860, + "total_human_wait_time_min": 22, + "flow_efficiency_pct": 76.1, + "note": "Phase timestamps recorded manually by the orchestrator; approximate to ~5 min. Only the user_spec phase had human wait instrumented." + } +} diff --git a/docs/roadmap-0.12.0.md b/docs/roadmap-0.12.0.md index 820c5295..19be0e7d 100644 --- a/docs/roadmap-0.12.0.md +++ b/docs/roadmap-0.12.0.md @@ -301,7 +301,24 @@ issue #122. TUI-first was never about saving effort — it was about not freezin ### [017] WAL and archiving area — one pass -- **Status:** planned +- **Status:** done (2026-08-06) — archived as `docs/features/archive/017-feat-wal-archiver`. All 34 + acceptance criteria pass: the automated half in the project CI image against PG 14–19 fixtures + (full suite green, zero races, `make lint` and `make vuln` clean), the manual half on a live stand, + including the check the manual gate exists for — the screen caption printed **exactly once on each + of the two entry paths**, verified separately for the `w` hotkey and the `W` menu. +- **What actually shipped, against the three-item plan below: four pieces, not three.** The fourth + was not scope creep but a defect the area pass exposed — the [010] verbose panel counted the + backlog through `pg_ls_dir`, which is superuser-only, so the `pg_monitor` role this roadmap keeps + citing as the typical monitoring role saw `n/a` and got no first signal at all. Moving it to + `pg_ls_archive_statusdir()` was the one-pass mandate applied honestly. Three other deltas worth + recording: the screen has **nine** columns, not the seven this document's cross-cutting policy + assumed (`ready` plus a `source` identity column on top of the `pg_stat_archiver` set); `-W` became + a string flag (`-W w` / `-W a`), an accepted **breaking** CLI change with its own release-notes + entry; and the "do not make an incident worse" principle below was tested rather than asserted — + the `archive_status` walk was measured on 200 005 `.ready` files, found to halve the refresh rate + at that size and be inside the noise at realistic ones, and deliberately left unthrottled. The + numbers are in `docs/decisions-log.md`, not in the tech-debt register: there is no intention to fix + it, so it is not a commitment. - **Value:** medium — low frequency, high severity (archiving failure → WAL accumulation → disk fill). Honest scoping: the *most* valuable archiving metric, the backlog of `.ready` files, **already ships** in the [010] verbose panel (`OverviewArchivingBacklog`). What diff --git a/docs/tech-debt.md b/docs/tech-debt.md index 08332a50..91d7d16b 100644 --- a/docs/tech-debt.md +++ b/docs/tech-debt.md @@ -7,6 +7,41 @@ Reviewed at the start of tech-spec planning to avoid worsening existing debt. ## Active Debt +### [034] `pgcenter report` exits with code 0 on every failure path + +**Added:** 2026-08-06 (surfaced during feature: 017-feat-wal-archiver) +**Severity:** Medium — loud on a terminal, silent to a script +**Area:** `cmd/pgcenter.go` (`main`), `cmd/report/report.go` + +`main()` prints the error and returns without `os.Exit(1)`, so every `report` failure exits 0. +Measured on the built binary, not inferred: `pgcenter report -W -f dump.tar` prints +`report type is not specified, quit` and exits 0, and `-W` as the last token prints cobra's +`flag needs an argument: 'W' in -W` and also exits 0. This is pre-existing and repo-wide — `-J q` +behaves identically — but the breaking `-W` change of 017 is what makes it bite: a legacy wrapper +like `pgcenter report -W -f dump.tar > out.txt || alert` now writes an empty file and reports +success, so the one shape most likely to break is the one least likely to be noticed. + +Not fixed inside the feature because the exit code is shared by every report type and every +subcommand error path — changing it is a behaviour change for all of them and needs its own review. +Called out in `doc/release-notes/v0.12.0.md` so users are not the ones to discover it. + +### [035] WAL segment names truncate to header width when the screen is opened before any value exists + +**Added:** 2026-08-06 (surfaced during feature: 017-feat-wal-archiver, stand run) +**Severity:** Low — needs the screen opened on a cluster that has archived nothing yet +**Area:** `internal/align/align.go`, `top/stat.go` (`printDataCell`, `alignViewToResult`) + +Column widths are computed from the first batch and `view.Aligned` is never reset while a screen +stays open, so entering `archiver` on a cluster that has never archived freezes the two WAL-name +columns at header width; values arriving later render as `000000010000~` / `0000000100~`. Entering +the screen when the values already exist shows the names in full. + +Pre-existing, confirmed by A/B against a `master`-built binary — the feature's diff touches neither +`align.SetAlign` nor `printDataCell` nor `alignViewToResult`. Recorded rather than left implicit +because WAL segment names differ only in their tail, so truncating from the right removes exactly the +part that identifies the segment — the reason the column is on the screen at all. + + ### [027] Messages printed after a dialog closes are never visible **Added:** 2026-08-03 (surfaced during feature: 015-feat-tui-papercuts, stand run) @@ -307,6 +342,23 @@ third unsafe consumer alongside `diff`, so fixing `diff` alone would not close t ## Resolved Debt +### [036] `go.mod` marks `spf13/pflag` as `// indirect` although a test imports it directly + +**Added:** 2026-08-06 (surfaced during feature: 017-feat-wal-archiver) +**Severity:** Trivial +**Area:** `go.mod`, `cmd/report/report_test.go` + +The flag-definition test imports `github.com/spf13/pflag` directly, but the `require` line still +carries `// indirect`. Nothing fails today — the default readonly module mode builds, tests and lints +fine, and CI has no `go mod tidy -diff` gate — but any build with `-mod=mod` rewrites `go.mod` and +dirties the working tree. Fix is one `go mod tidy` run at a moment when no parallel work holds the +branch. + +**Resolved:** 2026-08-06, during finalization of 017-feat-wal-archiver — `go mod tidy` moved +`github.com/spf13/pflag` into the direct requires, matching the direct import in +`cmd/report/report_test.go`. One line; build and the report tests verified after. + + ### [025] `PGresult.sort` does not bounds-check its sort key **Added:** 2026-07-25 (surfaced during feature: 013-feat-activity-xmin-horizon, security audit) diff --git a/go.mod b/go.mod index 19845684..a1ed5244 100644 --- a/go.mod +++ b/go.mod @@ -9,6 +9,7 @@ require ( github.com/jehiah/go-strftime v0.0.0-20171201141054-1d33003b3869 github.com/jroimartin/gocui v0.5.0 github.com/spf13/cobra v1.10.2 + github.com/spf13/pflag v1.0.10 github.com/stretchr/testify v1.11.1 golang.org/x/term v0.42.0 ) @@ -24,7 +25,6 @@ require ( github.com/nsf/termbox-go v1.1.1 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect github.com/rogpeppe/go-internal v1.14.1 // indirect - github.com/spf13/pflag v1.0.10 // indirect golang.org/x/sys v0.43.0 // indirect golang.org/x/text v0.39.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect diff --git a/internal/postgres/testing.go b/internal/postgres/testing.go index a72d8e3a..8906572c 100644 --- a/internal/postgres/testing.go +++ b/internal/postgres/testing.go @@ -1,6 +1,15 @@ package postgres -import "fmt" +import ( + "fmt" + "regexp" +) + +// testRoleNameRE constrains the role name SetupTestRole interpolates into its statements. The name +// is an SQL identifier, so it cannot travel as a $1 placeholder; this turns "callers pass literal +// constants" from a comment into an enforced invariant. Lowercase-only is not a restriction: the +// identifier positions are unquoted, so PostgreSQL down-folds the name anyway. +var testRoleNameRE = regexp.MustCompile(`^[a-z_][a-z0-9_]*$`) // NewTestConfig returns test config used for testing purposes. func NewTestConfig() (Config, error) { @@ -45,3 +54,41 @@ func NewTestConnectVersion(version int) (*DB, error) { } return Connect(config) } + +// SetupTestRole ensures a test role exists on the connected cluster and switches the session to it. +// Creation is idempotent (DO block guarded on pg_roles): the role is created NOLOGIN and +// non-superuser when missing, and granted pg_monitor when pgMonitor is true. The caller is +// responsible for RESET ROLE, normally in a defer immediately after a successful call. +// +// It returns an error rather than taking *testing.T on purpose: this file carries no build tag and +// is compiled into the released pgcenter binary, so it must not import the testing package. +// +// The role name is an SQL identifier, not a value, so it cannot travel as a $1 placeholder and is +// interpolated instead. Callers must pass literal constants - never user input. +func SetupTestRole(db *DB, name string, pgMonitor bool) error { + if !testRoleNameRE.MatchString(name) { + return fmt.Errorf("invalid test role name %q", name) + } + + create := fmt.Sprintf( + "DO $$ BEGIN IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = '%s') "+ + "THEN CREATE ROLE %s NOLOGIN NOSUPERUSER; END IF; END $$", name, name, + ) + if _, err := db.Exec(create); err != nil { + return fmt.Errorf("create role %s failed: %w", name, err) + } + + // GRANT is naturally idempotent, unlike bare CREATE ROLE. A role that must hold nothing gets no + // GRANT at all rather than a REVOKE, so two roles created by neighbouring tests never interfere. + if pgMonitor { + if _, err := db.Exec(fmt.Sprintf("GRANT pg_monitor TO %s", name)); err != nil { + return fmt.Errorf("grant pg_monitor to %s failed: %w", name, err) + } + } + + if _, err := db.Exec(fmt.Sprintf("SET ROLE %s", name)); err != nil { + return fmt.Errorf("set role %s failed: %w", name, err) + } + + return nil +} diff --git a/internal/query/archiver.go b/internal/query/archiver.go new file mode 100644 index 00000000..29792e40 --- /dev/null +++ b/internal/query/archiver.go @@ -0,0 +1,59 @@ +package query + +const ( + // PgStatArchiverDefault defines query for pg_stat_archiver plus the .ready archiving backlog. + // One row, 9 columns, layout (0-based): + // 0 source - literal 'Archiver', the stable row identity across samples. + // 1 ready - count of *.ready files in the archive status directory. + // 2 archived - archived_count, cumulative. + // 3 last_archived - last_archived_wal, NULL until something is archived. + // 4 archived_age - age of last_archived_time, NULL until something is archived. + // 5 failed - failed_count, cumulative. + // 6 last_failed - last_failed_wal, NULL until something fails. + // 7 failed_age - age of last_failed_time, NULL until something fails. + // 8 stats_age - age of stats_reset, never NULL. + // + // There is no version branch: pg_stat_archiver is schema-identical on PG 14 through PG 19 + // (verified against live pg_attribute on 14/17/18/19), and pg_ls_archive_statusdir() exists on + // every one of them - so one query text serves all supported versions. + // + // The ready sub-select calls pg_ls_archive_statusdir(), which is superuser + pg_monitor only - + // the same privilege class as the pg_ls_waldir() the wal screen already calls unconditionally. + // A role without pg_monitor therefore loses the WHOLE screen, by design (Decision 4): PostgreSQL + // checks EXECUTE at function-node initialisation, so hiding the call behind + // has_function_privilege() was measured not to work - CASE with an uncorrelated subquery, CASE + // with a correlated subquery and LEFT JOIN LATERAL all fail alike. + // + // Nothing here is diffed (the selector returns DiffIntvl{0,0}), so calculateDelta + // short-circuits before diff() and the four NULL-able columns never reach strconv.ParseInt(""). + // That is what makes them safe WITHOUT coalesce, and a blank cell is the honest rendering of + // "this cluster has never archived" (Decision 3; precedent ADR [013] backend_xid). The literal + // at column 0 is safe for the same reason. + // + // The two server-supplied WAL-name columns are rendered as-is, with no escape sanitisation + // (Decision 16): PostgreSQL only records names that passed its own VALID_XFN_CHARS filter (hex + // digits plus the .history/.backup/.partial suffixes), a set containing no ESC and no control + // characters, so these columns cannot carry a terminal escape sequence even if an operator + // hand-places a bogus .ready file. Tech-debt [029] is neither widened nor closed here. + PgStatArchiverDefault = "SELECT 'Archiver' AS source, " + + "(SELECT count(*) FILTER (WHERE name LIKE '%.ready') FROM pg_ls_archive_statusdir()) AS ready, " + + "archived_count AS archived, " + + "last_archived_wal AS last_archived, " + + "date_trunc('seconds', now() - last_archived_time)::text AS archived_age, " + + "failed_count AS failed, " + + "last_failed_wal AS last_failed, " + + "date_trunc('seconds', now() - last_failed_time)::text AS failed_age, " + + "date_trunc('seconds', now() - stats_reset)::text AS stats_age " + + "FROM pg_stat_archiver" +) + +// SelectStatArchiverQuery returns the query, column count and diff interval for the archiver screen. +// pg_stat_archiver is schema-stable across every supported version, so a single version-independent +// query is returned and the version parameter is unused (named _ per revive); it is kept for +// signature symmetry with SelectStatWALQuery and the other selectors. DiffIntvl{0,0} is not an unset +// placeholder - it states that nothing is diffed: both counters and the .ready backlog render as +// absolute values, which is what an operator cross-checks against the PostgreSQL log during an +// incident. +func SelectStatArchiverQuery(_ int) (string, int, [2]int) { + return PgStatArchiverDefault, 9, [2]int{0, 0} +} diff --git a/internal/query/archiver_test.go b/internal/query/archiver_test.go new file mode 100644 index 00000000..f80c7bcc --- /dev/null +++ b/internal/query/archiver_test.go @@ -0,0 +1,407 @@ +package query + +import ( + "database/sql" + "fmt" + "strconv" + "strings" + "testing" + + "github.com/jackc/pgx/v5/pgconn" + "github.com/lesovsky/pgcenter/internal/postgres" + "github.com/stretchr/testify/assert" +) + +// archiverVersions lists the Postgres majors the archiver query must work on. pg_stat_archiver is +// schema-identical on all of them and pg_ls_archive_statusdir() exists on all of them, so every +// version runs the same query text. +var archiverVersions = []int{140000, 150000, 160000, 170000, 180000, 190000} + +// archiverColumns is the column order locked by the user-spec (tech-spec Data Models / Decision 14). +// The live tests assert this list by NAME, not just its length: a column inserted mid-layout would +// keep the count right while shifting every index the view, record and report layers depend on. +var archiverColumns = []string{ + "source", "ready", "archived", "last_archived", "archived_age", + "failed", "last_failed", "failed_age", "stats_age", +} + +// Role names are specific to this test file so a role left behind on a long-lived cluster is +// attributable and cannot be confused with the roles other test files create through the same helper. +const ( + archiverRoleMonitor = "pgcenter_test_archiver_monitor" + archiverRoleNoRole = "pgcenter_test_archiver_norole" +) + +func Test_SelectStatArchiverQuery(t *testing.T) { + testcases := []struct { + version int + wantNcols int + wantDiffIntvl [2]int + }{ + {version: 140000, wantNcols: 9, wantDiffIntvl: [2]int{0, 0}}, + {version: 150000, wantNcols: 9, wantDiffIntvl: [2]int{0, 0}}, + {version: 160000, wantNcols: 9, wantDiffIntvl: [2]int{0, 0}}, + {version: 170000, wantNcols: 9, wantDiffIntvl: [2]int{0, 0}}, + {version: 180000, wantNcols: 9, wantDiffIntvl: [2]int{0, 0}}, + {version: 190000, wantNcols: 9, wantDiffIntvl: [2]int{0, 0}}, + // The selector ignores its argument by verified fact, so the invariant is "any argument", + // not "these six". A future major, a version below the project floor and the zero value all + // have to come back identical, or a branch was added. + {version: 200000, wantNcols: 9, wantDiffIntvl: [2]int{0, 0}}, + {version: 130000, wantNcols: 9, wantDiffIntvl: [2]int{0, 0}}, + {version: 0, wantNcols: 9, wantDiffIntvl: [2]int{0, 0}}, + } + + for _, tc := range testcases { + t.Run(fmt.Sprintf("version/%d", tc.version), func(t *testing.T) { + gotQuery, gotNcols, gotDiffIntvl := SelectStatArchiverQuery(tc.version) + + // Guards only against a version branch being introduced - the query text itself is + // pinned by Test_StatArchiverQuery_Structure. + assert.Equal(t, PgStatArchiverDefault, gotQuery) + assert.Equal(t, tc.wantNcols, gotNcols) + assert.Equal(t, tc.wantDiffIntvl, gotDiffIntvl) + }) + } +} + +// Test_StatArchiverQuery_Structure pins the query's shape with no server, so the locked column order +// and the privileged .ready predicate stay guarded on a plain host run, where every live test skips. +// The predicate needs this test specifically: the fixtures have an empty archive status directory, +// so count(*) FILTER (WHERE name LIKE '%.ready') and a bare count(*) are both 0 on every cluster and +// no live assertion can tell them apart. +func Test_StatArchiverQuery_Structure(t *testing.T) { + // The .ready filter is the only logic in the query and the feature's headline number. + assert.Contains(t, PgStatArchiverDefault, "count(*) FILTER (WHERE name LIKE '%.ready')", + "the backlog must count .ready files only - a bare count(*) or a wider pattern is a different number") + assert.Contains(t, PgStatArchiverDefault, "FROM pg_ls_archive_statusdir()", + "the backlog must come from pg_ls_archive_statusdir(), the pg_monitor-granted function") + + // Locked column order (Decision 14): every alias present, and in this exact sequence. The needle + // carries the alias's delimiter so "archived" cannot match inside "archived_age" (nor "failed" + // inside "failed_age") and blame the wrong column. + prev := -1 + for i, col := range archiverColumns { + needle := " AS " + col + "," + if i == len(archiverColumns)-1 { + needle = " AS " + col + " FROM" + } + + idx := strings.Index(PgStatArchiverDefault, needle) + assert.NotEqual(t, -1, idx, "query must select the %q column", col) + assert.Greater(t, idx, prev, "%q must follow the previous locked column", col) + prev = idx + } + + assert.True(t, strings.HasSuffix(PgStatArchiverDefault, "FROM pg_stat_archiver"), + "pg_stat_archiver must be the outer relation") + + // Decision 3: nothing on this screen is diffed, so calculateDelta short-circuits before diff() + // and the strconv.ParseInt("") trap that forces coalesce(...,0) elsewhere does not apply. A + // blank cell is the honest rendering of "this cluster has never archived". + assert.NotContains(t, strings.ToLower(PgStatArchiverDefault), "coalesce", + "no column on this screen is diffed, so no column may be coalesced (Decision 3)") +} + +// Test_StatArchiverQueries tests query execution against all supported Postgres versions and pins the +// live result shape: 9 columns in the locked order, exactly one row. +func Test_StatArchiverQueries(t *testing.T) { + for _, version := range archiverVersions { + t.Run(fmt.Sprintf("pg_stat_archiver/%d", version), func(t *testing.T) { + tmpl, wantNcols, _ := SelectStatArchiverQuery(version) + + opts := NewOptions(version, "f", "off", 256, "public") + q, err := Format(tmpl, opts) + assert.NoError(t, err) + assert.NotContains(t, q, "{{", "formatted query must carry no template artifacts") + + conn := connectArchiverFixture(t, version) + defer conn.Close() + + descs, nrows, err := runArchiverQuery(conn, q) + assert.NoError(t, err) + + assert.Len(t, descs, wantNcols) + assert.Equal(t, archiverColumns, descs, "live column names must match the locked order") + assert.Equal(t, 1, nrows, "pg_stat_archiver is a single-row view") + }) + } +} + +// Test_StatArchiverQuery_NullsStayNull verifies Decision 3 against a live cluster: the four columns +// that are NULL on a cluster that has never archived stay SQL NULL and are not coalesced into an +// invented value. It also pins the values that are fixed on such a cluster, so the 'Archiver' row +// identity, the two counters and the date_trunc truncation are falsifiable rather than assumed. +// The no-coalesce guard on the query text itself lives in Test_StatArchiverQuery_Structure, which +// needs no server. +func Test_StatArchiverQuery_NullsStayNull(t *testing.T) { + // The fixtures never archived: the two WAL names and the two age columns are NULL, while the + // literal, the .ready count, both bigint counters and stats_age are always set. + wantValid := map[string]bool{ + "source": true, + "ready": true, + "archived": true, + "last_archived": false, + "archived_age": false, + "failed": true, + "last_failed": false, + "failed_age": false, + "stats_age": true, + } + + for _, version := range archiverVersions { + t.Run(fmt.Sprintf("pg_stat_archiver/%d", version), func(t *testing.T) { + tmpl, _, _ := SelectStatArchiverQuery(version) + + q, err := Format(tmpl, NewOptions(version, "f", "off", 256, "public")) + assert.NoError(t, err) + + conn := connectArchiverFixture(t, version) + defer conn.Close() + + // Scan into sql.NullString receivers - the very type the stats pipeline uses + // (stat.PGresult.Values), so this asserts what the screen will actually see. + values := make([]sql.NullString, len(archiverColumns)) + pointers := make([]any, len(values)) + for i := range pointers { + pointers[i] = &values[i] + } + + err = conn.QueryRow(q).Scan(pointers...) + assert.NoError(t, err) + if err != nil { + return + } + + for i, col := range archiverColumns { + assert.Equal(t, wantValid[col], values[i].Valid, "column %q NULL-ness", col) + } + + assert.Equal(t, "Archiver", values[0].String, + "column 0 is the row identity the single-row screen matches itself by across samples") + assert.Equal(t, "0", values[2].String, "archived counter on a cluster that never archived") + assert.Equal(t, "0", values[5].String, "failed counter on a cluster that never archived") + assert.Regexp(t, `^-?(\d+ days? )?\d{2}:\d{2}:\d{2}$`, values[8].String, + "stats_age must be truncated to whole seconds by date_trunc") + + // ready counts a live directory, so assert its type rather than a value - pinning 0 + // would couple the suite to the state of the archive status directory. + ready, err := strconv.Atoi(values[1].String) + assert.NoError(t, err, "ready must be an integer, got %q", values[1].String) + assert.GreaterOrEqual(t, ready, 0) + }) + } +} + +// Test_StatArchiverQuery_PgMonitorRoleSucceeds proves Decision 4 in the positive direction: a role +// holding pg_monitor and nothing else can run the whole query, including the privileged +// pg_ls_archive_statusdir() call. The fixture connection is the superuser postgres, so the test +// asserts the session is really restricted BEFORE running the query - without that guard the test +// would pass identically as superuser and prove nothing about privileges. +func Test_StatArchiverQuery_PgMonitorRoleSucceeds(t *testing.T) { + for _, version := range archiverVersions { + t.Run(fmt.Sprintf("pg_stat_archiver/%d", version), func(t *testing.T) { + tmpl, wantNcols, _ := SelectStatArchiverQuery(version) + + q, err := Format(tmpl, NewOptions(version, "f", "off", 256, "public")) + assert.NoError(t, err) + + conn := connectArchiverFixture(t, version) + defer conn.Close() + + err = postgres.SetupTestRole(conn, archiverRoleMonitor, true) + assert.NoError(t, err) + if err != nil { + return + } + // RESET ROLE immediately after the successful SET ROLE, so it runs even when an + // assertion below fails - a leaked SET ROLE would silently change what later + // assertions on this connection see. + defer resetRole(t, conn, archiverRoleMonitor) + + if !assertRestrictedSession(t, conn, archiverRoleMonitor, true) { + return + } + + descs, nrows, err := runArchiverQuery(conn, q) + assert.NoError(t, err) + + assert.Len(t, descs, wantNcols) + assert.Equal(t, 1, nrows) + }) + } +} + +// Test_StatArchiverQuery_WithoutPgMonitorFails proves Decision 4 in the negative direction: a role +// with neither superuser nor pg_monitor loses the whole screen, and it loses it on +// pg_ls_archive_statusdir() specifically. The assertion is pinned to SQLSTATE 42501 and to the +// function name - asserting merely "an error occurred" would pass on a syntax typo. +func Test_StatArchiverQuery_WithoutPgMonitorFails(t *testing.T) { + for _, version := range archiverVersions { + t.Run(fmt.Sprintf("pg_stat_archiver/%d", version), func(t *testing.T) { + tmpl, _, _ := SelectStatArchiverQuery(version) + + q, err := Format(tmpl, NewOptions(version, "f", "off", 256, "public")) + assert.NoError(t, err) + + conn := connectArchiverFixture(t, version) + defer conn.Close() + + err = postgres.SetupTestRole(conn, archiverRoleNoRole, false) + assert.NoError(t, err) + if err != nil { + return + } + defer resetRole(t, conn, archiverRoleNoRole) + + if !assertRestrictedSession(t, conn, archiverRoleNoRole, false) { + return + } + + _, _, err = runArchiverQuery(conn, q) + assert.Error(t, err) + + var pgErr *pgconn.PgError + if assert.ErrorAs(t, err, &pgErr) { + assert.Equal(t, "42501", pgErr.Code, "must fail with insufficient_privilege") + assert.Contains(t, pgErr.Message, "pg_ls_archive_statusdir", + "the failure must name the privileged call, not just any error") + } + }) + } +} + +// assertRestrictedSession asserts the session really runs as the named non-superuser role AND that +// the role's privileges are exactly what the caller intends. It is the load-bearing guard of both +// privilege tests: deleting the SET ROLE would otherwise leave them silently passing as the fixture +// superuser - the exact failure mode that let a wrong privilege assumption survive a whole release. +// +// The membership assertion matters because the roles are cluster-global and SetupTestRole never +// normalises a role that already exists: without it the positive test could silently decay from +// "pg_monitor is sufficient" to "some privileged role works" after any stray GRANT. +// +// Returns false when the session is not as intended, so the caller can stop before the query and +// redden on the guard rather than on the query. +func assertRestrictedSession(t *testing.T, conn *postgres.DB, wantRole string, wantPgMonitor bool) bool { + t.Helper() + + var ( + currentUser string + isSuper bool + hasMonitor bool + memberOf []string + ) + err := conn.QueryRow( + "SELECT current_user::text, "+ + "(SELECT rolsuper FROM pg_roles WHERE rolname = current_user), "+ + "pg_has_role(current_user, 'pg_monitor', 'USAGE'), "+ + // DISTINCT because PG 16+ stores one pg_auth_members row per grantor, so a role + // re-granted by hand (the mutation procedure does exactly that) would list twice. + "coalesce((SELECT array_agg(DISTINCT r.rolname::text ORDER BY r.rolname::text) "+ + "FROM pg_auth_members m JOIN pg_roles r ON r.oid = m.roleid "+ + "WHERE m.member = (SELECT oid FROM pg_roles WHERE rolname = current_user)), ARRAY[]::text[])", + ).Scan(¤tUser, &isSuper, &hasMonitor, &memberOf) + if !assert.NoError(t, err) { + return false + } + + okUser := assert.Equal(t, wantRole, currentUser, "session must run as the test role") + okSuper := assert.False(t, isSuper, "the test role must not be a superuser") + okMonitor := assert.Equal(t, wantPgMonitor, hasMonitor, "pg_monitor membership of the test role") + + var okMembers bool + if wantPgMonitor { + okMembers = assert.Equal(t, []string{"pg_monitor"}, memberOf, + "the test role must hold pg_monitor and nothing else") + } else { + okMembers = assert.Empty(t, memberOf, "the deny role must hold no role membership at all") + } + + return okUser && okSuper && okMonitor && okMembers +} + +// resetRole restores the session to the fixture superuser and asserts the reset took effect. +// Both privilege tests use a dedicated connection they close at the end of the subtest, so a leaked +// SET ROLE cannot currently reach a later test - RESET ROLE is required by Decision 18 and is the +// belt to that braces. Should the two tests ever share one connection, this is the guard they rely on. +func resetRole(t *testing.T, conn *postgres.DB, role string) { + t.Helper() + + _, err := conn.Exec("RESET ROLE") + assert.NoError(t, err) + + var currentUser string + if assert.NoError(t, conn.QueryRow("SELECT current_user::text").Scan(¤tUser)) { + assert.NotEqual(t, role, currentUser, "RESET ROLE must leave the test role") + } +} + +// Test_SetupTestRole_RejectsUnsafeName pins the role-name guard in postgres.SetupTestRole. A role +// name is an SQL identifier, so it cannot travel as a $1 placeholder and the helper interpolates it; +// the guard is what keeps "callers pass literal constants" an invariant instead of a doc comment, in +// a file that has no build tag and ships in the released binary. Validation runs before the +// connection is touched, so a nil *postgres.DB suffices - and without the guard these names would +// reach db.Exec rather than being refused. +// +// It lives in this file rather than in internal/postgres because this task may modify only three +// files (acceptance criterion 1), and the helper's only callers are here. +func Test_SetupTestRole_RejectsUnsafeName(t *testing.T) { + unsafe := map[string]string{ + "statement separator": "a; DROP ROLE victim", + "trailing newline": "role\n; DROP ROLE victim", + "quote and comment": "role'--", + "dollar sign": "pgcenter_test$x", + "upper case": "PgCenter_Test", + "leading digit": "1role", + "empty": "", + } + + for name, role := range unsafe { + t.Run(name, func(t *testing.T) { + err := postgres.SetupTestRole(nil, role, false) + assert.Error(t, err, "unsafe role name must be refused before any statement is built") + assert.Contains(t, fmt.Sprint(err), "invalid test role name") + }) + } +} + +// connectArchiverFixture connects to the fixture cluster of the given version. A version missing +// from the port map fails instead of skipping: a forgotten entry would otherwise make every subtest +// for a new version pass while exercising nothing at all (internal/postgres/testing_test.go). +func connectArchiverFixture(t *testing.T, version int) *postgres.DB { + t.Helper() + + conn, err := postgres.NewTestConnectVersion(version) + if err != nil { + assert.NotContains(t, err.Error(), "no test cluster port mapping", + "version %d is missing from the test port map", version) + t.Skipf("postgres %d not available in test environment", version) + } + + return conn +} + +// runArchiverQuery executes q and returns the result's column names, its row count and the first +// error the driver surfaced. A server error may be reported either by Query itself or only when the +// result is drained, so both are collected here - a negative privilege test that inspected only the +// Query error could otherwise miss the failure entirely. +func runArchiverQuery(conn *postgres.DB, q string) ([]string, int, error) { + rows, err := conn.Query(q) + if err != nil { + return nil, 0, err + } + + descs := rows.FieldDescriptions() + names := make([]string, len(descs)) + for i, d := range descs { + names[i] = string(d.Name) + } + + var nrows int + for rows.Next() { + nrows++ + } + rows.Close() + + return names, nrows, rows.Err() +} diff --git a/internal/query/overview.go b/internal/query/overview.go index f53455d0..730d23dd 100644 --- a/internal/query/overview.go +++ b/internal/query/overview.go @@ -90,14 +90,19 @@ const ( "FROM pg_stat_activity" // OverviewArchivingBacklog reports the WAL archiving backlog in bytes: - // count(*.ready in pg_wal/archive_status) * wal_segment_size. This adapts the wal.go precedent - // count(1) * pg_size_bytes(current_setting('wal_segment_size')), replacing pg_ls_waldir() with - // pg_ls_dir('pg_wal/archive_status') filtered on the .ready suffix. + // count(.ready) * wal_segment_size, over the archive_status directory. This adapts the wal.go + // precedent count(1) * pg_size_bytes(current_setting('wal_segment_size')), replacing + // pg_ls_waldir() with pg_ls_archive_statusdir() filtered on the .ready suffix. // - // pg_ls_dir requires pg_monitor/superuser; this query MUST be run as its OWN QueryRow so a 42501 - // privilege error (or archive_mode=off) degrades only the archiving-backlog field to n/a without - // aborting the sample. The raw error (containing the path) must never be surfaced. Single column. + // The source function is pg_ls_archive_statusdir(), executable by superuser AND pg_monitor. Its + // predecessor here, pg_ls_dir('pg_wal/archive_status'), is superuser-only, so this field degraded + // to n/a for exactly the monitoring role the panel serves (Decision 8 supersedes ADR [010]). + // + // This query MUST still be run as its OWN QueryRow, so any error degrades only the + // archiving-backlog field to n/a instead of aborting the whole overview sample. The function is + // missing_ok=true: a cluster with no archive_status directory yields 0 rather than an error. + // Single column. OverviewArchivingBacklog = "SELECT " + "count(*) FILTER (WHERE name LIKE '%.ready') * pg_size_bytes(current_setting('wal_segment_size')) AS backlog " + - "FROM pg_ls_dir('pg_wal/archive_status') AS name" + "FROM pg_ls_archive_statusdir()" ) diff --git a/internal/query/overview_test.go b/internal/query/overview_test.go index a74a1550..abe89ad3 100644 --- a/internal/query/overview_test.go +++ b/internal/query/overview_test.go @@ -2,9 +2,11 @@ package query import ( "database/sql" + "fmt" "strings" "testing" + "github.com/jackc/pgx/v5/pgconn" "github.com/lesovsky/pgcenter/internal/postgres" "github.com/stretchr/testify/assert" ) @@ -12,6 +14,14 @@ import ( // overviewVersions enumerates the actively supported Postgres versions for live-PG tests. var overviewVersions = []int{140000, 150000, 160000, 170000, 180000, 190000} +// Role names owned by the archiving-backlog tests. They are deliberately distinct from the archiver +// tests' roles: roles are cluster-global and never dropped, so sharing them would couple the cluster +// state of two independent tasks. +const ( + backlogRoleMonitor = "pgcenter_test_backlog_monitor" + backlogRoleNoRole = "pgcenter_test_backlog_norole" +) + func Test_OverviewQueries(t *testing.T) { // Static (non-template) aggregates: each must execute AND scan into exactly the receivers // collectOverviewStat uses, so a column-count/type drift fails here rather than at runtime. @@ -121,9 +131,10 @@ func Test_OverviewQueries_Templates_Recovery(t *testing.T) { } func Test_ArchivingBacklogQuery_Degrades(t *testing.T) { - // The archiving backlog aggregate reads pg_wal/archive_status via pg_ls_dir, which requires - // pg_monitor/superuser. On the test clusters the fixtures role has access, so the query must - // either execute successfully or fail with an error the caller can catch (privilege/archive_mode=off) + // The archiving backlog aggregate reads archive_status via pg_ls_archive_statusdir(), which + // superuser and pg_monitor may execute. The fixtures role is postgres, a superuser, so this test + // says nothing about privileges (Test_ArchivingBacklogQuery_PgMonitorRole does): it asserts only + // that the query either executes successfully or fails with an error the caller can catch, // WITHOUT panicking and WITHOUT being run as part of a larger scan. for _, version := range overviewVersions { conn, err := postgres.NewTestConnectVersion(version) @@ -145,6 +156,99 @@ func Test_ArchivingBacklogQuery_Degrades(t *testing.T) { } } +// Test_ArchivingBacklogQuery_Structure pins the aggregate's arithmetic without a server, mirroring +// Test_StatArchiverQuery_Structure. The fixtures run archive_mode=off with an empty status +// directory, so every live assertion on the backlog reduces to 0 >= 0: dropping the .ready FILTER or +// the wal_segment_size multiplication would keep all the live tests green. Only substring fixation +// reddens on those two mutations. +func Test_ArchivingBacklogQuery_Structure(t *testing.T) { + assert.Contains(t, OverviewArchivingBacklog, "count(*) FILTER (WHERE name LIKE '%.ready')", + "only .ready files are backlog - a bare count(*) is a different number") + assert.Contains(t, OverviewArchivingBacklog, "pg_size_bytes(current_setting('wal_segment_size'))", + "the backlog is bytes, not a segment count") + assert.Contains(t, OverviewArchivingBacklog, "FROM pg_ls_archive_statusdir()", + "the pg_monitor-executable function is the whole point of Decision 8") + assert.NotContains(t, OverviewArchivingBacklog, "pg_ls_dir", + "the superuser-only predecessor must not come back") +} + +// Test_ArchivingBacklogQuery_PgMonitorRole is the whole point of the aggregate's rewrite: a role +// holding only pg_monitor - the role the verbose panel exists to serve - must get a number, not n/a. +// pg_ls_dir is superuser-only, so the old query 42501'd for that role on every tick and the operator +// never saw the first signal that archiving had stopped; pg_ls_archive_statusdir() is granted to +// pg_monitor. +// +// The fixture connection is the superuser postgres, so the restricted-session guard runs BEFORE the +// aggregate: without it the test would pass identically as superuser and prove nothing about +// privileges - which is exactly how the wrong assumption survived into ADR [010]. +func Test_ArchivingBacklogQuery_PgMonitorRole(t *testing.T) { + for _, version := range overviewVersions { + t.Run(fmt.Sprintf("backlog/%d", version), func(t *testing.T) { + conn, err := postgres.NewTestConnectVersion(version) + if err != nil { + t.Skipf("postgres %d not available in test environment", version) + } + defer conn.Close() + + err = postgres.SetupTestRole(conn, backlogRoleMonitor, true) + assert.NoError(t, err) + if err != nil { + return + } + // RESET ROLE immediately after the successful SET ROLE, so it runs even when an + // assertion below fails. + defer resetRole(t, conn, backlogRoleMonitor) + + if !assertRestrictedSession(t, conn, backlogRoleMonitor, true) { + return + } + + var backlog int64 + err = conn.QueryRow(OverviewArchivingBacklog).Scan(&backlog) + assert.NoError(t, err, "pg_monitor must be able to read the archiving backlog") + assert.GreaterOrEqual(t, backlog, int64(0)) + }) + } +} + +// Test_ArchivingBacklogQuery_NoPrivilegeRole is the negative half: moving off pg_ls_dir widens who +// can read the backlog, and this pins how far. A role holding neither superuser nor pg_monitor must +// still be refused, so the change is a privilege fix and not a privilege downgrade. The assertion is +// pinned to SQLSTATE 42501 and to the function name - "an error occurred" would also pass on a typo. +func Test_ArchivingBacklogQuery_NoPrivilegeRole(t *testing.T) { + for _, version := range overviewVersions { + t.Run(fmt.Sprintf("backlog/%d", version), func(t *testing.T) { + conn, err := postgres.NewTestConnectVersion(version) + if err != nil { + t.Skipf("postgres %d not available in test environment", version) + } + defer conn.Close() + + err = postgres.SetupTestRole(conn, backlogRoleNoRole, false) + assert.NoError(t, err) + if err != nil { + return + } + defer resetRole(t, conn, backlogRoleNoRole) + + if !assertRestrictedSession(t, conn, backlogRoleNoRole, false) { + return + } + + var backlog int64 + err = conn.QueryRow(OverviewArchivingBacklog).Scan(&backlog) + assert.Error(t, err) + + var pgErr *pgconn.PgError + if assert.ErrorAs(t, err, &pgErr) { + assert.Equal(t, "42501", pgErr.Code, "must fail with insufficient_privilege") + assert.Contains(t, pgErr.Message, "pg_ls_archive_statusdir", + "the failure must name the privileged call, not just any error") + } + }) + } +} + func Test_OverviewBgwriterColumns(t *testing.T) { // bgwr/ckpt reuses SelectStatBgwriterQuery and collectOverviewBgwriter maps values by column // NAME (positions differ across PG14-16/17/18). Verify the five names it reads are actually diff --git a/internal/query/wal.go b/internal/query/wal.go index c17286d5..07c515d0 100644 --- a/internal/query/wal.go +++ b/internal/query/wal.go @@ -19,10 +19,27 @@ const ( "wal_buffers_full AS buffers_full, " + "date_trunc('seconds', now() - stats_reset)::text AS stats_age " + "FROM pg_stat_wal" + + // PgStatWALPG19 defines query for pg_stat_wal (PG 19+). + // wal_fpi_bytes added in PG 19: the volume of WAL written as full page images. It sits right after + // the wal_fpi count so the number of full page images and the bytes they cost are adjacent, and + // inside the diffed range so both render as per-interval deltas. + PgStatWALPG19 = "SELECT 'WAL' AS source, " + + "(SELECT pg_size_pretty(count(1) * pg_size_bytes(current_setting('wal_segment_size'))) AS waldir_size FROM pg_ls_waldir()) AS waldir_size, " + + `round(wal_bytes / 1024, 2) AS "wal,KiB", ` + + "wal_records AS records, wal_fpi AS fpi, " + + `round(wal_fpi_bytes / 1024, 2) AS "fpi,KiB", ` + + "wal_buffers_full AS buffers_full, " + + "date_trunc('seconds', now() - stats_reset)::text AS stats_age " + + "FROM pg_stat_wal" ) // SelectStatWALQuery returns the proper query, column count and diff interval for pg_stat_wal based on PG version. func SelectStatWALQuery(version int) (string, int, [2]int) { + if version >= PostgresV19 { + // PG 19 added wal_fpi_bytes; stats_age is col 7 and must not be diffed. + return PgStatWALPG19, 8, [2]int{2, 6} + } if version >= 180000 { // PG 18 removed wal_write/wal_sync columns; stats_age is col 6 and must not be diffed. return PgStatWALDefault, 7, [2]int{2, 5} diff --git a/internal/query/wal_test.go b/internal/query/wal_test.go index c2fedfe4..de3bb399 100644 --- a/internal/query/wal_test.go +++ b/internal/query/wal_test.go @@ -2,9 +2,12 @@ package query import ( "fmt" + "strings" + "testing" + "github.com/lesovsky/pgcenter/internal/postgres" "github.com/stretchr/testify/assert" - "testing" + "github.com/stretchr/testify/require" ) func Test_SelectStatWALQuery(t *testing.T) { @@ -18,7 +21,10 @@ func Test_SelectStatWALQuery(t *testing.T) { {version: 170000, wantNcols: 11, wantDiffIntvl: [2]int{2, 9}}, // PG 18: removed wal_write/wal_sync; stats_age must be outside the diff interval. {version: 180000, wantNcols: 7, wantDiffIntvl: [2]int{2, 5}}, - {version: 190000, wantNcols: 7, wantDiffIntvl: [2]int{2, 5}}, + // PG 19: added wal_fpi_bytes as "fpi,KiB"; stats_age shifts to col 7 and stays outside. + {version: 190000, wantNcols: 8, wantDiffIntvl: [2]int{2, 6}}, + // Forward version: the PG 19 branch must fire for every future major, not only 190000. + {version: 200000, wantNcols: 8, wantDiffIntvl: [2]int{2, 6}}, } for _, tc := range testcases { @@ -30,27 +36,102 @@ func Test_SelectStatWALQuery(t *testing.T) { } } +// Test_SelectStatWALQuery_PG19ColumnOrder pins the position of the "fpi,KiB" column in the PG 19 +// select list: it must sit right after the wal_fpi counter (count and volume adjacent) and before +// wal_buffers_full, so it lands inside DiffIntvl {2,6} and renders as a per-interval delta. It also +// pins stats_age as the last selected column — a text date_trunc value pulled inside the interval +// aborts the whole sample at strconv.ParseInt. +func Test_SelectStatWALQuery_PG19ColumnOrder(t *testing.T) { + q, _, _ := SelectStatWALQuery(PostgresV19) + + idxFpi := strings.Index(q, "wal_fpi AS fpi") + idxFpiKiB := strings.Index(q, `AS "fpi,KiB"`) + idxBuffersFull := strings.Index(q, "wal_buffers_full") + + assert.NotEqual(t, -1, idxFpi, "PG 19 query must keep the wal_fpi counter") + assert.NotEqual(t, -1, idxFpiKiB, `PG 19 query must select the "fpi,KiB" column`) + assert.NotEqual(t, -1, idxBuffersFull, "PG 19 query must keep wal_buffers_full") + + assert.Less(t, idxFpi, idxFpiKiB, `"fpi,KiB" must follow the wal_fpi counter`) + assert.Less(t, idxFpiKiB, idxBuffersFull, `"fpi,KiB" must precede wal_buffers_full`) + + assert.True(t, strings.HasSuffix(q, "AS stats_age FROM pg_stat_wal"), + "stats_age must remain the last selected column, outside the diff interval") + + // The PG 19 query is the PG 18 query plus exactly one expression. Deriving it here pins what the + // name/position checks above cannot see: the body of the new expression (notably the /1024 that + // makes the "fpi,KiB" header truthful) and the fact that none of the other seven columns drifted. + assert.Equal(t, PgStatWALPG19, strings.Replace(PgStatWALDefault, + "wal_fpi AS fpi, wal_buffers_full", + `wal_fpi AS fpi, round(wal_fpi_bytes / 1024, 2) AS "fpi,KiB", wal_buffers_full`, 1), + "PG 19 query must be the PG 18 query plus exactly the fpi,KiB expression") +} + +// Test_SelectStatWALQuery_LegacyBranchesUntouched proves the PG 19 work did not leak into the older +// branches: the selector still returns exactly the pre-existing constants below PG 19, and neither of +// them mentions wal_fpi_bytes (a column that does not exist before PG 19). +func Test_SelectStatWALQuery_LegacyBranchesUntouched(t *testing.T) { + for _, version := range []int{140000, 150000, 170000, 179999} { + q, _, _ := SelectStatWALQuery(version) + assert.Equal(t, PgStatWALPG14, q, "version %d must return PgStatWALPG14", version) + } + + for _, version := range []int{180000, 189999} { + q, _, _ := SelectStatWALQuery(version) + assert.Equal(t, PgStatWALDefault, q, "version %d must return PgStatWALDefault", version) + } + + assert.NotContains(t, PgStatWALPG14, "wal_fpi_bytes", "wal_fpi_bytes does not exist before PG 19") + assert.NotContains(t, PgStatWALDefault, "wal_fpi_bytes", "wal_fpi_bytes does not exist before PG 19") +} + // Test_StatWALQueries tests query execution against all supported Postgres versions. func Test_StatWALQueries(t *testing.T) { versions := []int{140000, 150000, 160000, 170000, 180000, 190000} for _, version := range versions { t.Run(fmt.Sprintf("pg_stat_wal/%d", version), func(t *testing.T) { - tmpl, _, _ := SelectStatWALQuery(version) + tmpl, wantNcols, diffIntvl := SelectStatWALQuery(version) opts := NewOptions(version, "f", "off", 256, "public") q, err := Format(tmpl, opts) - assert.NoError(t, err) + require.NoError(t, err) conn, err := postgres.NewTestConnectVersion(version) if err != nil { t.Skipf("postgres %d not available in test environment", version) } + defer conn.Close() - _, err = conn.Exec(q) - assert.NoError(t, err) + // Fatal, not just failed: if wal_fpi_bytes is renamed at a later PG 19 beta/RC, the + // undefined-column error is the signal and must not be buried under derived failures. + rows, err := conn.Query(q) + require.NoError(t, err) - conn.Close() + var names []string + for _, fd := range rows.FieldDescriptions() { + names = append(names, string(fd.Name)) + } + rows.Close() + assert.NoError(t, rows.Err()) + + // The live result must have exactly as many columns as the selector declared — the view + // is configured from Ncols, so a mismatch misaligns the whole screen. + assert.Len(t, names, wantNcols) + + // stats_age must stay outside the diff interval on every version. + require.Greater(t, len(names), diffIntvl[1]+1, + "DiffIntvl upper bound must leave at least one column (stats_age) outside") + assert.Equal(t, "stats_age", names[diffIntvl[1]+1], + "the column after the diff interval must be stats_age") + + if version >= PostgresV19 { + assert.Equal(t, + []string{"source", "waldir_size", "wal,KiB", "records", "fpi", "fpi,KiB", "buffers_full", "stats_age"}, + names) + assert.Equal(t, "buffers_full", names[diffIntvl[1]], + "the last diffed column must be buffers_full") + } }) } } diff --git a/internal/stat/postgres.go b/internal/stat/postgres.go index 69722c8f..3bb11e7f 100644 --- a/internal/stat/postgres.go +++ b/internal/stat/postgres.go @@ -81,7 +81,7 @@ type PgstatOverview struct { RetainedValid bool // false when there are no slots (n/a) ArchivingBacklog int64 // count(.ready) * wal_segment_size, bytes - ArchivingBacklogValid bool // false on archive_mode=off / missing privilege (42501) -> n/a + ArchivingBacklogValid bool // false when the aggregate failed (e.g. missing privilege, 42501) -> n/a Senders int // active walsenders Receivers int // active walreceivers @@ -285,9 +285,10 @@ func collectOverviewStat(db *postgres.DB, props PostgresProperties, itv int, pre } } - // replication: archiving backlog. OWN QueryRow: pg_ls_dir requires pg_monitor/superuser; a 42501 - // privilege error or archive_mode=off degrades this field to n/a. The raw error (which contains a - // filesystem path) is deliberately swallowed and never logged or surfaced. + // replication: archiving backlog. OWN QueryRow so a failure degrades this field alone to n/a + // instead of aborting the sample; the aggregate needs superuser or pg_monitor + // (pg_ls_archive_statusdir). The error is deliberately swallowed and never logged or surfaced. + // archive_mode=off is not an error: nothing is queued, so the field is a real 0, not n/a. var backlog sql.NullInt64 if err := db.QueryRow(query.OverviewArchivingBacklog).Scan(&backlog); err == nil && backlog.Valid { s.ArchivingBacklog = backlog.Int64 diff --git a/internal/stat/postgres_test.go b/internal/stat/postgres_test.go index f564fb20..3a4fcaf8 100644 --- a/internal/stat/postgres_test.go +++ b/internal/stat/postgres_test.go @@ -221,7 +221,9 @@ func Test_collectOverviewStat_Degradation(t *testing.T) { assert.False(t, got.RetainedValid, "no slots -> retained WAL is n/a") assert.Equal(t, int64(0), got.SlotsCount) - // Archiving backlog: the fixtures role has pg_monitor, so the OWN-QueryRow aggregate must run. + // Archiving backlog: the fixtures role is postgres, a superuser, so the OWN-QueryRow aggregate + // runs here regardless of the privilege question (Test_collectOverviewStat_PgMonitorRole covers + // the pg_monitor-only role). // On archive_mode=off it is a real 0 with ArchivingBacklogValid=true; either way it must be a // non-negative value distinguishable from n/a, and its outcome must NOT have blanked other rows. if got.ArchivingBacklogValid { @@ -235,6 +237,85 @@ func Test_collectOverviewStat_Degradation(t *testing.T) { assert.GreaterOrEqual(t, got.TotalSize, int64(0)) } +// backlogRoleCollect is this package's own pg_monitor-only role. It deliberately does NOT share a +// name with the query package's role: SetupTestRole's DO block is not atomic, so two packages +// creating the same role against the same cluster would race on pg_authid outside `go test -p 1`. +const backlogRoleCollect = "pgcenter_test_backlog_collect" + +// Test_collectOverviewStat_PgMonitorRole is the end-to-end proof that the verbose panel now shows a +// number for the role it was written for. Under a role holding only pg_monitor the backlog aggregate +// used to fail with 42501 (pg_ls_dir is superuser-only) and the field degraded to n/a; with +// pg_ls_archive_statusdir() it returns a value, and the rest of the sample stays populated. +// +// The restricted-session guard runs before collect: the fixtures role is the superuser postgres, so +// a test that forgot to switch roles would pass here while proving nothing. +func Test_collectOverviewStat_PgMonitorRole(t *testing.T) { + conn, err := postgres.NewTestConnect() + assert.NoError(t, err) + defer conn.Close() + + // Read properties as the fixture superuser: the subject under test is the backlog aggregate, not + // the privileges of the properties probe. + props, err := GetPostgresProperties(conn) + assert.NoError(t, err) + + // Superuser baseline, taken before the role switch. It turns "the rest of the sample stays + // populated" into a falsifiable comparison: a privilege regression in any neighbouring aggregate + // now shows up as a divergence from what the same cluster reports unrestricted. + base, _ := collectOverviewStat(conn, props, 1, PgstatOverview{}, false) + + err = postgres.SetupTestRole(conn, backlogRoleCollect, true) + assert.NoError(t, err) + if err != nil { + return + } + defer func() { + _, err := conn.Exec("RESET ROLE") + assert.NoError(t, err) + + var user string + if assert.NoError(t, conn.QueryRow("SELECT current_user::text").Scan(&user)) { + assert.NotEqual(t, backlogRoleCollect, user, "RESET ROLE must leave the test role") + } + }() + + var ( + currentUser string + isSuper bool + hasMonitor bool + memberOf []string + ) + err = conn.QueryRow( + "SELECT current_user::text, "+ + "(SELECT rolsuper FROM pg_roles WHERE rolname = current_user), "+ + "pg_has_role(current_user, 'pg_monitor', 'USAGE'), "+ + // DISTINCT because PG 16+ stores one pg_auth_members row per grantor. + "coalesce((SELECT array_agg(DISTINCT r.rolname::text ORDER BY r.rolname::text) "+ + "FROM pg_auth_members m JOIN pg_roles r ON r.oid = m.roleid "+ + "WHERE m.member = (SELECT oid FROM pg_roles WHERE rolname = current_user)), ARRAY[]::text[])", + ).Scan(¤tUser, &isSuper, &hasMonitor, &memberOf) + assert.NoError(t, err) + okUser := assert.Equal(t, backlogRoleCollect, currentUser, "collect must run as the test role") + okSuper := assert.False(t, isSuper, "the test role must not be a superuser") + okMonitor := assert.True(t, hasMonitor, "the test role must hold pg_monitor") + // Roles are cluster-global and SetupTestRole never normalises an existing one, so without this + // the test could silently decay from "pg_monitor is sufficient" to "some privileged role works". + okMembers := assert.Equal(t, []string{"pg_monitor"}, memberOf, + "the test role must hold pg_monitor and nothing else") + if !okUser || !okSuper || !okMonitor || !okMembers { + return + } + + got, _ := collectOverviewStat(conn, props, 1, PgstatOverview{}, false) + assert.True(t, got.ArchivingBacklogValid, "pg_monitor must get a backlog number, not n/a") + assert.GreaterOrEqual(t, got.ArchivingBacklog, int64(0)) + + // The rest of the sample must be unaffected by running under a restricted role. + assert.Equal(t, base.TotalSizeValid, got.TotalSizeValid, "the db-size aggregate must not degrade under pg_monitor") + assert.Equal(t, base.DatabasesCount, got.DatabasesCount, "the databases aggregate must see the same databases under pg_monitor") + assert.GreaterOrEqual(t, got.DatabasesCount, int64(1)) +} + func TestGetPostgresProperties(t *testing.T) { conn, err := postgres.NewTestConnect() assert.NoError(t, err) diff --git a/internal/view/view.go b/internal/view/view.go index 53ce5302..fedb0b9d 100644 --- a/internal/view/view.go +++ b/internal/view/view.go @@ -138,6 +138,18 @@ func New() Views { Msg: "Show WAL statistics", Filters: map[int]*regexp.Regexp{}, }, + "archiver": { + Name: "archiver", + MinRequiredVersion: query.PostgresV14, + QueryTmpl: query.PgStatArchiverDefault, + DiffIntvl: [2]int{0, 0}, + Ncols: 9, + OrderKey: 0, + OrderDesc: true, + ColsWidth: map[int]int{}, + Msg: "Show archiver statistics (requires archive_mode=on)", + Filters: map[int]*regexp.Regexp{}, + }, "bgwriter": { Name: "bgwriter", MinRequiredVersion: query.PostgresV14, @@ -389,6 +401,13 @@ func (v Views) Configure(opts query.Options) error { case "wal": view.QueryTmpl, view.Ncols, view.DiffIntvl = query.SelectStatWALQuery(opts.Version) v[k] = view + case "archiver": + // Version-independent today: the selector returns the same three values the static + // entry carries, so this case is functionally a no-op. It is kept for symmetry with + // stat_io_time (also version-independent) and so a future version branch stays + // confined to query/archiver.go on the production side. + view.QueryTmpl, view.Ncols, view.DiffIntvl = query.SelectStatArchiverQuery(opts.Version) + v[k] = view case "bgwriter": view.QueryTmpl, view.Ncols, view.DiffIntvl = query.SelectStatBgwriterQuery(opts.Version) v[k] = view diff --git a/internal/view/view_test.go b/internal/view/view_test.go index 052352ec..b09b3251 100644 --- a/internal/view/view_test.go +++ b/internal/view/view_test.go @@ -8,7 +8,7 @@ import ( func TestNew(t *testing.T) { v := New() - assert.Equal(t, 27, len(v)) // 27 is the total number of views have to be returned + assert.Equal(t, 28, len(v)) // 28 is the total number of views have to be returned } // TestNew_StatementsJITView guards the statements_jit view wiring: it must be registered, @@ -96,6 +96,38 @@ func TestNew_BgwriterView(t *testing.T) { assert.Equal(t, "Show bgwriter / checkpointer statistics", bgwriter.Msg) } +// TestNew_ArchiverView guards the archiver view wiring: it must be registered, +// gated to PG14+, recordable, undiffed, and keyed by the constant 'Archiver' literal +// at column 0. +func TestNew_ArchiverView(t *testing.T) { + v := New() + archiver, ok := v["archiver"] + assert.True(t, ok) + // Name must equal the map key: top/config_view.go writes the working view back with + // views[view.Name], so a divergence silently parks column widths and filters in a + // phantom entry. It is also the report type string and the tar entry prefix. + assert.Equal(t, "archiver", archiver.Name) + assert.False(t, archiver.NotRecordable) + assert.Equal(t, query.PostgresV14, archiver.MinRequiredVersion) + // The static seed, before Configure() reassigns it from the selector. + assert.Equal(t, query.PgStatArchiverDefault, archiver.QueryTmpl) + assert.Equal(t, 9, archiver.Ncols) + // DiffIntvl{0,0} is deliberate: nothing on this screen is diffed, which is what + // keeps the NULL-able columns away from diff()/strconv.ParseInt. + assert.Equal(t, [2]int{0, 0}, archiver.DiffIntvl) + assert.Equal(t, 0, archiver.OrderKey) + assert.True(t, archiver.OrderDesc) + assert.Equal(t, 0, archiver.UniqueKey) + // Both maps must be non-nil: top/config_view.go:100 and :124 write into ColsWidth and :166 + // writes into Filters in place, so a nil map is a runtime panic inside a gocui key handler + // that no count test would catch. + assert.NotNil(t, archiver.ColsWidth) + assert.NotNil(t, archiver.Filters) + // Msg is load-bearing and pinned verbatim (Decision 5): it is the cmdline text on every + // switch to the screen, and the only place the archive_mode requirement is stated. + assert.Equal(t, "Show archiver statistics (requires archive_mode=on)", archiver.Msg) +} + func TestViews_Configure(t *testing.T) { testcases := []struct { version int @@ -190,6 +222,19 @@ func TestViews_Configure(t *testing.T) { assert.Equal(t, query.PgStatProgressBasebackupPG19, views["progress_basebackup"].QueryTmpl) assert.Equal(t, 12, views["progress_basebackup"].Ncols) assert.Equal(t, [2]int{10, 10}, views["progress_basebackup"].DiffIntvl) + // PG 19 added wal_fpi_bytes; this pins that Configure carries the new wal layout + // into the registered view, not just that the selector returns it. + assert.Equal(t, query.PgStatWALPG19, views["wal"].QueryTmpl) + assert.Equal(t, 8, views["wal"].Ncols) + assert.Equal(t, [2]int{2, 6}, views["wal"].DiffIntvl) + // pg_stat_archiver is schema-stable, so the archiver layout is the same on every version. + // Regression guard only: New() already sets these values, so deleting case "archiver": + // from Configure() cannot redden this. What it does catch is drift between + // SelectStatArchiverQuery and the static entry. The wal assertions above are the + // real wiring gate. + assert.Equal(t, query.PgStatArchiverDefault, views["archiver"].QueryTmpl) + assert.Equal(t, 9, views["archiver"].Ncols) + assert.Equal(t, [2]int{0, 0}, views["archiver"].DiffIntvl) case 140000: // Everything below PG 19 keeps today's progress layouts, byte for byte. assert.Equal(t, query.PgStatProgressVacuumDefault, views["progress_vacuum"].QueryTmpl) @@ -200,6 +245,15 @@ func TestViews_Configure(t *testing.T) { assert.Equal(t, query.PgStatProgressBasebackupDefault, views["progress_basebackup"].QueryTmpl) assert.Equal(t, 11, views["progress_basebackup"].Ncols) assert.Equal(t, [2]int{9, 9}, views["progress_basebackup"].DiffIntvl) + // The PG 14-17 wal layout must not move when the PG 19 branch is added. + assert.Equal(t, query.PgStatWALPG14, views["wal"].QueryTmpl) + assert.Equal(t, 11, views["wal"].Ncols) + assert.Equal(t, [2]int{2, 9}, views["wal"].DiffIntvl) + // pg_stat_archiver is schema-stable, so the archiver layout is the same on every version. + // Regression guard only — see the note in the 190000 arm above. + assert.Equal(t, query.PgStatArchiverDefault, views["archiver"].QueryTmpl) + assert.Equal(t, 9, views["archiver"].Ncols) + assert.Equal(t, [2]int{0, 0}, views["archiver"].DiffIntvl) case 130000: if tc.trackCommit == "on" { assert.Equal(t, query.PgStatReplicationExtended, views["replication"].QueryTmpl) @@ -257,9 +311,9 @@ func TestView_VersionOK(t *testing.T) { version int total int }{ - {version: 190000, total: 27}, - {version: 160000, total: 27}, - {version: 140000, total: 24}, + {version: 190000, total: 28}, + {version: 160000, total: 28}, + {version: 140000, total: 25}, {version: 130000, total: 19}, {version: 120000, total: 16}, {version: 110000, total: 14}, @@ -275,6 +329,6 @@ func TestView_VersionOK(t *testing.T) { total++ } } - assert.Equal(t, tc.total, total) + assert.Equal(t, tc.total, total, "version=%d", tc.version) } } diff --git a/record/record_test.go b/record/record_test.go index 44412056..f661109d 100644 --- a/record/record_test.go +++ b/record/record_test.go @@ -108,10 +108,11 @@ func Test_app_record(t *testing.T) { func Test_filterViews(t *testing.T) { testcases := []struct { - version int - pgssSchema string - wantN int - wantV int + version int + pgssSchema string + wantN int + wantV int + wantArchiver bool }{ // wantN counts filtered views (version-incompatible + statements_* without pgss); // wantV counts remaining views after filtering. After feature 008 no production @@ -129,22 +130,32 @@ func Test_filterViews(t *testing.T) { // On PG13 and below all five views are version-incompatible and dropped by the // version gate regardless of NotRecordable, so those rows are unchanged from the // pre-008 baseline. + // The archiver view (feature 017, MinRequiredVersion=PostgresV14, recordable) is the + // 28th registered view: it passes both gates on PG14+ and joins wantV there, and is + // dropped by the version gate on PG13 and below, where it joins wantN. That is the + // only difference between these numbers and the feature 008 baseline. + // wantArchiver asserts that survival by name, not by arithmetic: the counts alone + // would also be satisfied by dropping archiver while some other view's gate loosens. // On PG 19 nothing is filtered: the highest MinRequiredVersion in the registry is // PostgresV16, and with a pgss schema supplied no statements_* view is dropped either — - // so all 27 registered views survive. Copying the PG14 row here would be wrong. - {version: 190000, pgssSchema: "public", wantN: 0, wantV: 27}, - {version: 140000, pgssSchema: "", wantN: 9, wantV: 18}, - {version: 140000, pgssSchema: "public", wantN: 3, wantV: 24}, - {version: 130000, pgssSchema: "public", wantN: 8, wantV: 19}, - {version: 120000, pgssSchema: "public", wantN: 11, wantV: 16}, - {version: 110000, pgssSchema: "public", wantN: 13, wantV: 14}, - {version: 100000, pgssSchema: "public", wantN: 13, wantV: 14}, + // so all 28 registered views survive. Copying the PG14 row here would be wrong. + {version: 190000, pgssSchema: "public", wantN: 0, wantV: 28, wantArchiver: true}, + {version: 140000, pgssSchema: "", wantN: 9, wantV: 19, wantArchiver: true}, + {version: 140000, pgssSchema: "public", wantN: 3, wantV: 25, wantArchiver: true}, + {version: 130000, pgssSchema: "public", wantN: 9, wantV: 19, wantArchiver: false}, + {version: 120000, pgssSchema: "public", wantN: 12, wantV: 16, wantArchiver: false}, + {version: 110000, pgssSchema: "public", wantN: 14, wantV: 14, wantArchiver: false}, + {version: 100000, pgssSchema: "public", wantN: 14, wantV: 14, wantArchiver: false}, } for _, tc := range testcases { n, v := filterViews(tc.version, tc.pgssSchema, view.New()) - assert.Equal(t, tc.wantN, n) - assert.Equal(t, tc.wantV, len(v)) + assert.Equal(t, tc.wantN, n, "version=%d pgss=%q", tc.version, tc.pgssSchema) + assert.Equal(t, tc.wantV, len(v), "version=%d pgss=%q", tc.version, tc.pgssSchema) + // Membership by lookup rather than assert.Contains: a failing Contains on a view map + // dumps all 28 View structs, which buries the one bit of information needed here. + _, gotArchiver := v["archiver"] + assert.Equal(t, tc.wantArchiver, gotArchiver, "archiver kept? version=%d pgss=%q", tc.version, tc.pgssSchema) } } diff --git a/report/describe.go b/report/describe.go index ac0bacfa..15be67c4 100644 --- a/report/describe.go +++ b/report/describe.go @@ -146,6 +146,7 @@ Details: https://www.postgresql.org/docs/current/monitoring-stats.html#PG-STAT-U - wal,KiB wal_bytes Amount of WAL generated, in KiB - records wal_records Number of WAL records generated - fpi wal_fpi Number of WAL full page images generated +- fpi,KiB wal_fpi_bytes Amount of WAL generated by full page images, in KiB (PG 19+) - write wal_write Number of times WAL buffers were written out to disk via XLogWrite request - sync wal_sync Number of times WAL files were synced to disk via issue_xlog_fsync request - write,ms wal_write_time Amount of time spent writing WAL buffers to disk via XLogWrite request, in milliseconds @@ -154,6 +155,23 @@ Details: https://www.postgresql.org/docs/current/monitoring-stats.html#PG-STAT-U - stats_age stats_reset Age of collected statistics in the moment when stats are taken Details: https://www.postgresql.org/docs/current/monitoring-stats.html#PG-STAT-WAL-VIEW +` + + // pgStatArchiverDescription is the detailed description of pg_stat_archiver view + pgStatArchiverDescription = `WAL archiver statistics based on pg_stat_archiver view: + + column origin description +- source - Always has 'Archiver' value +- ready pg_ls_archive_statusdir Number of WAL segments waiting to be archived (*.ready files) +- archived archived_count Total number of WAL files successfully archived +- last_archived last_archived_wal Name of the last WAL file successfully archived, empty if nothing has been archived yet +- archived_age last_archived_time Time elapsed since the last successful archiving, empty if nothing has been archived yet +- failed failed_count Total number of failed attempts to archive a WAL file +- last_failed last_failed_wal Name of the WAL file of the last failed archiving attempt, empty if there were no failures +- failed_age last_failed_time Time elapsed since the last failed archiving attempt, empty if there were no failures +- stats_age stats_reset Age of collected statistics in the moment when stats are taken + +Details: https://www.postgresql.org/docs/current/monitoring-stats.html#PG-STAT-ARCHIVER-VIEW ` // pgStatSizesDescription is the detailed description of stats about tables sizes diff --git a/report/report.go b/report/report.go index 8ac3d171..daefef8a 100644 --- a/report/report.go +++ b/report/report.go @@ -672,6 +672,7 @@ func describeReport(w io.Writer, report string) error { "indexes": pgStatIndexesDescription, "functions": pgStatFunctionsDescription, "wal": pgStatWALDescription, + "archiver": pgStatArchiverDescription, "sizes": pgStatSizesDescription, "progress_vacuum": pgStatProgressVacuumDescription, "progress_cluster": pgStatProgressClusterDescription, diff --git a/report/report_record_archiver_test.go b/report/report_record_archiver_test.go new file mode 100644 index 00000000..bf29fd8a --- /dev/null +++ b/report/report_record_archiver_test.go @@ -0,0 +1,288 @@ +package report + +import ( + "archive/tar" + "bytes" + "database/sql" + "encoding/json" + "os" + "regexp" + "strings" + "testing" + "time" + + "github.com/lesovsky/pgcenter/internal/stat" + "github.com/stretchr/testify/assert" +) + +// archiverCols is the canonical 9-column layout of the archiver report, matching +// internal/query/archiver.go (SelectStatArchiverQuery returns Ncols=9, +// DiffIntvl [0,0], version-independent). +var archiverCols = []string{ + "source", "ready", "archived", "last_archived", "archived_age", + "failed", "last_failed", "failed_age", "stats_age", +} + +// Test_app_doReport_Archiver exercises the full doReport pipeline for the +// archiver report against a synthetic in-memory tar: a recording of two +// cumulative ticks plus a meta record whose version_num drives report-time +// view.Configure. The first tick is discarded by processData's first-snapshot +// rule (!prevStat.Valid -> continue), so two ticks produce exactly one data row. +// +// What this replay pins, and what it deliberately does not: +// +// - It pins the rendering and diff pipeline for this screen — above all the +// PASS-THROUGH property. SelectStatArchiverQuery returns DiffIntvl{0,0}, +// which short-circuits calculateDelta before diff() is ever entered +// (internal/stat/postgres.go:589-597), so every column is copied from the +// current tick verbatim. The assertions therefore pin ABSOLUTE current +// values, not deltas: a future change that gives this screen a diffed range +// turns the golden and the value sentinels red. +// - It pins that the four NULL-able columns render as blank cells rather than +// a "0" or a placeholder token (the never_archived subcase). +// - It does NOT pin the SQL. Report replays recorded stat.PGresult JSON, and +// the column names and their order come from the fixture this test writes, +// never from the query text. Reordering the aliases inside +// internal/query/archiver.go would not redden anything here; that layout is +// pinned by the query and view unit tests (internal/query, internal/view). +// - For the same reason it does not pin the view REGISTRY either: processData +// re-configures through a one-entry map it builds itself +// (report/report.go:282-284) and Views.Configure switches on the map key, so +// an unregistered "archiver" would still receive Ncols and DiffIntvl from +// the selector. What this test depends on from internal/view is the +// `case "archiver":` inside Configure, not the New() entry. +// +// The screen is version-independent (SelectStatArchiverQuery ignores its +// parameter), so a single golden with no version suffix suffices — the +// report_record_stat_io_time.golden convention. The two ticks are exactly one +// second apart so the rate divisor itv == 1; with nothing diffed that spacing +// only fixes the printed "rate:" value in the golden. +func Test_app_doReport_Archiver(t *testing.T) { + t.Run("populated", func(t *testing.T) { + // A cluster that has archived and has failures. Values are chosen so a + // pass-through row and a diffed row cannot be confused: were the screen + // diffed, archived would render 3 instead of 100003 and ready 4 instead + // of 14. A NotContains delta sentinel is deliberately NOT used: the + // would-be delta "3" is a substring of "100003", so it would be vacuous. + prev := archiverRow( + "Archiver", "10", "100000", "000000010000000000000021", "00:00:41", + "5", "000000010000000000000019", "00:12:02", "01:00:00", + ) + curr := archiverRow( + "Archiver", "14", "100003", "000000010000000000000024", "00:00:07", + "8", "000000010000000000000019", "00:14:02", "02:00:00", + ) + + out := runArchiverReplay(t, archiverCols, [][]sql.NullString{prev, curr}, true) + + assert.NotEmpty(t, out) + // Timestamp header line emitted by printStatSample matches "YYYY/MM/DD". + assert.Regexp(t, regexp.MustCompile(`\d{4}/\d{2}/\d{2}`), out) + // Row sentinel: localizes a failure to "row missing" rather than + // "golden differs". + assert.Contains(t, out, "Archiver") + // Pass-through sentinels: the CURRENT absolute values, not deltas. + assert.Contains(t, out, "100003") + assert.Contains(t, out, "000000010000000000000024") + + const wantFile = "testdata/report_record_archiver.golden" + if *update { + assert.NoError(t, os.WriteFile(wantFile, []byte(out), 0644)) + return + } + want, err := os.ReadFile(wantFile) + assert.NoError(t, err) + assert.Equal(t, string(want), out) + }) + + t.Run("never_archived", func(t *testing.T) { + // A cluster with archive_mode=on that has never archived anything: the + // four NULL-able columns arrive as SQL NULLs in both ticks. They must be + // sql.NullString{Valid: false} — not {String: "", Valid: true} — because + // that is what the recorder writes for a SQL NULL, and the difference is + // exactly what this subcase claims to be about. + // + // This is its own recording rather than a second row of the populated + // one on purpose: alignment is computed once from the first printed + // sample (formatStatSample returns early when view.Aligned), so a + // NULL-first recording would size the columns from blank cells and then + // truncate the 24-character WAL names of any later row. + null := sql.NullString{Valid: false} + mk := func(statsAge string) []sql.NullString { + return []sql.NullString{ + {String: "Archiver", Valid: true}, + {String: "0", Valid: true}, + {String: "0", Valid: true}, + null, + null, + {String: "0", Valid: true}, + null, + null, + {String: statsAge, Valid: true}, + } + } + + out := runArchiverReplay(t, archiverCols, [][]sql.NullString{mk("01:00:00"), mk("02:00:00")}, true) + + assert.NotEmpty(t, out) + + stripped := statIOStripANSI(out) + + // The header still names all nine columns — the blank cells do not + // collapse the layout. + for _, name := range archiverCols { + assert.Contains(t, stripped, name) + } + + // The load-bearing assertion, and the machine reading of the user-spec's + // "колонки пустые (не 0 и не прочерк)": the single data line splits into + // exactly five whitespace-separated fields — Archiver, 0, 0, 0, + // 02:00:00. If the four NULL cells rendered any token at all the count + // would be nine. + // + // No NotContains "n/a" assertion here: naLiteral (top/stat.go) is a + // TUI-only sentinel from the top package and can never appear in report + // output, so such an assertion would be vacuous by construction. + line := archiverDataLine(t, stripped) + assert.Equal(t, 5, len(strings.Fields(line)), "data line %q", line) + }) + + t.Run("no_archiver_entries", func(t *testing.T) { + // A recording that carries meta.* and sysinfo.* but no archiver.* entry + // at all — e.g. a recording made before the screen existed. Nothing + // matches the report type, so nothing is ever aligned and + // printStatHeader returns early on !v.Aligned: the buffer stays + // LITERALLY empty. The three INFO: lines live in printReportHeader, + // which the CLI path calls outside doReport, so they are not written + // here either. + // + // The claim is about the buffer, not the error return: doReport returns + // nil on every path (processData's error is printed with fmt.Println and + // swallowed), so the return value carries no information — see the + // comment on runArchiverReplay's assert.NoError. + out := runArchiverReplay(t, archiverCols, [][]sql.NullString{ + archiverRow("Archiver", "10", "100000", "000000010000000000000021", "00:00:41", + "5", "000000010000000000000019", "00:12:02", "01:00:00"), + archiverRow("Archiver", "14", "100003", "000000010000000000000024", "00:00:07", + "8", "000000010000000000000019", "00:14:02", "02:00:00"), + }, false) + + assert.Empty(t, out) + }) +} + +// archiverRow converts a tick's values into a row of non-NULL sql.NullString. +func archiverRow(vals ...string) []sql.NullString { + row := make([]sql.NullString, len(vals)) + for i, v := range vals { + row[i] = sql.NullString{String: v, Valid: true} + } + return row +} + +// archiverDataLine returns the data row of an ANSI-stripped report output: the +// line following the ", rate: " line printStatSample emits on its +// own line before the first row of a snapshot. +func archiverDataLine(t *testing.T, stripped string) string { + t.Helper() + + lines := strings.Split(stripped, "\n") + for i, l := range lines { + if strings.Contains(l, ", rate: ") && i+1 < len(lines) { + return lines[i+1] + } + } + t.Fatalf("no timestamp line found in output:\n%s", stripped) + return "" +} + +// runArchiverReplay builds a synthetic two-tick tar from the given rows, runs +// the full doReport pipeline over it and returns the produced output. +// +// writeStat toggles whether the archiver.* entries are written at all: with +// false the tar carries only meta.* and sysinfo.*, which is the no_archiver_entries +// fixture. That toggle is what makes the emptiness assertion meaningful — an +// empty buffer is also what a BROKEN fixture produces (TsStart/TsEnd not +// bracketing the filename dates, a three-part entry name failing isFilenameOK, a +// mistyped ReportType), so the same tar with the entries added back must produce +// output. +func runArchiverReplay(t *testing.T, cols []string, rows [][]sql.NullString, writeStat bool) string { + t.Helper() + + ncols := len(cols) + + // Meta result mirrors SelectCommonProperties (7-column shape; readMeta only + // consumes column index 1 for version_num, which drives the version-aware + // view.Configure at report time). 17 is used to make it plain the archiver + // screen does not branch on the version. + metaRes := stat.PGresult{ + Valid: true, Ncols: 7, Nrows: 1, + Cols: []string{"version", "version_num", "track_commit_timestamp", "max_connections", "autovacuum_max_workers", "recovery", "start_time_unix"}, + Values: [][]sql.NullString{ + { + {String: "17.1", Valid: true}, {String: "170000", Valid: true}, + {String: "off", Valid: true}, {String: "100", Valid: true}, {String: "3", Valid: true}, + {String: "false", Valid: true}, {String: "1622828486655396e-6", Valid: true}, + }, + }, + } + metaBytes, err := json.Marshal(metaRes) + assert.NoError(t, err) + + mkResult := func(row []sql.NullString) []byte { + res := stat.PGresult{ + Valid: true, Ncols: ncols, Nrows: 1, Cols: cols, + Values: [][]sql.NullString{row}, + } + b, e := json.Marshal(res) + assert.NoError(t, e) + return b + } + + sysinfoBytes := []byte(`{"ticks":100,"cpu_count":4}`) + + // Compose tar (two ticks; per-tick layout matches tarRecorder.write(): meta + // + archiver + sysinfo). Filenames use the recorder's 20060102T150405.000 + // format — isFilenameOK requires exactly four dot-separated parts, which is + // what the .000 millisecond field provides — and the two ticks are one + // second apart so itv == 1. + var tarBuf bytes.Buffer + tw := tar.NewWriter(&tarBuf) + writeEntry := func(name string, payload []byte) { + hdr := &tar.Header{Name: name, Size: int64(len(payload)), Mode: 0644} + assert.NoError(t, tw.WriteHeader(hdr)) + _, e := tw.Write(payload) + assert.NoError(t, e) + } + timestamps := []string{"20260519T100000.000", "20260519T100001.000"} + for i, ts := range timestamps { + writeEntry("meta."+ts+".json", metaBytes) + if writeStat { + writeEntry("archiver."+ts+".json", mkResult(rows[i])) + } + writeEntry("sysinfo."+ts+".json", sysinfoBytes) + } + assert.NoError(t, tw.Close()) + + // TsStart/TsEnd must bracket the filename dates or isFilenameTimestampOK + // silently skips every entry and any subcase degenerates into the + // empty-archive one while looking like a real recording. + config := Config{ + ReportType: "archiver", + TruncLimit: 32, + TsStart: time.Date(2026, 5, 19, 0, 0, 0, 0, time.Now().Location()), + TsEnd: time.Date(2026, 5, 19, 23, 59, 59, 0, time.Now().Location()), + } + + app := newApp(config) + var buf bytes.Buffer + app.writer = &buf + + tr := tar.NewReader(&tarBuf) + // Hygiene only, NOT evidence: doReport returns nil on every path + // (report/report.go:109-151), so this assertion can never turn red. Every + // claim this test makes is made about the buffer. + assert.NoError(t, app.doReport(tr)) + + return buf.String() +} diff --git a/report/report_record_wal_test.go b/report/report_record_wal_test.go new file mode 100644 index 00000000..d5ad2007 --- /dev/null +++ b/report/report_record_wal_test.go @@ -0,0 +1,243 @@ +package report + +import ( + "archive/tar" + "bytes" + "database/sql" + "encoding/json" + "os" + "regexp" + "testing" + "time" + + "github.com/lesovsky/pgcenter/internal/stat" + "github.com/stretchr/testify/assert" +) + +// Test_app_doReport_WAL exercises the full doReport pipeline for the +// version-aware wal report against a synthetic in-memory tar. Each subcase pins +// one recorded PostgreSQL version (18 / 19) via the meta record's version_num, +// which drives report-time view.Configure -> SelectStatWALQuery. The tar carries +// two cumulative ticks; the first is discarded by processData's first-snapshot +// rule (!prevStat.Valid -> continue) and the second produces the single data +// row. countDiff subtracts prev from curr inside the DiffIntvl range and copies +// everything else — waldir_size, stats_age — from the curr tick verbatim. +// +// Unlike the stat_io pair, whose two branches share one shape, the wal branches +// differ in BOTH Ncols and DiffIntvl (PG 18: 7 cols, {2,5}; PG 19: 8 cols, +// {2,6}, the extra fpi,KiB sitting inside the diffed range). So this replay does +// prove the version switch: feed the 8-column PG 19 sample and let Configure +// pick the PG 18 branch, and the diffed range lands on the wrong columns and the +// golden moves. +// +// What this replay does NOT prove: report replays recorded stat.PGresult JSON, +// so the column names and their order come from the fixtures below, never from +// the SQL. Reordering the aliases inside internal/query/wal.go would not redden +// anything here; that layout is pinned by the query and view unit tests +// (internal/query, internal/view). What is pinned here is the rendering and diff +// pipeline plus the version-driven Configure switch — Ncols, DiffIntvl, +// OrderKey, UniqueKey. +// +// The two ticks are exactly one second apart, so the rate divisor itv == 1 and +// each diffed column equals tick2 - tick1 with no scaling. That spacing is +// load-bearing, not incidental: at two seconds every delta halves. +// +// The fixture values are hostile to a wrongly widened diff range: waldir_size is +// a pretty string ("1040 MB" / "1088 MB") and stats_age an interval ("01:00:00" / +// "02:00:00"), both of which fail strconv.ParseInt — so a DiffIntvl that +// swallows either aborts the sample with "diff failed" rather than producing a +// plausible-looking number. +func Test_app_doReport_WAL(t *testing.T) { + testcases := []struct { + name string + versionNum string + versionStr string + cols []string + // prevVals / currVals are the cumulative values for the two ticks in + // column order. The diffed columns must grow from prev to curr; the + // absolute / text columns are taken from curr verbatim. + prevVals []string + currVals []string + // wantContains / wantNotContains are the pre-golden sentinels: they make + // a failure read as "row missing" or "delta wrong" rather than "golden + // differs". + wantContains []string + wantNotContains []string + wantFile string + }{ + { + // PG18: 7 cols, DiffIntvl [2,5]. Absolute: 0 source, 1 waldir_size, + // 6 stats_age. Diffed: 2..5 (wal,KiB .. buffers_full). + name: "pg18", + versionNum: "180000", + versionStr: "18.0", + cols: []string{ + "source", "waldir_size", "wal,KiB", + "records", "fpi", "buffers_full", + "stats_age", + }, + prevVals: []string{ + "WAL", "1040 MB", "2048.50", + "1000", "300", "12", + "01:00:00", + }, + currVals: []string{ + "WAL", "1088 MB", "3072.75", + "1500", "420", "19", + "02:00:00", + }, + // records delta 1500-1000=500 is the cross-version sentinel; the + // waldir_size string and stats_age are pass-through, and their + // presence proves the diffed range did not swallow them (it would + // have failed the whole sample, emptying the buffer). + wantContains: []string{"WAL", "500", "1088 MB", "02:00:00"}, + // The PG 18 layout has no fpi,KiB column: seeing one here would mean + // Configure picked the PG 19 branch. + wantNotContains: []string{"fpi,KiB"}, + wantFile: "testdata/report_record_wal_pg18.golden", + }, + { + // PG19: 8 cols, DiffIntvl [2,6]. Absolute: 0 source, 1 waldir_size, + // 7 stats_age. Diffed: 2..6, including the new fpi,KiB at index 5 — + // which pushed the end of the diffed range from 5 to 6 so that + // buffers_full stayed inside it. + name: "pg19", + versionNum: "190000", + versionStr: "19.0", + cols: []string{ + "source", "waldir_size", "wal,KiB", + "records", "fpi", "fpi,KiB", "buffers_full", + "stats_age", + }, + prevVals: []string{ + "WAL", "1040 MB", "2048.50", + "1000", "300", "600.00", "12", + "01:00:00", + }, + currVals: []string{ + "WAL", "1088 MB", "3072.75", + "1500", "420", "840.25", "19", + "02:00:00", + }, + // 240.25 = 840.25 - 600.00, formatted by diffPair as %.2f because + // the value contains a dot. Its presence — and the absence of the + // absolute 840.25 — is what proves fpi,KiB landed INSIDE the diffed + // range. + wantContains: []string{"WAL", "500", "fpi,KiB", "240.25", "1088 MB", "02:00:00"}, + wantNotContains: []string{"840.25"}, + wantFile: "testdata/report_record_wal_pg19.golden", + }, + } + + for _, tc := range testcases { + t.Run(tc.name, func(t *testing.T) { + ncols := len(tc.cols) + + // Meta result mirrors SelectCommonProperties (7-column shape; + // readMeta only consumes column index 1 for version_num, which + // drives the version-aware view.Configure at report time). + metaRes := stat.PGresult{ + Valid: true, Ncols: 7, Nrows: 1, + Cols: []string{"version", "version_num", "track_commit_timestamp", "max_connections", "autovacuum_max_workers", "recovery", "start_time_unix"}, + Values: [][]sql.NullString{ + { + {String: tc.versionStr, Valid: true}, {String: tc.versionNum, Valid: true}, + {String: "off", Valid: true}, {String: "100", Valid: true}, {String: "3", Valid: true}, + {String: "false", Valid: true}, {String: "1622828486655396e-6", Valid: true}, + }, + }, + } + metaBytes, err := json.Marshal(metaRes) + assert.NoError(t, err) + + mkRow := func(vals []string) []sql.NullString { + row := make([]sql.NullString, ncols) + for i, v := range vals { + row[i] = sql.NullString{String: v, Valid: true} + } + return row + } + + // Tick 1 (prev): discarded by processData's first-snapshot rule. + // UniqueKey defaults to 0 (the constant "WAL" source), so the single + // row pairs with curr. + statPrev := stat.PGresult{ + Valid: true, Ncols: ncols, Nrows: 1, Cols: tc.cols, + Values: [][]sql.NullString{mkRow(tc.prevVals)}, + } + prevBytes, err := json.Marshal(statPrev) + assert.NoError(t, err) + + // Tick 2 (curr): cumulative values larger than tick 1 in the diffed + // columns; produces the reported data row. + statCurr := stat.PGresult{ + Valid: true, Ncols: ncols, Nrows: 1, Cols: tc.cols, + Values: [][]sql.NullString{mkRow(tc.currVals)}, + } + currBytes, err := json.Marshal(statCurr) + assert.NoError(t, err) + + sysinfoBytes := []byte(`{"ticks":100,"cpu_count":4}`) + + // Compose tar (two ticks; per-tick layout matches + // tarRecorder.write(): meta + wal + sysinfo). The timestamp in each + // filename uses the recorder's 20060102T150405.000 format — four + // dot-separated parts, as isFilenameOK requires — and the two ticks + // are one second apart so itv == 1. + var tarBuf bytes.Buffer + tw := tar.NewWriter(&tarBuf) + writeEntry := func(name string, payload []byte) { + hdr := &tar.Header{Name: name, Size: int64(len(payload)), Mode: 0644} + assert.NoError(t, tw.WriteHeader(hdr)) + _, e := tw.Write(payload) + assert.NoError(t, e) + } + writeEntry("meta.20260519T100000.000.json", metaBytes) + writeEntry("wal.20260519T100000.000.json", prevBytes) + writeEntry("sysinfo.20260519T100000.000.json", sysinfoBytes) + writeEntry("meta.20260519T100001.000.json", metaBytes) + writeEntry("wal.20260519T100001.000.json", currBytes) + writeEntry("sysinfo.20260519T100001.000.json", sysinfoBytes) + assert.NoError(t, tw.Close()) + + // TsStart/TsEnd must bracket the filename dates or + // isFilenameTimestampOK silently skips every entry and the test + // degenerates into an empty report while looking like a real one. + config := Config{ + ReportType: "wal", + TruncLimit: 32, + TsStart: time.Date(2026, 5, 19, 0, 0, 0, 0, time.Now().Location()), + TsEnd: time.Date(2026, 5, 19, 23, 59, 59, 0, time.Now().Location()), + } + + app := newApp(config) + var buf bytes.Buffer + app.writer = &buf + + tr := tar.NewReader(&tarBuf) + assert.NoError(t, app.doReport(tr)) + + out := buf.String() + assert.NotEmpty(t, out) + // Timestamp header line emitted by printStatSample matches + // "YYYY/MM/DD". + assert.Regexp(t, regexp.MustCompile(`\d{4}/\d{2}/\d{2}`), out) + + for _, s := range tc.wantContains { + assert.Contains(t, out, s) + } + for _, s := range tc.wantNotContains { + assert.NotContains(t, out, s) + } + + if *update { + assert.NoError(t, os.WriteFile(tc.wantFile, buf.Bytes(), 0644)) + return + } + + want, err := os.ReadFile(tc.wantFile) + assert.NoError(t, err) + assert.Equal(t, string(want), out) + }) + } +} diff --git a/report/report_test.go b/report/report_test.go index 5bb49519..994c5f5f 100644 --- a/report/report_test.go +++ b/report/report_test.go @@ -1182,6 +1182,7 @@ func Test_describeReport(t *testing.T) { {report: "indexes", want: pgStatIndexesDescription}, {report: "functions", want: pgStatFunctionsDescription}, {report: "wal", want: pgStatWALDescription}, + {report: "archiver", want: pgStatArchiverDescription}, {report: "sizes", want: pgStatSizesDescription}, {report: "progress_vacuum", want: pgStatProgressVacuumDescription}, {report: "progress_cluster", want: pgStatProgressClusterDescription}, @@ -1280,6 +1281,143 @@ func Test_describeActivityColumnOrder(t *testing.T) { } } +// describeRow locates the row documenting column c in a describe constant and returns its offset +// together with the whole row text. The marker is anchored on both sides: "\n- " keeps it off words +// inside the prose, and the trailing tab keeps a short name off a longer row - without it +// "\n- archived" matches the "- archived_age" row, "\n- fpi" matches "- fpi,KiB" and "\n- write" +// matches "- write,ms", so the offsets compared by the callers would not be the ones being checked. +// Presence is required here rather than asserted by the caller: strings.Index returns -1 for a +// missing marker, and -1 is less than anything, so an ordering-only assertion would pass on a row +// that is not there at all. +func describeRow(t *testing.T, text string, c string) (int, string) { + t.Helper() + + pos := strings.Index(text, "\n- "+c+"\t") + require.NotEqual(t, -1, pos, "description must contain a row for %q", c) + + row := text[pos+1:] + if i := strings.Index(row, "\n"); i != -1 { + row = row[:i] + } + + return pos, row +} + +// assertDescribeColumns checks that text documents exactly the given columns, in the given order, +// each with the given origin. Fields are split on whitespace, which collapses the alignment tabs: +// no column name and no origin contains a space, so the row always reads "-", name, origin, prose. +func assertDescribeColumns(t *testing.T, text string, columns []struct{ name, origin string }) { + t.Helper() + + prev := -1 + for _, c := range columns { + pos, row := describeRow(t, text, c.name) + assert.Greater(t, pos, prev, "row %q is out of order", c.name) + prev = pos + + // Strictly greater than 3: a row of exactly name, origin and nothing else documents nothing, + // which is malformed too. + fields := strings.Fields(row) + require.Greater(t, len(fields), 3, "row %q is malformed: %q", c.name, row) + assert.Equal(t, c.origin, fields[2], "row %q documents the wrong origin", c.name) + } + + // The loop above is bounded from below only - it cannot see a row that should not be there, and + // strings.Index reports the first hit, so a duplicated row is invisible to it as well. + assert.Equal(t, len(columns), strings.Count(text, "\n- "), + "description documents a row that is not in the column list") +} + +func Test_describeArchiverColumnOrder(t *testing.T) { + // Same reason as Test_describeActivityColumnOrder: Test_describeReport compares descriptions by + // identity and cannot notice a row that landed in the wrong slot. The list below is a copy of the + // column order of query.PgStatArchiverDefault (internal/query/archiver.go) and must be kept in + // sync with archiverColumns in internal/query/archiver_test.go - it is the layout that + // `report -d -W a` claims to document. + columns := []struct{ name, origin string }{ + {"source", "-"}, + {"ready", "pg_ls_archive_statusdir"}, + {"archived", "archived_count"}, + {"last_archived", "last_archived_wal"}, + {"archived_age", "last_archived_time"}, + {"failed", "failed_count"}, + {"last_failed", "last_failed_wal"}, + {"failed_age", "last_failed_time"}, + {"stats_age", "stats_reset"}, + } + + assertDescribeColumns(t, pgStatArchiverDescription, columns) + + // The source column is a literal in the query, and this constant sits next to + // pgStatWALDescription whose source row says 'WAL' - exactly the value a copy-paste gets wrong. + _, row := describeRow(t, pgStatArchiverDescription, "source") + assert.Contains(t, row, "'Archiver'", "the source row must name the literal the query emits") +} + +func Test_describeArchiverDetailsURL(t *testing.T) { + // Decision 13 keeps `pgcenter report` out of the README, so this line is the only pointer a user + // gets to the upstream documentation of the screen. HasSuffix rather than Contains, so a URL that + // survives but is no longer the closing line is caught too. + assert.True(t, + strings.HasSuffix(strings.TrimRight(pgStatArchiverDescription, "\n"), + "https://www.postgresql.org/docs/current/monitoring-stats.html#PG-STAT-ARCHIVER-VIEW"), + "description must end with the pg_stat_archiver docs URL") +} + +func Test_describeArchiverBlankCells(t *testing.T) { + // The four NULL-able columns render as blank cells by design (Decision 3, nothing is diffed and + // no coalesce is applied), and describe is the only place a user finds out a blank is not an + // error. Each row is asserted on its own, so deleting the clause from one row still reddens. + testcases := []struct { + column string + clause string + }{ + {column: "last_archived", clause: "empty if nothing has been archived yet"}, + {column: "archived_age", clause: "empty if nothing has been archived yet"}, + {column: "last_failed", clause: "empty if there were no failures"}, + {column: "failed_age", clause: "empty if there were no failures"}, + } + + for _, tc := range testcases { + t.Run(tc.column, func(t *testing.T) { + _, row := describeRow(t, pgStatArchiverDescription, tc.column) + assert.Contains(t, row, tc.clause, "row %q must say what a blank cell means", tc.column) + }) + } +} + +func Test_describeWALColumnOrder(t *testing.T) { + // Same reason as Test_describeActivityColumnOrder. The list below is the PG 14 superset layout + // the constant documents (it keeps write/sync, removed from pg_stat_wal in PG 18, and includes + // the PG 19 fpi,KiB) - describe has no version awareness, see the constant's comment. The point + // here is that fpi,KiB exists and sits between fpi and write. + columns := []struct{ name, origin string }{ + {"source", "-"}, + {"waldir_size", "-"}, + {"wal,KiB", "wal_bytes"}, + {"records", "wal_records"}, + {"fpi", "wal_fpi"}, + {"fpi,KiB", "wal_fpi_bytes"}, + {"write", "wal_write"}, + {"sync", "wal_sync"}, + {"write,ms", "wal_write_time"}, + {"sync,ms", "wal_sync_time"}, + {"buffers_full", "wal_buffers_full"}, + {"stats_age", "stats_reset"}, + } + + assertDescribeColumns(t, pgStatWALDescription, columns) +} + +func Test_describeWALFPIVersionNote(t *testing.T) { + // wal_fpi_bytes exists only on PG 19+, while the constant is static and version-unaware, so the + // row is printed when describing a PG 14-18 archive as well. The annotation is what keeps that + // honest; without it the row silently claims a column those versions do not have. + _, row := describeRow(t, pgStatWALDescription, "fpi,KiB") + assert.True(t, strings.HasSuffix(row, "(PG 19+)"), + "the fpi,KiB row must be annotated with the version that introduced wal_fpi_bytes") +} + func Test_describeActivityCaveats(t *testing.T) { // Each of the three new columns can be misread into a wrong pg_terminate_backend decision, // and there is nothing to catch in code for any of them - the caveats in this block are the diff --git a/report/testdata/report_record_archiver.golden b/report/testdata/report_record_archiver.golden new file mode 100644 index 00000000..a95e783a --- /dev/null +++ b/report/testdata/report_record_archiver.golden @@ -0,0 +1,3 @@ +source ready archived last_archived archived_age failed last_failed failed_age stats_age  +2026/05/19 10:00:01, rate: 1s +Archiver 14 100003 000000010000000000000024 00:00:07 8 000000010000000000000019 00:14:02 02:00:00 diff --git a/report/testdata/report_record_wal_pg18.golden b/report/testdata/report_record_wal_pg18.golden new file mode 100644 index 00000000..72f1a156 --- /dev/null +++ b/report/testdata/report_record_wal_pg18.golden @@ -0,0 +1,3 @@ +source waldir_size wal,KiB records fpi buffers_full stats_age  +2026/05/19 10:00:01, rate: 1s +WAL 1088 MB 1024.25 500 120 7 02:00:00 diff --git a/report/testdata/report_record_wal_pg19.golden b/report/testdata/report_record_wal_pg19.golden new file mode 100644 index 00000000..1a245169 --- /dev/null +++ b/report/testdata/report_record_wal_pg19.golden @@ -0,0 +1,3 @@ +source waldir_size wal,KiB records fpi fpi,KiB buffers_full stats_age  +2026/05/19 10:00:01, rate: 1s +WAL 1088 MB 1024.25 500 120 240.25 7 02:00:00 diff --git a/top/config_view.go b/top/config_view.go index 671dece2..3657e8bb 100644 --- a/top/config_view.go +++ b/top/config_view.go @@ -247,6 +247,14 @@ func switchViewTo(app *app, c string) func(g *gocui.Gui, _ *gocui.View) error { viewSwitchHandler(app.config, progressNextView(app.config.view.Name)) case "statio": viewSwitchHandler(app.config, statioNextView(app.config.view.Name)) + // Unlike every other cycle here, "wal" is simultaneously the name of the group and the name + // of its first view - the view cannot be renamed, it is the 'report -W wal' report type and + // the tar entry prefix in recorded archives. This is the group's single dispatch point, and + // walNextView's default arm returns "wal", so 'w' pressed on any other screen still lands on + // the wal screen exactly as it did before the cycle existed. Do not "fix" this into a + // separate group name. + case "wal": + viewSwitchHandler(app.config, walNextView(app.config.view.Name)) default: viewSwitchHandler(app.config, c) } @@ -286,6 +294,21 @@ func statioNextView(current string) string { return next } +// walNextView depending on current WAL view returns next view. +func walNextView(current string) string { + var next string + + switch current { + case "wal": + next = "archiver" + case "archiver": + next = "wal" + default: + next = "wal" + } + return next +} + // statementsNextView depending on current statements view returns next view. func statementsNextView(current string) string { var next string diff --git a/top/config_view_test.go b/top/config_view_test.go index 4175f2d3..8aeb4f72 100644 --- a/top/config_view_test.go +++ b/top/config_view_test.go @@ -621,6 +621,12 @@ func Test_switchViewTo(t *testing.T) { {current: "activity", to: "statio", want: "stat_io"}, {current: "stat_io", to: "statio", want: "stat_io_time"}, {current: "stat_io_time", to: "statio", want: "stat_io"}, + // The 'w' cycle. The first row is the one that proves the dispatch case exists: the other + // two land on "wal" through walNextView's default arm as well, so they stay green even + // without the case. + {current: "wal", to: "wal", want: "archiver"}, + {current: "archiver", to: "wal", want: "wal"}, + {current: "activity", to: "wal", want: "wal"}, } wg := sync.WaitGroup{} @@ -638,8 +644,12 @@ func Test_switchViewTo(t *testing.T) { fn := switchViewTo(app, tc.to) assert.NoError(t, fn(nil, nil)) + + // Inside the closure, not after it: the assertion lives in the goroutine above, and + // waiting outside would let a failing row call t.Errorf on a finished subtest - which + // panics ("Fail in goroutine after ... has completed") instead of naming the row. + wg.Wait() }) - wg.Wait() } close(app.config.viewCh) @@ -682,6 +692,21 @@ func Test_statioNextView(t *testing.T) { } } +func Test_walNextView(t *testing.T) { + testcases := []struct { + current string + want string + }{ + {current: "wal", want: "archiver"}, + {current: "archiver", want: "wal"}, + {current: "unknown", want: "wal"}, + } + + for _, tc := range testcases { + assert.Equal(t, tc.want, walNextView(tc.current)) + } +} + func Test_statementsNextView(t *testing.T) { testcases := []struct { current string diff --git a/top/help.go b/top/help.go index 31993db0..783ca0b8 100644 --- a/top/help.go +++ b/top/help.go @@ -11,12 +11,13 @@ const ( general actions: a,b,f,o mode: 'a' activity, 'b' bgwriter/checkpointer, 'f' functions, 'o' replication slots, - r,w 'r' replication, 'w' WAL, + r 'r' replication, s,t,i 's' tables sizes, 't' tables, 'i' indexes. d,D 'd' pg_stat_database switch, 'D' pg_stat_database menu. x,X 'x' pg_stat_statements switch, 'X' pg_stat_statements menu. p,P 'p' pg_stat_progress_* switch, 'P' pg_stat_progress_* menu. j,J 'j' pg_stat_io switch (operations/timings), 'J' pg_stat_io menu. + w,W 'w' pg_stat_wal / pg_stat_archiver switch, 'W' WAL statistics menu. S 'S' per-process system stats (local mode only; Shift+S). Left,Right,<,/ 'Left,Right' change column sort, '<' desc/asc sort toggle, '/' set filter. \ '\' clear all filters of the current screen. @@ -42,7 +43,7 @@ activity actions: other actions: , Q ',' show system tables on/off, 'Q' reset postgresql statistics counters - ('Q' does not reset shared stats: pg_stat_io, bgwriter, wal). + ('Q' does not reset shared stats: pg_stat_io, bgwriter, wal, archiver). z 'z' set refresh interval. h,F1 show this tab. q,Ctrl+Q quit. diff --git a/top/help_test.go b/top/help_test.go index 1efbb6e9..08c90214 100644 --- a/top/help_test.go +++ b/top/help_test.go @@ -110,6 +110,48 @@ func Test_helpTemplate_pauseLiftingActions(t *testing.T) { assert.True(t, strings.HasSuffix(cont, "resume it."), "continuation line is %q", cont) } +// The 'w' key stopped being a one-way switch to the WAL screen and became a two-screen cycle with +// a menu of its own, so the help entry moved out of the 'a,b,f,o' block into a row of its own next +// to its 'j,J' precedent. The marker is "'w' " with the trailing space rather than the longer +// "'w' pg_stat_wal": helpEntryLine fails on an AMBIGUOUS marker, which is what makes putting the +// old "'w' WAL," clause back on the 'r' line fail here. +func Test_helpTemplate_walEntry(t *testing.T) { + entryIdx, entry := helpEntryLine(t, "'w' ") + statioIdx, statio := helpEntryLine(t, "'j' pg_stat_io switch") + + assert.True(t, strings.HasPrefix(entry, " w,W"), "entry line is %q", entry) + + // Pinned word for word, so a reworded description fails here instead of shipping. Only the + // description is compared - the padding in front of it is the alignment check's business. + assert.Equal(t, "'w' pg_stat_wal / pg_stat_archiver switch, 'W' WAL statistics menu.", entry[descColumn(entry):]) + + // It sits directly after its 'j,J' precedent, and shares the block's description column. + assert.Equal(t, statioIdx+1, entryIdx) + assert.Equal(t, descColumn(statio), descColumn(entry)) +} + +// The other half of the same edit: the 'r,w' row lost its 'w' and kept the replication clause +// alone. Both the key token and the description are pinned, because a half-applied edit (the key +// token left as 'r,w', or the WAL clause left in place) is what a reader would actually hit. +func Test_helpTemplate_replicationEntry(t *testing.T) { + _, entry := helpEntryLine(t, "'r' replication,") + _, sizes := helpEntryLine(t, "'s' tables sizes") + + assert.True(t, strings.HasPrefix(entry, " r "), "entry line is %q", entry) + assert.NotContains(t, entry, "'w'") + assert.Equal(t, "'r' replication,", entry[descColumn(entry):]) + + assert.Equal(t, descColumn(sizes), descColumn(entry)) +} + +// 'Q' does not reset the shared statistics, and the archiver screen is now one of them. Asserted +// on that single line: a bare "archiver" substring would match anywhere in the template. +func Test_helpTemplate_resetCaveat(t *testing.T) { + _, caveat := helpEntryLine(t, "'Q' does not reset shared stats") + + assert.Contains(t, caveat, "pg_stat_io, bgwriter, wal, archiver") +} + // helpTemplate is a format string for fmt.Fprintf (see showHelp). An ordinary stray or // mismatched verb is already caught harder and earlier by go vet's printf check, which // go test runs as part of the build — this test does not add to that. What it does add is diff --git a/top/keybindings.go b/top/keybindings.go index 0a09d31e..49cd0d3d 100644 --- a/top/keybindings.go +++ b/top/keybindings.go @@ -15,7 +15,23 @@ type key struct { // keybindings set up key bindings with handlers. func keybindings(app *app) error { - var keys = []key{ + app.ui.InputEsc = true + + for _, k := range keybindingsList(app) { + if err := app.ui.SetKeybinding(k.viewname, k.key, gocui.ModNone, k.handler); err != nil { + return fmt.Errorf("setup keybindings failed: %w", err) + } + } + + return nil +} + +// keybindingsList returns the table of key bindings. It is split out of keybindings() because a +// handler is observable only through this slice: gocui keeps its registered bindings unexported and +// offers no way to fetch or run one, so which handler a key carries could otherwise be checked by +// reading the table only. With the table returned, a test can pick a row and call its handler. +func keybindingsList(app *app) []key { + return []key{ {"", gocui.KeyCtrlC, app.quit()}, {"", gocui.KeyCtrlQ, app.quit()}, {"sysstat", 'q', app.quit()}, @@ -47,6 +63,7 @@ func keybindings(app *app) error { {"sysstat", 'X', menuOpen(menuPgss, app.config, app.postgresProps.ExtPGSSSchema)}, {"sysstat", 'P', menuOpen(menuProgress, app.config, "")}, {"sysstat", 'J', menuOpen(menuStatIO, app.config, "")}, + {"sysstat", 'W', menuOpen(menuWAL, app.config, "")}, {"sysstat", 'l', showPgLog(app.db, app.postgresProps.VersionNum, app.uiExit)}, {"sysstat", 'C', showPgConfig(app.db, app.uiExit)}, {"sysstat", '~', runPsql(app.db, app.uiExit)}, @@ -83,14 +100,4 @@ func keybindings(app *app) error { {"help", gocui.KeyEsc, closeHelp}, {"help", 'q', closeHelp}, } - - app.ui.InputEsc = true - - for _, k := range keys { - if err := app.ui.SetKeybinding(k.viewname, k.key, gocui.ModNone, k.handler); err != nil { - return fmt.Errorf("setup keybindings failed: %w", err) - } - } - - return nil } diff --git a/top/keybindings_test.go b/top/keybindings_test.go new file mode 100644 index 00000000..65e6ed27 --- /dev/null +++ b/top/keybindings_test.go @@ -0,0 +1,143 @@ +package top + +import ( + "testing" + "time" + + "github.com/jroimartin/gocui" + "github.com/lesovsky/pgcenter/internal/view" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// newKeybindingsApp builds an app whose keybindings have been registered on a zero-value Gui. +// That is enough: gocui.SetKeybinding only appends to a slice and DeleteKeybinding scans it, +// neither touches a terminal. +func newKeybindingsApp(t *testing.T) *app { + t.Helper() + + app := &app{config: newConfig(), ui: &gocui.Gui{}} + require.NoError(t, keybindings(app)) + return app +} + +// boundHandler returns the handler the keys table carries for the given view and key, failing the +// test when the row is absent. Uniqueness is Test_keybindingsWAL's business; this helper is about +// WHICH handler a key carries, which no assertion on the registered gocui bindings can reach. +// +// The returned closure is freshly constructed by keybindingsList rather than the very one gocui +// registered - equivalent as long as every row is a stateless constructor over app/app.config, +// which is what the table contains today. +func boundHandler(t *testing.T, app *app, viewname string, k any) func(*gocui.Gui, *gocui.View) error { + t.Helper() + + for _, b := range keybindingsList(app) { + if b.viewname == viewname && b.key == k { + return b.handler + } + } + + t.Fatalf("no binding for key %v on view %q", k, viewname) + return nil +} + +// Test_keybindingsWAL pins the 'W' binding of the WAL menu. 'W' being free was a reading of +// keybindings.go before this test existed; DeleteKeybinding turns that reading into a regression +// guard - it removes the first match and reports "keybinding not found" otherwise, so a single +// successful delete followed by a failing one means exactly one binding claims the key. +// +// Probing is destructive, so every assertion group gets a freshly registered app. +func Test_keybindingsWAL(t *testing.T) { + t.Run("registered on sysstat exactly once", func(t *testing.T) { + app := newKeybindingsApp(t) + + assert.NoError(t, app.ui.DeleteKeybinding("sysstat", 'W', gocui.ModNone)) + + err := app.ui.DeleteKeybinding("sysstat", 'W', gocui.ModNone) + require.Error(t, err) + assert.Equal(t, "keybinding not found", err.Error()) + }) + + t.Run("not claimed by any other view", func(t *testing.T) { + for _, viewname := range []string{"", "menu", "dialog", "help"} { + app := newKeybindingsApp(t) + + err := app.ui.DeleteKeybinding(viewname, 'W', gocui.ModNone) + require.Error(t, err, "view %q must not bind 'W'", viewname) + assert.Equal(t, "keybinding not found", err.Error()) + } + }) + + // The other edge of the lower-case half: Test_keybindingsWALCycles runs the handler the TABLE + // carries, which stays green if the registration loop ever skips the row. This probe watches + // what gocui actually received. + // + // Exactly once, like the 'W' probe above, and for a sharper reason: gocui's execKeybindings + // runs EVERY matching handler rather than stopping at the first, so a duplicated 'w' row would + // cycle twice per press (wal -> archiver -> wal) and read as a dead key. + t.Run("lower-case 'w' still registered", func(t *testing.T) { + app := newKeybindingsApp(t) + + assert.NoError(t, app.ui.DeleteKeybinding("sysstat", 'w', gocui.ModNone)) + + err := app.ui.DeleteKeybinding("sysstat", 'w', gocui.ModNone) + require.Error(t, err) + assert.Equal(t, "keybinding not found", err.Error()) + }) +} + +// Test_keybindingsWALOpensMenu pins WHICH menu 'W' opens - the half that a uniqueness probe cannot +// see. Without it, binding 'W' to menuOpen(menuStatIO, ...) leaves the whole suite green while the +// user-visible behaviour is wrong. +// +// The handler is run rather than compared: closures are not comparable, and the effect is what +// matters. It also drives menuOpen for real, so the title and the item strings - the only text the +// 'W' path shows before a selection is made - are pinned here rather than by review. +// +// menuOpen ignores its *gocui.View argument, and on a zero-value &gocui.Gui{} its SetView, +// SetCurrentView and menuDraw calls all work without a terminal (see Test_menuSelectWAL). +func Test_keybindingsWALOpensMenu(t *testing.T) { + app := &app{config: newConfig(), ui: &gocui.Gui{}} + + require.NoError(t, boundHandler(t, app, "sysstat", 'W')(app.ui, nil)) + + assert.Equal(t, menuWAL, app.config.menu.menuType) + assert.Equal(t, " Choose WAL / archiver mode (Enter to choose, Esc to exit): ", app.config.menu.title) + assert.Equal(t, []string{" pg_stat_wal", " pg_stat_archiver"}, app.config.menu.items) + + mv, err := app.ui.View("menu") + require.NoError(t, err) + assert.Equal(t, app.config.menu.title, mv.Title) + + // Not the same assertion as the items slice above: that one reads the style menuOpen stored in + // the config, this one reads what menuDraw actually wrote into the window. Without it, a + // menuDraw that draws nothing is invisible to the whole package. + for _, item := range app.config.menu.items { + assert.Contains(t, mv.Buffer(), item) + } +} + +// Test_keybindingsWALCycles is the lower-case half. The 'w' row is byte-identical to what it was +// before this feature - only its MEANING changed, because switchViewTo gained the "wal" case - so +// nothing but a diff review stood between the primary entry point of the feature and silent +// deletion. Running the bound handler from the wal screen and expecting the archiver screen pins +// both facts at once: 'w' is still bound, and it is bound to the cycle. +// +// Same intentional leak as Test_menuSelectWAL: printCmdline calls g.Update, whose goroutine parks +// forever on the zero Gui's nil userEvents channel. +func Test_keybindingsWALCycles(t *testing.T) { + app := &app{config: newConfig(), ui: &gocui.Gui{}} + app.config.view = app.config.views["wal"] + + received := make(chan view.View, 1) + go func() { received <- <-app.config.viewCh }() + + require.NoError(t, boundHandler(t, app, "sysstat", 'w')(app.ui, nil)) + + select { + case v := <-received: + assert.Equal(t, "archiver", v.Name) + case <-time.After(time.Second): + t.Fatal("'w' did not send the next view on viewCh") + } +} diff --git a/top/menu.go b/top/menu.go index fdaf1cd7..ef62290b 100644 --- a/top/menu.go +++ b/top/menu.go @@ -19,6 +19,7 @@ const ( menuProgress // menu with pg_stat_progress_* stats menuConf // menu with configuration files menuStatIO // menu with pg_stat_io stats + menuWAL // menu with pg_stat_wal / pg_stat_archiver stats // Directions allowed when working with menu. moveUp direction = iota // move up @@ -93,6 +94,15 @@ func selectMenuStyle(t menuType) menuStyle { " pg_stat_io timings", }, } + case menuWAL: + s = menuStyle{ + menuType: menuWAL, + title: " Choose WAL / archiver mode (Enter to choose, Esc to exit): ", + items: []string{ + " pg_stat_wal", + " pg_stat_archiver", + }, + } default: s = menuStyle{ menuType: menuNone, @@ -201,6 +211,16 @@ func menuSelect(app *app) func(g *gocui.Gui, v *gocui.View) error { viewSwitchHandler(app.config, "stat_io") } printCmdline(app.ui, "%s", app.config.view.Msg) + case menuWAL: + switch cy { + case 0: + viewSwitchHandler(app.config, "wal") + case 1: + viewSwitchHandler(app.config, "archiver") + default: + viewSwitchHandler(app.config, "wal") + } + printCmdline(app.ui, "%s", app.config.view.Msg) case menuConf: switch cy { case 0: diff --git a/top/menu_test.go b/top/menu_test.go index 1809249f..917a15ca 100644 --- a/top/menu_test.go +++ b/top/menu_test.go @@ -1,8 +1,13 @@ package top import ( - "github.com/stretchr/testify/assert" "testing" + "time" + + "github.com/jroimartin/gocui" + "github.com/lesovsky/pgcenter/internal/view" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" ) func Test_selectMenuStyle(t *testing.T) { @@ -16,6 +21,7 @@ func Test_selectMenuStyle(t *testing.T) { {menu: menuProgress, want: 6}, {menu: menuConf, want: 4}, {menu: menuStatIO, want: 2}, + {menu: menuWAL, want: 2}, } for _, tc := range testcases { @@ -23,3 +29,66 @@ func Test_selectMenuStyle(t *testing.T) { assert.Equal(t, tc.want, len(got.items)) } } + +// Test_menuSelectWAL drives the real menuSelect closure over the menuWAL branch: the cursor +// position is resolved to a view name, the view is sent on viewCh and the menu is reset. +// +// A zero-value &gocui.Gui{} is enough for that: SetView builds a real *gocui.View from the passed +// coordinates without touching a terminal (ErrUnknownView is its "created" signal, which menu.go +// keys off too), while DeleteView/SetCurrentView only scan a slice. The "sysstat" view exists +// because menuClose focuses it at the end of every menuSelect path; without it menuSelect would +// return ErrUnknownView. The "menu" view is taller than the production geometry (menu.go sizes it +// 0,5..72,6+len(items)) so SetCursor can reach a row beyond the two items and exercise the +// default arm — out-of-view rows are rejected with "invalid point". +// +// Note the leak: printCmdline calls g.Update, which spawns a goroutine that parks forever on the +// zero Gui's nil userEvents channel — one per sub-test. Same intentional class as +// Test_showExtraCloseLifts (top/pause_test.go): a goroutine-leak detector would need an exemption +// here rather than a "fix". +// +// What this does NOT cover: menuOpen itself. The menu state is built by hand from +// selectMenuStyle(menuWAL) — the same state menuOpen would leave — so the 'W' → menu-window step +// (title, geometry, menuDraw) is covered by Test_selectMenuStyle plus the stand run. +func Test_menuSelectWAL(t *testing.T) { + testcases := []struct { + name string + cy int + want string + }{ + {name: "first item", cy: 0, want: "wal"}, + {name: "second item", cy: 1, want: "archiver"}, + {name: "out of range", cy: 5, want: "wal"}, + } + + for _, tc := range testcases { + t.Run(tc.name, func(t *testing.T) { + g := &gocui.Gui{} + + _, err := g.SetView("sysstat", 0, 0, 20, 20) + require.Equal(t, gocui.ErrUnknownView, err) + + mv, err := g.SetView("menu", 0, 5, 72, 20) + require.Equal(t, gocui.ErrUnknownView, err) + require.NoError(t, mv.SetCursor(0, tc.cy)) + + app := &app{config: newConfig(), ui: g} + app.config.view = app.config.views["activity"] + app.config.menu = selectMenuStyle(menuWAL) + + received := make(chan view.View, 1) + go func() { received <- <-app.config.viewCh }() + + assert.NoError(t, menuSelect(app)(g, mv)) + + select { + case v := <-received: + assert.Equal(t, tc.want, v.Name) + case <-time.After(time.Second): + t.Fatal("menu selection did not send the new view on viewCh") + } + + // The reset at the end of menuSelect is what keeps the next menu press sane. + assert.Equal(t, menuNone, app.config.menu.menuType) + }) + } +} diff --git a/top/pause_test.go b/top/pause_test.go index bba4e9f3..b254f54d 100644 --- a/top/pause_test.go +++ b/top/pause_test.go @@ -558,14 +558,14 @@ func Test_noOpHandlersKeepPause(t *testing.T) { // has nothing to lift; and the branch calls it directly instead of going through viewSwitchHandler, // which is where the lift for every other screen switch lives. // -// menuSelect itself is unreachable from a unit test, but NOT because of its *gocui.View argument: -// a zero-value &gocui.View{} is constructible from outside the gocui package and answers v.Cursor() -// with (0,0), which is enough to route into the menuConf branch. The real blocker is one line -// later - menuSelect ends with an unconditional `return menuClose(g, v)` on EVERY branch, and -// menuClose calls g.DeleteView("menu") and g.SetCurrentView("sysstat") on the *gocui.Gui -// (top/menu.go). A nil Gui panics there, and a live one comes only from gocui.NewGui, which opens a -// real terminal backend. No formulation of the test can dodge it, because no branch skips -// menuClose. +// menuSelect itself IS reachable from a unit test - Test_menuSelectWAL (top/menu_test.go) drives it +// over a zero-value &gocui.Gui{}, which is enough for every branch: SetView builds a real view from +// the passed coordinates without a terminal, and the unconditional `return menuClose(g, v)` that +// ends menuSelect only makes DeleteView/SetCurrentView scan a slice. (A NIL Gui does panic there; +// that is a different thing from a zero-value one.) What still needs a real environment is narrower +// than "menuSelect": only the local-DB editor path inside editPgConfig, past its `!db.Local` early +// return (top/pgconfig.go) - the branch up to that return is drivable with &gocui.Gui{} plus +// &postgres.DB{Local: false}, the idiom top/config_view_test.go already uses. // // An earlier revision of this file did have a Test_menuConfPathDoesNotLift. It built a local // config, called editPgConfig, and asserted the local config was still paused - on a config the