Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
history_version: 1
target:
kind: infrastructure
path: scripts/audit_causal_graphs.py
slug: causal-graph-connectivity-metric
session:
id: 2026-08-08T044057Z-claude-code-ef6836
timestamp: '2026-08-08T04:40:57Z'
actors:
- type: ai_agent
name: claude-code
links:
issues:
- https://github.com/CultureBotAI/TraitMech/issues/359
events:
- type: EDIT
outcome: changed
sections:
- causal_graphs
summary: Measure component structure, the connectivity number retyping cannot move
details: 'Issue 359 observed that UNREACHABLE_FROM_TRAIT falls when a node is retyped into
a TRAIT anchor inside an existing island, without the graph becoming more connected. Neither
headline count can tell that apart from a real gain: FRAGMENTED_GRAPH reports one finding
per split graph however many pieces it is in, so 3 components to 2 does not move it either.
Added connectivity_rows(), emitting reports/causal_graph_connectivity.tsv with one row
per graph carrying wired_nodes, components, largest_component and the component sizes,
plus a corpus summary line on the audit. Measured on the worked example from 352, oxygen_preference.yaml:
main and the RETYPED version are bit-for-bit identical at components=3 largest=8 attached=57.1
percent, while the MERGED version is components=2 largest=11 attached=84.6 percent - and
UNREACHABLE_FROM_TRAIT is 1296 for both fixes, which is exactly the blindness 359 describes.
Corpus baseline: 353 graphs, 864 components over 4137 wired nodes, attached 69.5 percent.
The metric is anchor-free for the same reason _components is, so no amount of retyping
or renaming moves it. Scoped to edge-referenced nodes to match FRAGMENTED_GRAPH, since
an unwired node is ORPHAN_NODE''s business and counting it here would let one defect depress
two metrics. Topology extraction is now a shared _topology() helper used by both audit()
and the metric so the ratchet and the measurement cannot drift on what connected means;
verified behaviour-preserving by diffing regenerated causal_graph_audit.tsv against the
committed copy - identical. Carries no ratchet of its own: it is a measurement, not a
verdict, and never affects the exit code. Wired into audit-derived-reports for staleness,
passing --connectivity-out to the temp dir, because omitting it would have made a staleness
check write the file it was judging. 524 tests pass (5 new), qc green.'
31 changes: 31 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -584,7 +584,13 @@ audit-derived-reports:
# more fragmented" with "the committed report is out of date". So the status
# is deliberately ignored and only missing output is fatal.
cga=causal_graph_audit.tsv
cgc=causal_graph_connectivity.tsv
# --connectivity-out is NOT optional here. Omitting it would let this
# check write the connectivity report to its default path in the working
# tree -- a staleness check that mutates the file it is judging, which is
# the failure this recipe's header warns about.
uv run python scripts/audit_causal_graphs.py --out "$tmp/$cga" \
--connectivity-out "$tmp/$cgc" \
> "$tmp/gen.log" 2>&1 || true
if [ ! -s "$tmp/$cga" ]; then
echo "ERROR: audit_causal_graphs.py produced no report. Its output:" >&2
Expand Down Expand Up @@ -615,6 +621,31 @@ audit-derived-reports:
fail=1
fi

# --- causal_graph_connectivity.tsv, compared against git (#359) ----------
# Written by the same generator invocation above, so it needs no second
# run. It carries no ratchet of its own: it is a MEASUREMENT of component
# structure, and the whole point of #359 is that a number which cannot be
# gamed by retyping is worth having even when nothing gates on it. Staleness
# still matters -- an out-of-date copy would misreport whether a PR actually
# connected anything, which is the one question it exists to answer.
if [ ! -s "$tmp/$cgc" ]; then
echo "ERROR: audit_causal_graphs.py produced no connectivity report. Its output:" >&2
cat "$tmp/gen.log" >&2
exit 1
fi
if ! git show "HEAD:reports/$cgc" > "$tmp/committed_$cgc" 2>/dev/null; then
echo " MISSING reports/$cgc is not in git at HEAD" >&2
stale_cga=1
fail=1
elif diff -q "$tmp/committed_$cgc" "$tmp/$cgc" >/dev/null; then
echo " OK reports/$cgc (vs git)"
else
echo " STALE reports/$cgc — the COMMITTED copy is not what audit-graphs produces:" >&2
{ diff -u "$tmp/committed_$cgc" "$tmp/$cgc" | sed -n '1,20p' >&2; } || true
stale_cga=1
fail=1
fi

# --- predicate_domain_audit.tsv, compared against git (#301) -------------
# Same shape as causal_graph_audit.tsv above: this generator's exit code is
# its RATCHET VERDICT (--fail-on new), owned by `audit-predicate-domains`
Expand Down
Loading
Loading