Overlap (venn) analysis of comments from different models/systems used in paper

[dangng2004]

Summary

Refactors the review_analysis venn/cluster plumbing into a shared helper module, then adds two new comparison axes on top of it, and regenerates all paper Venn figures in a consistent main-paper style.

Refactor

• utils.py — shared load / para_set / regions_{2,3} / draw_venn{2,3} / save_fig; plots now written to plots/ in both PNG and PDF
• analysis.py, analysis_gpt_claude.py — refactored to use utils; old top-level venn PNGs deleted (regenerated under plots/)

New comparisons

• analysis_three_systems.py — 3-way paragraph-index overlap of coarse / OpenAIReview / Reviewer 3, computed on the perturbation benchmark (best model per system)
• analysis_with_humans.py — overlap between human OpenReview reviewers and the AI-system union; two-pass LLM concern-extraction + paragraph-mapping with on-disk .cache/
• cluster_new.py — KMeans clustering for the two new comparisons

Figure regeneration (second commit)

• All Venns restyled to the single-panel main-paper look: model-name titles, no in-figure Jaccard (reported in table columns instead), larger fonts, consistent palette
• analysis_gpt_claude.py retargeted to the perturbation results tree (one cell per domain x paper x error_type)
• analysis_claude_gpt_efficient.py / analysis_claude_gpt_efficient_outcomes.py — 3-way Claude vs GPT vs efficient-model union, on the perturbation and quality-proxy papers respectively
• analysis_union_models.py — three-system overlap where each system unions over all its backbone models
• regen_appendix_venns.py — regenerates the appendix per-model Venn grids from the region averages reported in the appendix tables

Other

• .gitignore — cover plots/, .cache/, generated cluster_*.json / per_paper_*.json, and the local frontier_subset_progressive symlink

Note: _combine_gpt_claude.py (combined GPT-or-Claude recall) was pulled out to #97; it is recall tooling, not overlap analysis.

Test plan

• python analysis.py produces plots/venn_cp.{png,pdf} and plots/venn_all.{png,pdf} without errors
• python analysis_three_systems.py produces the 3-way overlap plot
• python analysis_with_humans.py produces the human-vs-AI venn (uses cached LLM outputs on rerun)
• python analysis_gpt_claude.py produces the Claude-vs-GPT venn from the perturbation tree
• python analysis_union_models.py and the two analysis_claude_gpt_efficient* scripts produce their venns
• python regen_appendix_venns.py regenerates venn_cp / venn_all grids

🤖 Generated with Claude Code