Skip to content

Shrink demo data for Bioconductor + fix protein-name resolution + fix cross-model drug selection - #39

Merged
eboyer221 merged 2 commits into
mainfrom
dev-shrink-demo-data
Jul 10, 2026
Merged

eboyer221 merged 2 commits into
mainfrom
dev-shrink-demo-data

Conversation

@eboyer221

@eboyer221 eboyer221 commented Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

The bundled inst/extdata parquets have more data than the dashboard can display, pushing several files over Bioconductor's size expectations. This PR trims them, adds reproducible regeneration scripts, and fixes two rendering bugs. Total inst/extdata: 55 MB → 7.3 MB.

1. Top-features subset : inst/scripts/subset_demo_top_features.R

Regenerates every {all,country,year}_top_features.parquet:

  • Drops shuffled == TRUE rows (permutation-null baseline; read by no code path. I confirmed with @AbhirupaGhosh that these aren't a dashboard input).
  • Keeps the top 100 features per model, matching the ≤100 caps on the Feature Importance and Cross-model sliders. Drops the unread filename / seed_from_name columns from the stratified files.
file before after
sonnei/country_top_features 22.0 MB 1.38 MB
sonnei/all_top_features 15.6 MB 0.80 MB
sonnei/year_top_features 5.3 MB 0.81 MB
flexneri/all_top_features 11.8 MB 0.35 MB
flexneri/country_top_features 0.67 MB 0.09 MB
flexneri/year_top_features 0.67 MB 0.09 MB

"Model" granularity: top-100 is applied per
(species, drug, feature_type, feature_subtype[, strat_label, strat_value]),

  1. Name-map subset :inst/scripts/subset_demo_name_maps.R
    Trims whole-proteome/-genome name maps to only the IDs a top-features Variable can reference (run after script 1). sonnei protein_names.parquet:
    10.5 MB (9.56M rows) → 0.10 MB (6,135 rows).

  2. Fix: protein names never resolved: R/utils_annotation.R protein_names.parquet keys use fig|624... while the top-features Variable uses fig.624..., and load_feature_name_map() didn't normalize the |, so the Feature Importance protein lookup matched 0 rows. Now normalizes |→. (as the annotated-join path already does), so protein features get labels.

  3. Fix: cross-model performance ignored the selected drug : R/plots_crossmodel.R, makeCrossModelPerformancePlot() filtered with a bare drug, but the performance data has a column literally named drug. Result: the plot ignored the drug selection entirely and rendered the same drug-agnostic matrix for every drug (identical output for AMP / CIP / TET, and even for a nonsense value). For flexneri the broken filter matched 0 rows, so the panel was blank.
    Fixes:

  • drug → .env$drug so the selected drug is used.
  • Replaced the self-eval if_else (which throws when strat_value_test is an all-NA logical column, e.g. flexneri) with a type-robust coalesce, otherwise fixing the filter would surface a crash instead of a blank.
  • Blank panels now show an explanatory title (e.g. "No data for selection", or "Not enough data to compare: needs models trained on 2+ countries…") instead of an empty white box, matching the placeholder convention already used by makeCrossModelRidgePlot().
  • Updated the two test-plots.R cases that used the old NULL return to expect a plotly placeholder.

Not touched (kept intentionally)

  • sonnei/cluster_feature.parquet is currently unreferenced by the dashboard, but may be the raw source for regenerating sonnei's COG annotation file?
  • sonnei/MDR_* is not currently being used, however these files were kept because MDR plots will be a future dashboard feature.

Verification

  • Network, Feature Importance, and Cross-model render for both Shigella species.
  • Protein names resolve 0 → 6,135 (sonnei).
  • Cross-model performance now varies by drug (AMP/CIP/TET distinct); flexneri shows an explanatory message instead of a blank panel, no crash.
  • Full test suite passes; lint clean.

@eboyer221 eboyer221 self-assigned this Jul 6, 2026

@amcim amcim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Three things to note:

  1. I got a note from R CMD CHECK
❯ checking R code for possible problems ... NOTE
  makeCrossModelPerformancePlot: no visible binding for global variable
    ‘.env’
  Undefined global functions or variables:
    .env

I added this to the global variables. The note is basically because .env looks undefined when you are telling filter to use the argument and not the column name.

  1. If its true that Shigella_sonnei/cluster_feature.parquet, then eventually we can put this in a data-raw folder in the root. Once features and data stop being added, that will be the time to make these decisions
  2. I confirmed that the created tarball is <10 MB (8.1). So things look good on this front and the issue is solved so long as no more is added.

Approved.

@eboyer221
eboyer221 merged commit 6f6fd63 into main Jul 10, 2026
8 checks passed
@eboyer221
eboyer221 deleted the dev-shrink-demo-data branch July 10, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants