[AbhirupaGhosh]

Description

What kind of change(s) are included?

• Feature (adds or updates new capabilities)
• Bug fix (fixes an issue).
• Enhancement (adds functionality).
• Breaking change (these changes would cause existing functionality to not work as expected).

Checklist

Please ensure that all boxes are checked before indicating that this pull request is ready for review.

• I have read and followed the CONTRIBUTING.md guidelines.
• I have searched for existing content to ensure this is not a duplicate.
• I have performed a self-review of these additions (including spelling, grammar, and related).
• I have added comments to my code to help provide understanding.
• I have added a test which covers the code changes found within this PR.
• I have deleted all non-relevant text in this pull request template.
• Reviewer assignment: Tag a relevant team member to review and approve the changes.

[eboyer221]

I pushed a set of changes to adapt the dashboard to the new amRml parquet
schema and to clean up a couple of data-file issues.

Note on the data: these parquets are new upstream results, not just renamed versions of the old demo data - e.g. the flexneri baseline went from 69 → 922 rows, picked up a couple more drug classes, and reports raw mcc instead of normalized nmcc. This will warrant consideration before the eventual AMR_legacy deployment merge.

Code changes (schema adaptation)

• Species discovery fix: listAmRmlSpeciesFolders was excluding the
  baseline all_perf.parquet from its own "has a baseline?" check, so no
  species folders were ever found. Dropped all from the exclusion regex.
• Per-species metadata: files are no longer prefixed by species code, so
  get_metadata_path() / available_metadata_species now address metadata by
  species directory ({root}/{Species}/metadata.parquet) instead of a
  filename prefix.
• Metric nmcc → mcc : the new perf files ship raw MCC ([-1, 1], 0 =
  random) rather than normalized nMCC ([0, 1]). Updated the Model-Performance
  plots/axes/labels and the reference line to match.
• Cross-tested models : these now live in their own cross_*_perf files
  rather than a cross_test column, so the loader tags cross_test from the
  source filename and guarantees the columns the cross-model views expect.

Data cleanup

• Removed the old Sfl_*_ML_*.parquet files : the new _perf glob also matched
  them, which double-loaded every row.
• Removed cluster_feature.parquet (5.2 MB) : it isn't referenced anywhere in
  the code; the annotation tab uses cluster_feature_COG.parquet (the
  subset from PR Subsetting bundled cluster/cog annotations to reduce file and package size #31).

I confirmed the full test suite passes.

Leaving this PR open for now rather than merging, until we meet (@amcim @AbhirupaGhosh) this afternoon (06/24/26) to discuss some data size concerns:

• all_top_features.parquet is 12.3 MB. It is used (feature importance),
  but it's far larger than the old equivalent. Likely candidate to subset or move to ExperimentHub.
• The annotation tab needs a derived file that new species don't have. The
  tab reads cluster_feature_COG.parquet which is a trimmed, COG-annotated version of the raw cluster_feature.parquet that was generated manually in PR Subsetting bundled cluster/cog annotations to reduce file and package size #31. flexneri still carries that old derived file, but sonnei (Add Shigella sonnei example files #37) only ships the raw cluster_feature.parquet, so its annotation tab will be empty. We need to decide who/what regenerates the COG file per species (relevant to the EH plan).
• Cross-model tab shows only self-evaluation ("Same") for flexneri since it has
  no cross_* files; full cross behavior was verified against sonnei's
  cross_drug_perf.parquet.

Holding off on merge until we talk through the file-size / EH plan.

[AbhirupaGhosh]

We need to change the code to map all features to clusters instead of only COGs. The file cluster_feature.parquet is required and cluster_feature_COG.parquet needs to go.

[eboyer221]

Looks great! Tested, all checks passed. Approving.