Add specificity databases and immvoke annotate

.github/workflows/ci.yml

@@ -28,7 +28,11 @@ jobs:
       - name: Install package
         run: pip install .
       - name: Unit tests
-        run: python -m unittest discover -v
+        # Specificity keeps its tests beside its source (see pyproject.toml),
+        # where discovery from the top level does not reach.
+        run: |
+          python -m unittest discover -v
+          python -m unittest discover -v -s src/immvoke/Sources/Specificity/tests -t src
 
   python-versions:
     runs-on: ubuntu-latest
@@ -46,7 +50,11 @@ jobs:
           python -m pip install --upgrade pip
           pip install .
       - name: Unit tests
-        run: python -m unittest discover -v
+        # Specificity keeps its tests beside its source (see pyproject.toml),
+        # where discovery from the top level does not reach.
+        run: |
+          python -m unittest discover -v
+          python -m unittest discover -v -s src/immvoke/Sources/Specificity/tests -t src
 
   # Proves that the packaged schema snapshots and the runtime dependencies both
   # ship in the wheel. Running from /tmp guarantees the source tree cannot be

.github/workflows/release.yml

@@ -78,7 +78,11 @@ jobs:
           python -m pip install --upgrade pip build twine
           pip install .
       - name: Unit tests
-        run: python -m unittest discover -v
+        # Specificity keeps its tests beside its source (see pyproject.toml),
+        # where discovery from the top level does not reach.
+        run: |
+          python -m unittest discover -v
+          python -m unittest discover -v -s src/immvoke/Sources/Specificity/tests -t src
       - name: Build sdist and wheel
         run: python -m build
       - name: Check distribution metadata

NEWS.rst

@@ -1,6 +1,88 @@
 Release Notes
 ================================================================================
 
+Unreleased
+-------------------------------------------------------------------------------
+
+Specificity:
+
++ Added specificity databases IEDB_, CoV-AbDab_ and ASD_ under
+  ``immvoke specificity <db> search|download``. IEDB's BCR and TCR tables
+  are downloaded as schema-validated AIRR rearrangement TSVs with one record
+  per chain, linked by ``cell_id``, taking IEDB's calculated (IMGT
+  standardized) V/D/J-gene calls over the curated ones, every receptor
+  species kept unless ``--air_species`` names one (e.g. ``--air_species
+  "Homo sapiens"``; a species includes its strains, and a chain with no
+  species recorded is left out), and every assay
+  outcome unless ``--outcome`` names one (``Positive`` or ``Negative``).
+  IEDB's B-cell assay tables (``bcell``, ``bcr_to_bcell``) are not
+  offered: each receptor record already carries its epitope, assay ids and
+  outcome.
+  ``--antigen_species`` keeps receptors whose epitope comes from an antigen
+  organism, by any name IEDB uses, ignoring case (``--antigen_species
+  "Influenza A virus"`` takes in every flu strain); it asks IEDB's API for
+  the organisms of the table's epitopes when the table is downloaded.
++ ``immvoke specificity <db> search <table>`` counts what the table holds
+  once its filters are applied (rows, receptor groups, epitopes and the most
+  common loci, receptor, antigen species and structure types and outcomes,
+  ``--top`` of each), instead of
+  printing the download URL; ``--out`` writes every count as a TSV. Its and
+  download's filters are grouped in ``--help``, each saying what it filters
+  on, and ``--limit`` is left only on ``download all``, the one place it
+  had an effect.
++ IEDB's bcr and tcr tables gain ``iedb_antigen``,
+  ``iedb_antigen_source_species`` and ``iedb_antigen_structure_type``, each
+  record's epitope's antigen, its source species and the epitope's
+  structure type, looked up on IEDB's API at download time.
++ ``--antigen_structure_type`` keeps receptors whose epitope has that IEDB
+  structure type: ``Linear peptide``, ``Discontinuous peptide`` (which
+  includes ``Discontinuous peptide on multi chain``) or ``Non-peptidic``
+  (haptens, glycans, lipids...), IEDB's own Epitope Structure choice.
++ IEDB's bcr and tcr take ``--assay_type`` (e.g. ``neutralization``) and
+  ``--disease`` (e.g. ``COVID-19``; a disease includes those under it),
+  looked up on IEDB's assay API only when given, and ``--paired_only``,
+  keeping receptors with both chains of a pair.
++ IEDB's bulk ZIP is cached in ``~/.cache/immvoke/iedb/`` for a day, shared
+  by search and download, so ``bcr`` then ``tcr`` fetch it once.
++ Filter flags generated from a source's schema are spelled with
+  underscores, as the field is (``--air_species``), and so are
+  ``immvoke specificity ... download``'s own flags (``--dry_run``,
+  ``--no_resume``, ``--strict_airr``).
++ Each specificity download lands in its database's own folder,
+  ``<outdir>/<db>/``: the normalized ``<db>_<table>.tsv`` (e.g.
+  ``iedb/iedb_bcr.tsv``) and its own ``download_metadata.yml``; IEDB's
+  ``raw/`` (the bulk ZIP and what was taken from it) is deleted once the
+  tables are written.
++ Added ``immvoke annotate``, to annotate a query AIRR table against a
+  reference one (e.g. IEDB's BCR table) through symscan_: the heavy chain
+  CDR3, full VDJ or pooled CDR1-3 (``--sequence-region``), within the same
+  V/J-gene (``--partition-by``), exact or at an identity threshold
+  (``--identity-threshold``) under Hamming or Levenshtein distance
+  (``--distance-metric``), or heavy+light chain of the same antibody
+  (``--paired``, one chain of each kind per cell picked by
+  ``--count-field``).
++ ``immvoke annotate --outdir`` writes ``annotation.tsv`` and
+  ``annotation_metadata.yml`` (the options, inputs, reference database and
+  counts of the run), each named after the reference's database and table,
+  e.g. ``iedb_bcr_annotation.tsv``. The annotation keeps each query's best
+  hits, the highest identity (all of them on a tie): ``sequence_id``,
+  ``identity``, ``hit_id`` and that hit's ``antigen``, ``epitope`` and
+  ``antigen_source_species`` -- long rather than wide, one row per (query
+  row, matching reference row), since no separator is safe in IEDB's own
+  epitope names. A summary of the run in prose is written beside it as
+  ``annotation_summary.txt``.
+  ``--full-dist-matrix`` also writes ``annotation_full_matrix.txt``:
+  every pair the filters admit and how close it is (``sequence_id``,
+  ``hit_id``, ``identity``). ``--identity-threshold`` does not apply to it
+  -- it answers how close a query is to everything it was compared with,
+  not what it matched -- but ``--partition-by`` does, and under ``hamming``
+  only equal-length sequences have a distance at all.
+
+.. _IEDB: https://www.iedb.org/
+.. _CoV-AbDab: https://opig.stats.ox.ac.uk/webapps/covabdab/
+.. _ASD: https://naturalantibody.com/agab/
+.. _symscan: https://github.com/yutanagano/symscan
+
 Version 0.1.0: 2026.09.28
 -------------------------------------------------------------------------------
 

pyproject.toml

@@ -37,6 +37,17 @@ dependencies = [
     "pandas>=2.2.3",
     "airr>=2.0",
     "tqdm>=4.64",
+    # ASD's data unit is a Delta Lake of Parquet partitions, staged in a
+    # Google Drive folder: pyarrow reads the partitions, gdown fetches the
+    # folder, since it has no plain HTTP download. The floor is 16 for the
+    # same reason as the pandas floor above: pyarrow wheels before 16 were
+    # built against numpy 1.x and fail to import under numpy>=2.
+    "pyarrow>=16",
+    "gdown>=5.0",
+    # Symscan.py: annotate's matching engine for exact, Hamming and
+    # Levenshtein matching (a Rust symmetric-deletion index; numpy, which it
+    # returns its pairs in, comes with it and with pandas).
+    "symscan>=0.8",
 ]
 
 [project.optional-dependencies]
@@ -56,6 +67,11 @@ path = "src/immvoke/Version.py"
 # automatically as part of the package. Do not add a force-include for them: it
 # duplicates every file and the build fails.
 packages = ["src/immvoke"]
+# Specificity's own tests live alongside its source
+# (Sources/Specificity/tests/) rather than under the top level tests/
+# directory; exclude them here so a built wheel still ships no test code,
+# matching every other source under src/immvoke.
+exclude = ["src/immvoke/Sources/Specificity/tests/*"]
 
 [tool.ruff]
 line-length = 90

requirements.txt

@@ -4,3 +4,6 @@ PyYAML>=6.0
 pandas>=2.2.3
 airr>=2.0
 tqdm>=4.64
+pyarrow>=16
+gdown>=5.0
+symscan>=0.8