Added values from NF Portal Dataset schema to MC2 DataCatalog valid v… - #221
Merged
Conversation
update Parent, validation rules, add columnType for all attributes
Added CDE Public IDs for matches identified based on an old mapping to the CCDI model. Removed some redundant attributes and corrected a few typos
Add "Pending Annotation" as a valid value
While Imaging Level 3 Image files are composed of Imaging Level 2 images, it is possible for these to be shared independently, so Imaging Level 3 Image files should have the same metadata fields available as Imaging Level 2
Temporarily removed string_list from columnType (replaced with string), since schematic is not yet able to handle this value
schematic v25.8.1
Generated by: pip freeze > requirements.txt
aditya-nath-sage
requested review from
Bankso and
aditigopalan
as code owners
September 16, 2025 18:59
Bankso
force-pushed
the
data-catalog-model-nf-harmonization
branch
from
September 16, 2025 19:30
4d01767 to
544f0e7
Compare
Bankso
added
major
Valid Synapse JSON schema (registered: https://repo-prod.prod.sagebase.org//repo/v1/schema/type/registered/MC2Center-DataCatalog-12.0.8) Modified from original schematic output (removed minItems, added types for untyped attributes, added descriptions)
aditigopalan
left a comment
aditigopalan
left a comment
Contributor
There was a problem hiding this comment.
Great work! I have some comments, mostly that a more comprehensive PR description would be helpful:
- Is this documented anywhere? For terms added; include a brief mapping note (source term, rationale) in modules/dataCatalog/annotationProperty.csv or a PR comment.
- For “not added” NF terms (dataType, diseaseFocus, includedInDataCatalog, individualCount, manifestation, studyID), document the rationale or open follow-ups.
- Is this a breaking change? How does this change any templates?
- Idk how documentation works here, but add/adjust docs:
docs/model/dataset.mdor a newdocs/model/datacatalog.md
aditigopalan
left a comment
aditigopalan
left a comment
Contributor
There was a problem hiding this comment.
Do docs exist already for using the QC model?
vpchung
approved these changes
Oct 30, 2025
vpchung
left a comment
vpchung
left a comment
Member
There was a problem hiding this comment.
Found two instances where the columnType colname was not used, not sure if that would impact downstream processes, so just highlighted them for just in case!
Co-authored-by: Verena Chung <9377970+vpchung@users.noreply.github.com>
Co-authored-by: Verena Chung <9377970+vpchung@users.noreply.github.com>
A temporary fix to remove string_list as columnType, since this is not yet supported.
Contributor
@aditigopalan yep, QC model usage is described during step 2 here: https://sagebionetworks.jira.com/wiki/spaces/CCKPD/pages/4000776229/INTERNAL+Portal+Database+Syncing |
Clashes with number, the only numeric validation rule currently allowed by schematic
Bankso
approved these changes
Dec 9, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on the NF Portal Dataset schema (https://github.com/nf-osi/nf-metadata-dictionary/blob/main/registered-json-schemas/PortalDataset.json), I updated annotationProperty.csv for the DataCatalog schema for MC2 with missing terms in the NF model. Terms I didn't dd that were in the NF schema:
related to #136