Skip to content

What if all validation records were in .jsonl files?! #548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yarikoptic wants to merge 1 commit into from
Closed

What if all validation records were in .jsonl files?! #548

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions 2d_mb_pcasl/derivatives/dandi-validator/duct_info.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"command": "dandi validate -o 2d_mb_pcasl/derivatives/dandi-validator//output.jsonl 2d_mb_pcasl", "system": {"cpu_total": 20, "memory_total": 67077734400, "hostname": "bilena", "uid": 47521, "user": "yoh"}, "env": {}, "gpu": [{"index": "0", "name": "NVIDIA RTX A5500 Laptop GPU", "bus_id": "00000000:01:00.0", "driver_version": "590.48.01", "memory.total": "16384 MiB", "compute_mode": "Default"}], "duct_version": "0.19.0", "schema_version": "0.2.2", "execution_summary": {"exit_code": 1, "command": "dandi validate -o 2d_mb_pcasl/derivatives/dandi-validator//output.jsonl 2d_mb_pcasl", "logs_prefix": "2d_mb_pcasl/derivatives/dandi-validator//duct_", "wall_clock_time": 1.8002088069915771, "peak_rss": 154517504, "average_rss": 84203520.0, "peak_vsz": 1081970688, "average_vsz": 550584320.0, "peak_pmem": 0.2, "average_pmem": 0.1, "peak_pcpu": 232.0, "average_pcpu": 116.0, "num_samples": 2, "num_reports": 2, "start_time": 1773957885.2293978, "end_time": 1773957887.0296066, "working_directory": "/home/yoh/proj/bids/bids-examples"}, "output_paths": {"stdout": "2d_mb_pcasl/derivatives/dandi-validator//duct_stdout", "stderr": "2d_mb_pcasl/derivatives/dandi-validator//duct_stderr", "usage": "2d_mb_pcasl/derivatives/dandi-validator//duct_usage.jsonl", "info": "2d_mb_pcasl/derivatives/dandi-validator//duct_info.json", "prefix": "2d_mb_pcasl/derivatives/dandi-validator//duct_"}, "working_directory": "/home/yoh/proj/bids/bids-examples", "message": ""}
6 changes: 6 additions & 0 deletions 2d_mb_pcasl/derivatives/dandi-validator/duct_stderr
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
2026-03-19 18:04:46,241 [ WARNING] No BIDS reference root provided.
2026-03-19 18:04:46,241 [ INFO] No schema path specified, defaulting to the bundled schema, `/home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/lib/python3.13/site-packages/bidsschematools/data/schema.json`.
2026-03-19 18:04:46,345 [ INFO] SUCCESS: All files are BIDS valid and no BIDS-required files are missing.
2026-03-19 18:04:46,345 [ INFO] No schema path specified, defaulting to the bundled schema, `/home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/lib/python3.13/site-packages/bidsschematools/data/schema.json`.
2026-03-19 18:04:46,889 [ INFO] Validation output written to 2d_mb_pcasl/derivatives/dandi-validator//output.jsonl
2026-03-19 18:04:46,889 [ INFO] Logs saved in /home/yoh/.local/state/dandi-cli/log/2026.03.19-22.04.45Z-917583.log
Empty file added 2d_mb_pcasl/derivatives/dandi-validator/duct_stdout
View file
Open in desktop
Empty file.
2 changes: 2 additions & 0 deletions 2d_mb_pcasl/derivatives/dandi-validator/duct_usage.jsonl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"timestamp": "2026-03-19T18:04:45.268950-04:00", "num_samples": 1, "processes": {"917583": {"pcpu": 0.0, "pmem": 0.0, "rss": 13889536, "vsz": 19197952, "timestamp": "2026-03-19T18:04:45.268950-04:00", "etime": "00:00", "stat": {"Rs": 1}, "cmd": "/home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/bin/python3 /home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/bin/dandi validate -o 2d_mb_pcasl/derivatives/dandi-validator//output.jsonl 2d_mb_pcasl"}}, "totals": {"pmem": 0.0, "pcpu": 0.0, "rss": 13889536, "vsz": 19197952}, "averages": {"rss": 13889536, "vsz": 19197952, "pmem": 0.0, "pcpu": 0.0, "num_samples": 1}}
{"timestamp": "2026-03-19T18:04:46.306512-04:00", "num_samples": 1, "processes": {"917583": {"pcpu": 232.0, "pmem": 0.2, "rss": 154517504, "vsz": 1081970688, "timestamp": "2026-03-19T18:04:46.306512-04:00", "etime": "00:01", "stat": {"Rsl": 1}, "cmd": "/home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/bin/python3 /home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/bin/dandi validate -o 2d_mb_pcasl/derivatives/dandi-validator//output.jsonl 2d_mb_pcasl"}}, "totals": {"pmem": 0.2, "pcpu": 232.0, "rss": 154517504, "vsz": 1081970688}, "averages": {"rss": 154517504, "vsz": 1081970688, "pmem": 0.2, "pcpu": 232.0, "num_samples": 1}}
58 changes: 58 additions & 0 deletions 2d_mb_pcasl/derivatives/dandi-validator/output.jsonl
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions 7t_trt/derivatives/dandi-validator/duct_info.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"command": "dandi validate -o 7t_trt/derivatives/dandi-validator//output.jsonl 7t_trt", "system": {"cpu_total": 20, "memory_total": 67077734400, "hostname": "bilena", "uid": 47521, "user": "yoh"}, "env": {}, "gpu": [{"index": "0", "name": "NVIDIA RTX A5500 Laptop GPU", "bus_id": "00000000:01:00.0", "driver_version": "590.48.01", "memory.total": "16384 MiB", "compute_mode": "Default"}], "duct_version": "0.19.0", "schema_version": "0.2.2", "execution_summary": {"exit_code": 1, "command": "dandi validate -o 7t_trt/derivatives/dandi-validator//output.jsonl 7t_trt", "logs_prefix": "7t_trt/derivatives/dandi-validator//duct_", "wall_clock_time": 5.447799205780029, "peak_rss": 464748544, "average_rss": 207805781.33333334, "peak_vsz": 25539035136, "average_vsz": 5005863594.666666, "peak_pmem": 0.6000000000000001, "average_pmem": 0.25, "peak_pcpu": 243.0, "average_pcpu": 173.48333333333335, "num_samples": 6, "num_reports": 2, "start_time": 1773957887.305338, "end_time": 1773957892.753137, "working_directory": "/home/yoh/proj/bids/bids-examples"}, "output_paths": {"stdout": "7t_trt/derivatives/dandi-validator//duct_stdout", "stderr": "7t_trt/derivatives/dandi-validator//duct_stderr", "usage": "7t_trt/derivatives/dandi-validator//duct_usage.jsonl", "info": "7t_trt/derivatives/dandi-validator//duct_info.json", "prefix": "7t_trt/derivatives/dandi-validator//duct_"}, "working_directory": "/home/yoh/proj/bids/bids-examples", "message": ""}
6 changes: 6 additions & 0 deletions 7t_trt/derivatives/dandi-validator/duct_stderr
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
2026-03-19 18:04:48,317 [ WARNING] No BIDS reference root provided.
2026-03-19 18:04:48,318 [ INFO] No schema path specified, defaulting to the bundled schema, `/home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/lib/python3.13/site-packages/bidsschematools/data/schema.json`.
2026-03-19 18:04:49,462 [ INFO] SUCCESS: All files are BIDS valid and no BIDS-required files are missing.
2026-03-19 18:04:49,463 [ INFO] No schema path specified, defaulting to the bundled schema, `/home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/lib/python3.13/site-packages/bidsschematools/data/schema.json`.
2026-03-19 18:04:52,588 [ INFO] Validation output written to 7t_trt/derivatives/dandi-validator//output.jsonl
2026-03-19 18:04:52,588 [ INFO] Logs saved in /home/yoh/.local/state/dandi-cli/log/2026.03.19-22.04.47Z-917716.log
Empty file added 7t_trt/derivatives/dandi-validator/duct_stdout
View file
Open in desktop
Empty file.
2 changes: 2 additions & 0 deletions 7t_trt/derivatives/dandi-validator/duct_usage.jsonl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"timestamp": "2026-03-19T18:04:47.335094-04:00", "num_samples": 1, "processes": {"917716": {"pcpu": 200.0, "pmem": 0.0, "rss": 12222464, "vsz": 17739776, "timestamp": "2026-03-19T18:04:47.335094-04:00", "etime": "00:00", "stat": {"Rs": 1}, "cmd": "/home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/bin/python3 /home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/bin/dandi validate -o 7t_trt/derivatives/dandi-validator//output.jsonl 7t_trt"}}, "totals": {"pmem": 0.0, "pcpu": 200.0, "rss": 12222464, "vsz": 17739776}, "averages": {"rss": 12222464, "vsz": 17739776, "pmem": 0.0, "pcpu": 200.0, "num_samples": 1}}
{"timestamp": "2026-03-19T18:04:52.528377-04:00", "num_samples": 5, "processes": {"918224": {"pcpu": 120.0, "pmem": 0.4, "rss": 280940544, "vsz": 24428642304, "timestamp": "2026-03-19T18:04:50.454275-04:00", "etime": "00:00", "stat": {"Rl": 1}, "cmd": "bids-validator-deno --json --outfile /home/yoh/.tmp/tmplfmha8pm/out.json /home/yoh/proj/bids/bids-examples/7t_trt"}, "917716": {"pcpu": 243.0, "pmem": 0.3, "rss": 230723584, "vsz": 1158295552, "timestamp": "2026-03-19T18:04:52.528377-04:00", "etime": "00:05", "stat": {"Rsl": 4, "Ssl": 1}, "cmd": "/home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/bin/python3 /home/yoh/proj/dandi/dandi-cli-enh-validators/.venv/bin/dandi validate -o 7t_trt/derivatives/dandi-validator//output.jsonl 7t_trt"}}, "totals": {"pmem": 0.6000000000000001, "pcpu": 243.0, "rss": 464748544, "vsz": 25539035136}, "averages": {"rss": 246922444.8, "vsz": 6003488358.4, "pmem": 0.30000000000000004, "pcpu": 168.18, "num_samples": 5}}
Loading
Loading