[Draft] Atlanta scene-combination benchmark: run_benchmark.sh + study notebook (#169) - #198
Draft
bpurinton wants to merge 1 commit into
Draft
[Draft] Atlanta scene-combination benchmark: run_benchmark.sh + study notebook (#169)#198bpurinton wants to merge 1 commit into
bpurinton wants to merge 1 commit into
Conversation
The systematic half of #169: given five same-pass WorldView-2 scenes, which combinations and which processing flow give the best DEM. The six DEMs scored in the MVS notebook answer it for one triple; the full matrix needs the seven remaining pairs (all ten span 5-32 degrees of convergence), a nested quad for the 2->3->4->5 scene curve, a wide-spread triple, and dem_mosaic of all ten pairs and of the six well-converged ones. run_benchmark.sh produces exactly those, with the settings of the existing runs and every step skipped when its product exists. The left-image crop windows for nadir8/16/21 were derived from the bundle_adjust clean matches inside the nadir13 crop the way run_pairwise.sh derived nadir10's; the derivation reproduces nadir10's window exactly, which is the check on the method. The two dangling *_corr.tif symlinks left by the earlier cleanup are handled. The notebook is built to be valid at every stage: it scores whatever DEMs exist on disk with DEMBenchmark and lists the rest as pending, so as committed it covers the six existing DEMs (3 of 10 pairs, 3 of 4 chain points) and fills itself in once the runs land. No runs have been launched; that is the decision this branch waits on. No docs card or CHANGELOG line until the study is complete.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft, stacked on #197 (base
issue-169-dem-benchmark; GitHub will retarget it tomainwhen #197 merges). The processing script and the study notebook for the systematic part of #169 — no stereo runs have been launched; that is the one decision this PR is waiting on.What's in it
notebooks/WorldView/run_benchmark.sh— idempotent (every step skipped when its product exists), takes the data directory as its argument, identical stereo settings torun_mvs.sh/run_pairwise.sh, ASP bin appended to PATH. Produces, in order:wv_correct× 5 (the*_corr.tifintermediates were purged; two were dangling symlinks — handled).dem_mosaicof all ten pairs and of the six pairs > 15° (13-8, 8-16, 13-10, 8-21, 10-16, 10-21) — the test of whether the mosaic's problem is averaging, or averaging weak pairs in.{13,10,16,21}(pair 13-10 ⊂ MVS3 ⊂ quad ⊂ MVS5) and a wide-spread triple{13,8,21}.≈ 11–12 h on the laptop (pair ≈ 1 h, triple ≈ 2 h, quad ≈ 2.5 h), ~30 GB. The new left-image crop windows (nadir8
5461 9789 13354 12980, nadir166083 8756 12879 11461, nadir216467 8177 12495 10687) were derived from the BA clean matches inside the nadir13 crop exactly asCROP10was — and the method reproducesCROP10(5540 13223 13206 12931) bit for bit.notebooks/WorldView/worldview_spacenet_atlanta_benchmark.ipynb— the study, built so it is valid at every stage: it parsesba/run-convergence_angles.txtinto the pair table, reproduces the crop-window derivation, documents the script, then scores whatever DEMs exist on disk withDEMBenchmark(listing the rest as pending) and draws the three analyses — NMAD vs. convergence angle over the pairs, the 2→3→4→5 nested-scene curve, and the MVS-vs-mosaic flow table at 3 and 5 scenes. Executed as committed against the six existing DEMs, so the figures are partial (3 of 10 pairs, 3 of 4 chain points) by design.Not yet
multi_stereo --mode dem_mosaic(ASP's own pairwise+mosaic flow, unreleased) is not in the local 3.8.0-alpha build; if a newer daily build is available it would be a good extra row.To launch
Then re-execute the notebook; the pending lists empty themselves and the write-up gets finished.