DM-51163: Enable multi-tract matching#1286
Conversation
| doc="The type of sharding (spatial splitting) for the target catalog", | ||
| allowed={"tract": "Tract-based shards", "none": "No sharding at all"}, | ||
| default="tract", | ||
| ) |
There was a problem hiding this comment.
What is no sharding? Can this work with healpix?
There was a problem hiding this comment.
No sharding literally means no sharding at all - no spatial dimension, not even a skymap.
The reason I added this option is some catalogs are small enough that they are entirely within one tract (9813 for COSMOS, 5063 for ECDFS), so it seemed wasteful to have to reingest the same catalog multiple times for skymaps that often have identical tract definitions - especially so if you end up matching two such small external catalogs with each other. Having said that, in retrospect, the benefits aren't that great and I'm not sure if I want to recommend doing it this way.
In principle yes, one could add healpix support, but it would not be completely trivial.
This allows writing the matched catalog with a single task instead of deferring to a second task, avoiding unnecessary I/O.
2 participants
This also updates various configs to DRP v2 pipelines standards.