Allow zipped zarr stores as an input for SpatialData

[pennycuda]

This PR goes with the following PR to SpatialData that adds the functionality to read zipped zarr stores in as SpatialData objects.
scverse/spatialdata#1141

This was motivated by HuBMAP Consortium's desire to reduce the number of files in our filesystem while maintaining our requirement that our output files be easily readable by our users. HuBMAP has already built other infrastructure to support zipped zarr stores for AnnData objects as well.

Please let me know what changes I need to make to get this PR merged; I am happy to work with you all!

[codecov]

Codecov Report

❌ Patch coverage is 48.14815% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.86%. Comparing base (66478bb) to head (445e918).

Files with missing lines	Patch %	Lines
ome_zarr/io.py	48.14%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #587      +/-   ##
==========================================
- Coverage   85.42%   84.86%   -0.56%     
==========================================
  Files          14       14              
  Lines        1921     1943      +22     
==========================================
+ Hits         1641     1649       +8     
- Misses        280      294      +14     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[will-moore]

Hi @pennycuda - thanks for the PR,

We are actually moving away from the io functionality in ome-zarr-py at the moment. There are a lot of issues such as the ones you're seeing when we try to wrap the zarr-python group & store creation, and that code shouldn't be needed anymore.

We have already stopped recommending the use of parse_url() for writing - instead you can pass a Zarr Group to write_image(data, group,...).

We are close to merging a new API for reading and writing (#515) which will mean we should be able to fully deprecate the usage of parse_url() and io.py.

It would be great if you could have a look at that PR and check that you can work with Zip stores - I don't think we've explicitly been testing that cc @jo-mueller (NB: it's easier to read the updated docs if you build them rather than reading the notebooks).

Thanks!

[pennycuda]

@will-moore Thank you for your response! I will check out that PR and see if it works with my SpatialData updates for zip stores then.

[pennycuda]

Hi again @will-moore ! I have a couple of updates from this morning. My colleague @keller-mark found a SpatialData PR that is much closer to being merged than mine but does not currently work with zipped stores because of it's ome-zarr-py IO usage. scverse/spatialdata#1107
I am now working off of this SpatialData branch, and I pulled in @jo-mueller branch from #515 for ome-zarr-py which also does not work with ZipStores as is. I will provide an update once I can get everything working together.

Thank you!
Penny

[jo-mueller]

Hi @pennycuda thanks a lot for working on this, reading from Zipstores is definitely a very much desired featured here. We'll merge #515 as soon as possible so that contributors have an easier time finding the right development head 👍

[melonora]

Hi @pennycuda, thanks for the PR. I would also like to point you to this: zarr-developers/zarr-python#3612. This allows users to move specific zarr groups to any destination store. Also, avoiding the whole decoding / encoding step (copying raw bytes). It took a while to get this in, but we would take this particular method into account for SpatialData in order to have it working as efficiently as possible.

This might be useful for when people are submitting data to HuBMAP.

[pennycuda]

Thanks @melonora for pointing me to that! I will look at those changes to see how they would pertain to my SpatialData changes. I'm closing this PR bc I just opened another one #587 for the new image class that was merged into ome-zarr-py yeterday. You might hear from me soon on another github thread. :)