Fix bilinear cache type decoding - #702
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #702 +/- ##
=======================================
Coverage 93.67% 93.67%
=======================================
Files 89 89
Lines 13621 13656 +35
=======================================
+ Hits 12759 12792 +33
- Misses 862 864 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
547f873 to
a5192ed
Compare
Manny7717
left a comment
There was a problem hiding this comment.
Review — Approved
Verified locally (shapely 2.1.2, zarr installed): test_save_and_load_bil_info passes, including the new mask_slices.dtype == bool assertion.
Root cause is right: zarr stores Python bools as uint8 without dtype metadata, so da.array(fid[val]) re-exposed the raw uint8. open_zarr(...).data goes through xarray's dtype decoding and restores bool. The removed import zarr is safe — the only remaining zarr references in the module are the zarr_out variable and xarray's Dataset.to_zarr() / open_zarr, which lazy-import zarr themselves.
Non-blocking note (inline): caches written by the previous version will still decode as uint8 masks under the new load path (no dtype marker was stored). Harmless in practice since caches regenerate, but a comment in the changelog or a cache-format version bump would save someone a confusing debug session.
| """Load bilinear resampling look-up tables and initialize the resampler.""" | ||
| try: | ||
| fid = zarr.open(filename, mode='r') | ||
| fid = open_zarr(filename, chunks="auto") |
There was a problem hiding this comment.
Minor: caches written before this change (raw uint8 bools via zarr.open) will still come back as uint8 under open_zarr, since no dtype marker was persisted. Caches regenerate on the next save, so it's benign — a changelog note (or cache version bump) would prevent future confusion.
Fixes bilinear cache type decoding:
mask_slices are boolean mask, and are stored as int8 via zarr.
boolean information is part of xarray metadata, so when skipping it we get back int8.
now mask_slices is not a mask but rather an array of 0, 1 indices making the mask useless.
I opted into chunks=auto automatic rechunking so Dask can pick the optimal chunk size, which gave a small performance boost on my data when using the cache: