In a recent project, I needed some downscaled CMIP6 data. I created a Virtual Zarr store of the NEX-GDDP-CMIP6 dataset. I wanted to share it if anyone has use of this dataset and doesn’t want to go through the work of re-virtualizing it.
With VirtualiZarr, we can take 38TB of NetCDF files and make a virtual Icechunk datatree that is under a GB (740MB).
Hi, not sure if this is of any interest to you, but we also zarrified a bunch of CMIP6 datasets. As opposed to using VirtualiZarr , these are native Zarrs (we did it the hard way). You can find them at CMIP6 - Earth Data Hub
We are considering to do more.
Hey @Nicola_Masotti, super cool to see people making CMIP easier to use!
Any reason why you preferred native Zarr?
In our case we wanted to fine tune the read performance of the result, thus wee needed to fully control the chunking. Considering the relatively low complexity of the CMIP6 daily data we converted ou to now, I do agree that also using VirtualiZarr could have been an option 