CMIP6 Zarr datasets on AWS — useful for interactive exploration?

[rabernat]

Thanks for this interesting and useful question @guigrpa – and welcome to the forum!

I’ll try to write a detailed response within a few days. In the meantime, this other post contains many points that are relevant to your question:

Best practices to go from 1000s of netcdf files to analyses on a HPC cluster?

What? 8759 netcdf files totalling 17TB of HYCOM ocean model (u,v) velocity data at two depth levels (and bottom velocity), at hourly time steps. So the total data arrays of interest are [9000 (X) by 7055 (Y) by 8759 (time) by 2 (Depth)] for both u and v; Using nco tools, I can reduce these files as an example to 365 netcdf files, totaling 4.3TB, one file per day (24 hourly steps), for one depth level only, so data arrays are 9000 (X) by 7055 (Y) by 8759 (time) for the u,v components. How? Us…