[Data Announcement] Revamped chunking for ERA5 Zarr datasets on Earth Data Hub
The Earth Data Hub just migrated all its ERA5 Zarr datasets to Zarr v3 and revamped their chunking along the way!
Here’s what’s changed:
Reduced chunk size: while each ERA5 dataset on Earth Data Hub retains its own tailored chunking, we have generally reduced the chunk size across the collection.
Better Regional Access: this new chunking schema drastically improves data access time for users dealing with regional maps and transient events (e.g., floods, heatwaves, etc.).
Balanced Performance: we carefully fine-tuned the chunk sizes to ensure you get faster regional slices without the overhead of making too many requests, i.e. long time series requests (30+ years) will not slow down.
Quick Reminder: the data within our ERA5-Zarr mirror is always available up to the last closed month and continues to be updated on a reliable monthly basis.
Try it out:
import xarray as xr
EDH_API_KEY = "your_EDH_api_key"
# can be found at: https://earthdatahub.destine.eu/account-settings
xr.open_dataset(
f"https://edh:{EDH_API_KEY}@api.earthdatahub.destine.eu/era5/era5-single-levels-atmosphere-v0.zarr",
chunks={},
engine="zarr",
)