# \#zarr

**URL:** https://discourse.pangeo.io/tag/zarr/44.md

[Latest](https://discourse.pangeo.io/latest.md) · [Categories](https://discourse.pangeo.io/categories.md) · [Tags](https://discourse.pangeo.io/tags.md)

---

## [Feedback wanted: recipe-based tool for building aligned radar/satellite/model/obs datasets (CONUS)](https://discourse.pangeo.io/t/feedback-wanted-recipe-based-tool-for-building-aligned-radar-satellite-model-obs-datasets-conus/5832)

<div class="topic-metadata">

**Author:** [@yuchen\_wei3667](https://discourse.pangeo.io/u/yuchen_wei3667)\
**Replies:** 0\
**Last updated:** [October 6, 2026, 1:43am UTC](https://discourse.pangeo.io/t/feedback-wanted-recipe-based-tool-for-building-aligned-radar-satellite-model-obs-datasets-conus/5832 "2026-10-06T01:43:58Z")

</div>

Hi all, I’m designing an open-source Python tool for building analysis-ready datasets that combine US radar, satellite, NWP, surface observations, and severe-weather reports, and I’d really value this community’s input …

---

## [Experiment: querying large Zarr datasets directly from PostgreSQL/PostGIS without ETL](https://discourse.pangeo.io/t/experiment-querying-large-zarr-datasets-directly-from-postgresql-postgis-without-etl/5809)

<div class="topic-metadata">

**Author:** [@HamzaMPSY](https://discourse.pangeo.io/u/HamzaMPSY)\
**Replies:** 3\
**Last updated:** [September 16, 2026, 6:55am UTC](https://discourse.pangeo.io/t/experiment-querying-large-zarr-datasets-directly-from-postgresql-postgis-without-etl/5809 "2026-09-16T06:55:35Z")

</div>

Experiment: querying Zarr directly from PostgreSQL/PostGIS I’ve been working on a Zarr FDW for PostgreSQL and I’d like to get feedback from people who work with real Zarr datasets. The idea came from a workflow I keep …

---

## [Public Virtual Zarr of the NASA (NEX-GDDP-CMIP6) Dataset](https://discourse.pangeo.io/t/public-virtual-zarr-of-the-nasa-nex-gddp-cmip6-dataset/5802)

<div class="topic-metadata">

**Author:** [@norlandrhagen](https://discourse.pangeo.io/u/norlandrhagen)\
**Replies:** 3\
**Last updated:** [August 24, 2026, 8:52am UTC](https://discourse.pangeo.io/t/public-virtual-zarr-of-the-nasa-nex-gddp-cmip6-dataset/5802 "2026-08-24T08:52:30Z")

</div>

In a recent project, I needed some downscaled CMIP6 data. I created a Virtual Zarr store of the NEX-GDDP-CMIP6 dataset. I wanted to share it if anyone has use of this dataset and doesn’t want to go through the work of re…

---

## [Virtual GRIB ingestion via Gribberish](https://discourse.pangeo.io/t/virtual-grib-ingestion-via-gribberish/5785)

<div class="topic-metadata">

**Author:** [@TomNicholas](https://discourse.pangeo.io/u/TomNicholas)\
**Replies:** 0\
**Last updated:** [July 15, 2026, 10:30pm UTC](https://discourse.pangeo.io/t/virtual-grib-ingestion-via-gribberish/5785 "2026-07-15T22:30:57Z")

</div>

Ever wonder why GRIB is the way it is? Read our new blog post on virtually ingesting GRIB data into Icechunk using Gribberish and VirtualiZarr!

---

## [\[Data Announcement\] Revamped chunking for ERA5 Zarr datasets on Earth Data Hub](https://discourse.pangeo.io/t/data-announcement-revamped-chunking-for-era5-zarr-datasets-on-earth-data-hub/5776)

<div class="topic-metadata">

**Author:** [@Nicola\_Masotti](https://discourse.pangeo.io/u/Nicola_Masotti)\
**Replies:** 0\
**Last updated:** [July 6, 2026, 3:34pm UTC](https://discourse.pangeo.io/t/data-announcement-revamped-chunking-for-era5-zarr-datasets-on-earth-data-hub/5776 "2026-07-06T15:34:28Z")

</div>

\[Data Announcement\] Revamped chunking for ERA5 Zarr datasets on Earth Data Hub The Earth Data Hub just migrated all its ERA5 Zarr datasets to Zarr v3 and revamped their chunking along the way! Here’s what’s changed: …

---

## [How to see all the public repos in arraylake programmatically?](https://discourse.pangeo.io/t/how-to-see-all-the-public-repos-in-arraylake-programmatically/5764)

<div class="topic-metadata">

**Author:** [@ahuang11](https://discourse.pangeo.io/u/ahuang11)\
**Replies:** 2\
**Last updated:** [June 26, 2026, 6:43pm UTC](https://discourse.pangeo.io/t/how-to-see-all-the-public-repos-in-arraylake-programmatically/5764 "2026-06-26T18:43:37Z")

</div>

In Old format, no problem!: Cloud-optimizing the GOES-16 archive as Virtual Zarr | Earthmover it shows how to easily access: import arraylake as al import xarray as xr client = al.Client() client.login() repo = client…

---

## [Zarr-layer: a typescript library for visualizing zarr & icechunk in the browser](https://discourse.pangeo.io/t/zarr-layer-a-typescript-library-for-visualizing-zarr-icechunk-in-the-browser/5574)

<div class="topic-metadata">

**Author:** [@norlandrhagen](https://discourse.pangeo.io/u/norlandrhagen)\
**Replies:** 0\
**Last updated:** [April 28, 2026, 6:43pm UTC](https://discourse.pangeo.io/t/zarr-layer-a-typescript-library-for-visualizing-zarr-icechunk-in-the-browser/5574 "2026-04-28T18:43:02Z")

</div>

Hey folks! Just wanted to share a library my co-worker Shane Loeffler has been working on. At CarbonPlan we’ve been using @carbonplan-maps for a while now for visualizing zarr directly in the browser. It works great, bu…

---

## [Insert a small zarr into a big zarr in time order](https://discourse.pangeo.io/t/insert-a-small-zarr-into-a-big-zarr-in-time-order/4526)

<div class="topic-metadata">

**Author:** [@Jiaming\_Tan](https://discourse.pangeo.io/u/Jiaming_Tan)\
**Replies:** 4\
**Last updated:** [April 28, 2026, 2:15pm UTC](https://discourse.pangeo.io/t/insert-a-small-zarr-into-a-big-zarr-in-time-order/4526 "2026-04-28T14:15:15Z")

</div>

I have a combined zarr named bigzarrno5.zarr，it has dimensions valid\_time, longitude, latitude, pressure\_level，and it contain day 1,2,3,4,6 and the valid\_time format is datatime64 like ‘2020-11-01T00:00:00.000000000’,eac…

---

## [Join us: Advanced Analysis with Sentinel GeoZarr (20 April) - EOPF Explorer + openEO](https://discourse.pangeo.io/t/join-us-advanced-analysis-with-sentinel-geozarr-20-april-eopf-explorer-openeo/5556)

<div class="topic-metadata">

**Author:** [@Sabrina](https://discourse.pangeo.io/u/Sabrina)\
**Replies:** 0\
**Last updated:** [April 7, 2026, 9:06am UTC](https://discourse.pangeo.io/t/join-us-advanced-analysis-with-sentinel-geozarr-20-april-eopf-explorer-openeo/5556 "2026-04-07T09:06:07Z")

</div>

Hello! Thank you again to everyone from the Pangeo community who joined the first EOPF Explorer webinar. Following the strong response from that session, we are organising a hands-on workshop that lets you try out analy…

---

## [Guidance on using Zarr on HPC (chunking, multi-file layouts, and node scaling for optimal CPU/memory use)](https://discourse.pangeo.io/t/guidance-on-using-zarr-on-hpc-chunking-multi-file-layouts-and-node-scaling-for-optimal-cpu-memory-use/5535)

<div class="topic-metadata">

**Author:** [@lakshmitharun](https://discourse.pangeo.io/u/lakshmitharun)\
**Replies:** 0\
**Last updated:** [March 15, 2026, 10:08am UTC](https://discourse.pangeo.io/t/guidance-on-using-zarr-on-hpc-chunking-multi-file-layouts-and-node-scaling-for-optimal-cpu-memory-use/5535 "2026-03-15T10:08:18Z")

</div>

Hi all, I’m working on an HPC workflow where we store large gridded datasets (SST-like, 3D arrays with dimensions similar to time × 3600 × 7200) in Zarr format on a parallel file system. I’m looking for any pointers to …

---

## [Community Notebook Competition - Sentinel data in Zarr](https://discourse.pangeo.io/t/community-notebook-competition-sentinel-data-in-zarr/5510)

<div class="topic-metadata">

**Author:** [@Sabrina](https://discourse.pangeo.io/u/Sabrina)\
**Replies:** 4\
**Last updated:** [February 21, 2026, 10:45am UTC](https://discourse.pangeo.io/t/community-notebook-competition-sentinel-data-in-zarr/5510 "2026-02-21T10:45:42Z")

</div>

Dear PANGEO community, ESA is planning to distribute Sentinel data in Zarr format in the future and wants to give users a chance to try it out first. Here’s how you can participate: Community Notebook Competition Wri…

---

## [Live Browsing Sentinel Zarr - EOPF Explorer Launch Webinar (9 March) - GeoZarr Conventions for Sentinels](https://discourse.pangeo.io/t/live-browsing-sentinel-zarr-eopf-explorer-launch-webinar-9-march-geozarr-conventions-for-sentinels/5515)

<div class="topic-metadata">

**Author:** [@j08lue](https://discourse.pangeo.io/u/j08lue)\
**Replies:** 1\
**Last updated:** [February 18, 2026, 1:58pm UTC](https://discourse.pangeo.io/t/live-browsing-sentinel-zarr-eopf-explorer-launch-webinar-9-march-geozarr-conventions-for-sentinels/5515 "2026-02-18T13:58:55Z")

</div>

Dear PANGEO Community, You’ve probably heard about EOPF before, ESA’s upcoming cloud-native product format for all Sentinel data. What’s new: ESA tasked us (DevSeed, EOX, thriveGEO and partners) to devise a visualisati…

---

## [Very slow zarr appends from non cf compliant hdf files](https://discourse.pangeo.io/t/very-slow-zarr-appends-from-non-cf-compliant-hdf-files/5484)

<div class="topic-metadata">

**Author:** [@Kaboom\_Official](https://discourse.pangeo.io/u/Kaboom_Official)\
**Replies:** 0\
**Last updated:** [January 12, 2026, 9:50pm UTC](https://discourse.pangeo.io/t/very-slow-zarr-appends-from-non-cf-compliant-hdf-files/5484 "2026-01-12T21:50:28Z")

</div>

Hey I am working with gpm dpr ku band data and data when read through xarray creates phony dims.The structure of hdf files is like this Dataset: AlgorithmRuntimeInfo, shape: (1,), dtype: |S1175 Group: FS Group: FS/CS…

---

## [Guidance on Correctly Serving Zarr Datasets Through pygeoapi](https://discourse.pangeo.io/t/guidance-on-correctly-serving-zarr-datasets-through-pygeoapi/5448)

<div class="topic-metadata">

**Author:** [@Kaboom\_Official](https://discourse.pangeo.io/u/Kaboom_Official)\
**Replies:** 0\
**Last updated:** [November 18, 2025, 7:47pm UTC](https://discourse.pangeo.io/t/guidance-on-correctly-serving-zarr-datasets-through-pygeoapi/5448 "2025-11-18T19:47:57Z")

</div>

Hello everyone. I’m trying to serve my data using pygeoapi, but I’m not able to get it working properly. I keep running into errors even after following the documentation closely. I tried loading the example Zarr file u…

---

## [Best Practices for Storing EURO-CORDEX CMIP6 Datasets on S3 (Zarr, Icechunk, Kerchunk)](https://discourse.pangeo.io/t/best-practices-for-storing-euro-cordex-cmip6-datasets-on-s3-zarr-icechunk-kerchunk/5394)

<div class="topic-metadata">

**Author:** [@larsbuntemeyer](https://discourse.pangeo.io/u/larsbuntemeyer)\
**Replies:** 5\
**Last updated:** [September 11, 2025, 2:27pm UTC](https://discourse.pangeo.io/t/best-practices-for-storing-euro-cordex-cmip6-datasets-on-s3-zarr-icechunk-kerchunk/5394 "2025-09-11T14:27:09Z")

</div>

Hi all, I’m preparing to upload new EURO-CORDEX CMIP6 datasets to our bucket on AWS S3 and would love some advice on the best storage layout. It has been some time since i was last dealing with this (mostly through pang…

---

## [Best way to structure satellite data in Zarr for time-based indexing and batching with DataLoader](https://discourse.pangeo.io/t/best-way-to-structure-satellite-data-in-zarr-for-time-based-indexing-and-batching-with-dataloader/5384)

<div class="topic-metadata">

**Author:** [@gdravilas](https://discourse.pangeo.io/u/gdravilas)\
**Replies:** 1\
**Last updated:** [September 6, 2025, 8:38pm UTC](https://discourse.pangeo.io/t/best-way-to-structure-satellite-data-in-zarr-for-time-based-indexing-and-batching-with-dataloader/5384 "2025-09-06T20:38:41Z")

</div>

Hi everyone, I ‘m working on a project with satellite data that spans across multiple timestamps, and I ‘d love some advice on structuring it efficiently. The data For each variable (satellite channels), I have several…

---

## [Benchmark Notebook for Xarray to zarr on S3](https://discourse.pangeo.io/t/benchmark-notebook-for-xarray-to-zarr-on-s3/5346)

<div class="topic-metadata">

**Author:** [@emfdavid](https://discourse.pangeo.io/u/emfdavid)\
**Replies:** 3\
**Last updated:** [August 23, 2025, 12:17am UTC](https://discourse.pangeo.io/t/benchmark-notebook-for-xarray-to-zarr-on-s3/5346 "2025-08-23T00:17:09Z")

</div>

I built a IO Benchmark Notebook to see how fast can we read/write chunked xarray datatsets to zarr on s3 from EC2 in 2025. I would love feedback and suggestions or alternate examples from the experts here! I am interest…

---

## [How to spatially concatenate (along latitude and longitude) multiple Kerchunk-referenced NetCDF files into a valid mosaic.json](https://discourse.pangeo.io/t/how-to-spatially-concatenate-along-latitude-and-longitude-multiple-kerchunk-referenced-netcdf-files-into-a-valid-mosaic-json/5231)

<div class="topic-metadata">

**Author:** [@Pouria\_Rezaei](https://discourse.pangeo.io/u/Pouria_Rezaei)\
**Replies:** 3\
**Last updated:** [July 25, 2025, 4:34pm UTC](https://discourse.pangeo.io/t/how-to-spatially-concatenate-along-latitude-and-longitude-multiple-kerchunk-referenced-netcdf-files-into-a-valid-mosaic-json/5231 "2025-07-25T16:34:01Z")

</div>

Hi, I’m working with a large dataset , each one covering a distinct spatial tile (no overlaps), but all sharing the same time axis. Each file has shape (281, 168, 168) corresponding to (time, latitude, longitude). Each…

---

## [Reading a Larger than RAM NetCDF4 using Xarray](https://discourse.pangeo.io/t/reading-a-larger-than-ram-netcdf4-using-xarray/5178)

<div class="topic-metadata">

**Author:** [@Ben\_Zeidell](https://discourse.pangeo.io/u/Ben_Zeidell)\
**Replies:** 7\
**Last updated:** [June 24, 2025, 4:48am UTC](https://discourse.pangeo.io/t/reading-a-larger-than-ram-netcdf4-using-xarray/5178 "2025-06-24T04:48:02Z")

</div>

Hey there Pangeo community, I’m dealing with an unwieldy 40GB NetCDF4 file that is a collection of various in-situ snow cover datasets. There are 10,118 stations reporting at varying temporal frequencies between 1979 an…

---

## [Using kerchunk's MultiZarrToZarr on a dataset with multiple vertical coordinates](https://discourse.pangeo.io/t/using-kerchunks-multizarrtozarr-on-a-dataset-with-multiple-vertical-coordinates/5188)

<div class="topic-metadata">

**Author:** [@cbaerzsw](https://discourse.pangeo.io/u/cbaerzsw)\
**Replies:** 1\
**Last updated:** [June 3, 2025, 11:41am UTC](https://discourse.pangeo.io/t/using-kerchunks-multizarrtozarr-on-a-dataset-with-multiple-vertical-coordinates/5188 "2025-06-03T11:41:11Z")

</div>

Hello community, I have a dataset of historical runs of the ICON-D2 configuration of the ICON model from German Weather Forecasting Agency (DWD). The dataset has dimensions (atmospheric quantity, latitude,longitude, ve…

---

## [Memory Error using xvec.zonal\_stats](https://discourse.pangeo.io/t/memory-error-using-xvec-zonal-stats/5151)

<div class="topic-metadata">

**Author:** [@mlummus](https://discourse.pangeo.io/u/mlummus)\
**Replies:** 1\
**Last updated:** [May 19, 2025, 9:45pm UTC](https://discourse.pangeo.io/t/memory-error-using-xvec-zonal-stats/5151 "2025-05-19T21:45:53Z")

</div>

Hello! I’m trying to use xvec zonal statistics to calculate the mean snow fraction for each month in a 20 year timespan in 934 hydrologic basin geometries. My workstation has 64 GB of memory. The main dataset I am using …

---

## [Slow append to existing s3 zarr store using xarray](https://discourse.pangeo.io/t/slow-append-to-existing-s3-zarr-store-using-xarray/5117)

<div class="topic-metadata">

**Author:** [@markmuetz](https://discourse.pangeo.io/u/markmuetz)\
**Replies:** 6\
**Last updated:** [May 13, 2025, 8:41pm UTC](https://discourse.pangeo.io/t/slow-append-to-existing-s3-zarr-store-using-xarray/5117 "2025-05-13T20:41:43Z")

</div>

I have a large dataset that has already been written, publicly available here: https://hackathon-o.s3-ext.jc.rl.ac.uk/sim-data/dev/v5/glm.n2560\_RAL3p3/um.PT1H.hp\_z10.zarr/. I would like to append two new fields to this …

---

## [Fundamentals: What is Cloud-Optimized Scientific Data?](https://discourse.pangeo.io/t/fundamentals-what-is-cloud-optimized-scientific-data/5064)

<div class="topic-metadata">

**Author:** [@TomNicholas](https://discourse.pangeo.io/u/TomNicholas)\
**Replies:** 0\
**Last updated:** [April 17, 2025, 8:43pm UTC](https://discourse.pangeo.io/t/fundamentals-what-is-cloud-optimized-scientific-data/5064 "2025-04-17T20:43:36Z")

</div>

I wrote the article about the cloud that I wish I could have read back when I first heard of Zarr and cloud-native science in 2018. I hope that people here find this useful, as a reference or as clarification of certa…

---

## [Proposal: Expanding the xstac python tool to automate a few more of the hard parts](https://discourse.pangeo.io/t/proposal-expanding-the-xstac-python-tool-to-automate-a-few-more-of-the-hard-parts/5025)

<div class="topic-metadata">

**Author:** [@KeynesYouDigIt](https://discourse.pangeo.io/u/KeynesYouDigIt)\
**Replies:** 2\
**Last updated:** [March 27, 2025, 2:06pm UTC](https://discourse.pangeo.io/t/proposal-expanding-the-xstac-python-tool-to-automate-a-few-more-of-the-hard-parts/5025 "2025-03-27T14:06:49Z")

</div>

Hi everyone! I am a developer/MLOPs person getting increasingly into the geospatial space and loving every second. I’ve done a little bit of work on pystac and am having a blast. I am a bit new, so please excuse the que…

---

## [Webinar: Data Version Control for Arrays with Icechunk on 2/18 @ 1 PM EST](https://discourse.pangeo.io/t/webinar-data-version-control-for-arrays-with-icechunk-on-2-18-1-pm-est/4900)

<div class="topic-metadata">

**Author:** [@ir028](https://discourse.pangeo.io/u/ir028)\
**Replies:** 0\
**Last updated:** [February 14, 2025, 5:45pm UTC](https://discourse.pangeo.io/t/webinar-data-version-control-for-arrays-with-icechunk-on-2-18-1-pm-est/4900 "2025-02-14T17:45:52Z")

</div>

Earthmover is hosting a webinar on Tuesday at 1 PM EST on Data Version Control for Arrays with Icechunk. Register here: https://share.hsforms.com/194OuRULJTnG2H0uxvAwJqgdl5u6 Icechunk is an open-source library and speci…

---

## [Best Practices for Reading and Handling Zarr Collections](https://discourse.pangeo.io/t/best-practices-for-reading-and-handling-zarr-collections/4822)

<div class="topic-metadata">

**Author:** [@RasterBlaster](https://discourse.pangeo.io/u/RasterBlaster)\
**Replies:** 0\
**Last updated:** [January 20, 2025, 8:48pm UTC](https://discourse.pangeo.io/t/best-practices-for-reading-and-handling-zarr-collections/4822 "2025-01-20T20:48:35Z")

</div>

Hello Pangeo Community, I’m currently working with Daymet Zarr collections (https://planetarycomputer.microsoft.com/api/stac/v1/collections/daymet-annual-pr) and exploring different ways to read and handle these dataset…

---

## [Zarr-Python 3 release](https://discourse.pangeo.io/t/zarr-python-3-release/4803)

<div class="topic-metadata">

**Author:** [@jhamman](https://discourse.pangeo.io/u/jhamman)\
**Replies:** 6\
**Last updated:** [January 10, 2025, 4:29pm UTC](https://discourse.pangeo.io/t/zarr-python-3-release/4803 "2025-01-10T16:29:53Z")

</div>

We released Zarr-Python 3 today, complete with support for the v3 spec, new extensions (chunk-sharding) and significant performance gains. Xarray v2025.01.1 was also released today with full support for Zarr-Python 3. …

---

## [Advice on modernizing a climate visualization web application for the Pangeo software stack](https://discourse.pangeo.io/t/advice-on-modernizing-a-climate-visualization-web-application-for-the-pangeo-software-stack/4789)

<div class="topic-metadata">

**Author:** [@jalder](https://discourse.pangeo.io/u/jalder)\
**Replies:** 2\
**Last updated:** [January 3, 2025, 10:33pm UTC](https://discourse.pangeo.io/t/advice-on-modernizing-a-climate-visualization-web-application-for-the-pangeo-software-stack/4789 "2025-01-03T22:33:52Z")

</div>

Hello Pangeo community, I developed and maintain a climate projection dashboard application (link) that I’m hoping to update in 2025 to the Pangeo toolset and modern formats. I’m primarily looking for suggestions on the …

---

## [Saving larger-than-memory objects to zarr using dask and xarray](https://discourse.pangeo.io/t/saving-larger-than-memory-objects-to-zarr-using-dask-and-xarray/4723)

<div class="topic-metadata">

**Author:** [@ks905383](https://discourse.pangeo.io/u/ks905383)\
**Replies:** 9\
**Last updated:** [December 3, 2024, 6:16pm UTC](https://discourse.pangeo.io/t/saving-larger-than-memory-objects-to-zarr-using-dask-and-xarray/4723 "2024-12-03T18:16:48Z")

</div>

Hey all, I’m trying to use dask to save a larger-than-memory zarr store. Concretely, I’m trying to: Concatenate several NetCDF files (on the same geo grid, same variables) by time Regrid them to a different grid Save …

---

## [Xarray operations (e.g., preprocess) running locally (post open\_mfdataset) instead of on Dask distributed cluster](https://discourse.pangeo.io/t/xarray-operations-e-g-preprocess-running-locally-post-open-mfdataset-instead-of-on-dask-distributed-cluster/4637)

<div class="topic-metadata">

**Author:** [@lbesnard](https://discourse.pangeo.io/u/lbesnard)\
**Replies:** 2\
**Last updated:** [November 13, 2024, 3:13am UTC](https://discourse.pangeo.io/t/xarray-operations-e-g-preprocess-running-locally-post-open-mfdataset-instead-of-on-dask-distributed-cluster/4637 "2024-11-13T03:13:01Z")

</div>

For reference, I’ve also asked the same question at and in the coiled slack page. I will update this discussion in case I have a positive answer from those places. I’m using dask.distributed and a coiled cluster to op…

[Next page](https://discourse.pangeo.io/tag/zarr/44.md?match_all_tags=true&page=1&tags%5B%5D=zarr)