# Super large Zarr array limits

**URL:** <https://discourse.pangeo.io/t/super-large-zarr-array-limits/2034>\
**Category:** Uncategorized\
**Created:** [December 22, 2021, 8:19am UTC](https://discourse.pangeo.io/t/super-large-zarr-array-limits/2034 "2021-12-22T08:19:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Christophe\_Noel](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/christophe_noel/32/1327_2.png) [@Christophe\_Noel](https://discourse.pangeo.io/u/Christophe_Noel)\
**Post date:** [December 22, 2021, 8:20am UTC](https://discourse.pangeo.io/t/super-large-zarr-array-limits/2034/1 "2021-12-22T08:20:00Z")

</div>

Hello,

We plan to generate a super large data array (lat, lon, time) from the 56.000 Sentinel-2 tiles. The longitude variable would hold about 3 million coordinates and about 2 billion chunks (per year).

- What would be the metadata size for that ? Is there an explicit mapping to each chunk, or is it computed based on index?
- Coordinates size would be around 8 MB ?
- How fast would be a query resolved ? Is Zarr-python efficient with such large index ?

Thank you very much for your ideas.

---

<div class="post-metadata">

**Author:** ![raybellwaves](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/raybellwaves/32/782_2.png) [@raybellwaves](https://discourse.pangeo.io/u/raybellwaves)\
**Post date:** [December 26, 2021, 4:08am UTC](https://discourse.pangeo.io/t/super-large-zarr-array-limits/2034/2 "2021-12-26T04:08:38Z")

</div>

Would like to hear your experience.

An alternative to a large zarr (I.e. same store) is a directory full of zarrs (or netcdf) and working with [kerchunk/combine.py at main · fsspec/kerchunk · GitHub](https://github.com/fsspec/kerchunk/blob/main/kerchunk/combine.py#L16)