Zarr+xarray takes more time from day to day by writing with region parameter

Data
1

I have built up a monthy Zarr archive with a template of dimension 30x96x1200x800 .

And every day an Array of 1x96x1200x800 is written using to_zarr(region=).

Now I am observing an increasing time to write the same amount of data from day by day. After 2/3 of the 30 days , it takes twice as long as it took on the first day.

Does anybody know why this happens? And maybe has an explanation for that?

Thanks and best regards
Daniel

2

It has to read and rewrite the whole chunk every time you update because your chunksize 30x… Is bigger than the update size 1x…

2 Likes
3

Thanks for the answer. That really helped me to understand the issue.