First we would need to see what the chunking of ds
and ds_mean
are before rechunking
ds_mean = ds_mean.chunk(dict(time=31411, lat=5, lon=5, scenario=-1))
Then, I would try “p2p rechunking” and report here: Share your experiences with P2P shuffling · dask/distributed · Discussion #7509 · GitHub or in that Discourse thread.