Pangeo
Very big memory load when using fast parallel file system
HPC
Thomas_Moore
November 13, 2019, 5:17pm
12
Thanks for the reply
@TomAugspurger
- I’ll DM you.
show post in topic
Related topics
Topic
Replies
Views
Activity
Best practice advice on parallel processing a suite of zarr files with Dask and xarray
Data
10
248
June 18, 2025
Dask Execution Issue: Successful in Notebook, Fails in SLURM
HPC
6
406
August 1, 2023
Optimizing Dask worker memory for writing Zarr files from GeoTIFs
Data
7
250
September 7, 2024
Quartile Calculations on reshaped arrays - using dask + xarray + s3 and still having memory problems
Data
2
614
May 31, 2023
Xarray unable to allocate memory, How to "size up" problem
Data
location-uw
9
3069
July 27, 2023