FYI these kind of frustrations with dask are what motivated development of Cubed, which aims to be transparent and predictable about per-worker runtime memory usage.
3 Likes
FYI these kind of frustrations with dask are what motivated development of Cubed, which aims to be transparent and predictable about per-worker runtime memory usage.