Best practices to go from 1000s of netcdf files to analyses on a HPC cluster?

[kmader]

Incredible answer and phenomenal detailed explanation! I wish someone would have done this in detail years ago it would have saved a lot of spark, dask, hadoop headaches.