Hi @geynard! Thanks for roping us in on this.
Ryan is basically right. The layout of our data is certainly an impediment to the sort of workflows that Pangeo has enabled in the Earth sciences, particularly in the cloud. We are tied to the FITS format for the foreseeable future and there has not been a ton of progress into understanding how this data format can be used with more scalable compute, either on HPC or in the cloud.
I cannot speak much for the nighttime astronomy community, but there is definitely interest in the solar physics community, especially among ECRs, in these types of workflows. If you’re looking for examples to add to a talk, I gave a poster on using Dask (and dask-jobqueue) to accelerate EUV image analysis on HPC (specifically NASA Pleiades). You can find the ADS entry here and a PDF of the poster here. A lot of these ideas were born out of the early work that Stuart and I, assisted by Matt and others, did with the demo Pangeo cluster a few years ago.
Also, I tried to click the link to your talk abstract but it is giving me a 404.