Processing Terabyte-Scale NASA Cloud Datasets with Coiled

Cloud
1

Hi All,

I’ve been working with groups that process NASA data (folks like @asteiker @betolink @andypbarrett @jinbow) to help move their workloads to the cloud. It turns out a common pattern for lots of workloads is a big parallel for-loop.

Here’s a blogpost where we wrote about this analysis pattern and discussed cloud performance and cost Processing Terabyte-Scale NASA Cloud Data | Coiled (Spoiler: it’s surprisingly cheap to churn through a bunch of cloud data when using spot instances, the right region, etc.)

Hopefully this can serve as a useful template for others with similar use cases

3 Likes
2

Just to mention that we are having a community call tomorrow to talk about Coiled-powered workflows for cloud data Openscapes Community Call: NASA Earthdata Cloud with Coiled
This call is open and it would be great if Pangeo folks can attend and give their 2c on this approach.