We’ve created a new “Cloud HPC” Category here on the Pangeo discourse to discuss setting up and running HPC applications like Met/Ocean/Hydro models on the Cloud.
With the advent of faster networking, disk and larger instance types, the Cloud is becoming a viable contender for many simulations that previously we needed to run at HPC facilities.
At USGS, we started exploring running our Coupled Ocean Atmosphere Wave and Sediment Transport (COAWST) modeling system on AWS after reading the blog post A Scientist’s Guide to Cloud-HPC: Example with AWS ParallelCluster, Slurm, Spack, and WRF in March 2019.
Then in September 2019, Danny Arevalo gave a great talk showing that NAVGEM on AWS could outperform the Navy DoD Supercomputing Resource Center machine they were using.
That convinced us to try again, and we discovered that AWS Parallel Cluster had significantly advanced and HPC deployment solutions other than Jiawei’s recipe were appearing:
- AWS No-Tears Cluster (@zflamig tipped us off on that!)
- IOOS Cloud Sandbox
- Compute Canada’s Magic Castle
Then we saw this post about running coastal ocean models on AWS at the Water Institute, and we had a great chat with @zcobell, after which we decided to start this new topic.
If you are working in this area, please introduce yourself, tell us a bit about your project, experiences and anything else you think would help us!