Announcing the new "Cloud HPC" Category!

We’ve created a new “Cloud HPC” Category here on the Pangeo discourse to discuss setting up and running HPC applications like Met/Ocean/Hydro models on the Cloud.

With the advent of faster networking, disk and larger instance types, the Cloud is becoming a viable contender for many simulations that previously we needed to run at HPC facilities.

At USGS, we started exploring running our Coupled Ocean Atmosphere Wave and Sediment Transport (COAWST) modeling system on AWS after reading the blog post A Scientist’s Guide to Cloud-HPC: Example with AWS ParallelCluster, Slurm, Spack, and WRF in March 2019.

Then in September 2019, Danny Arevalo gave a great talk showing that NAVGEM on AWS could outperform the Navy DoD Supercomputing Resource Center machine they were using.

That convinced us to try again, and we discovered that AWS Parallel Cluster had significantly advanced and HPC deployment solutions other than Jiawei’s recipe were appearing:

Then we saw this post about running coastal ocean models on AWS at the Water Institute, and we had a great chat with @zcobell, after which we decided to start this new topic.

If you are working in this area, please introduce yourself, tell us a bit about your project, experiences and anything else you think would help us!

Hi all -

Happy to have joined this community. I work at The Water Institute of the Gulf, which is a nonprofit focused on coastal research and challenges facing coastal communities. My project work typically consists of numerical modeling of hurricane storm surge and waves as well as other hydrodynamic simulations of mostly larger scales (Atlantic ocean to global scale). My main project focus is on the Louisiana Coastal Master Plan (the link above) and I lead the storm surge modeling team. I spend a lot of my time both doing both project work as well as serving as the maintainer for the ADCIRC numerical model. I typically spend my time programming in C++ and Fortran, but I’ve been known to write a python script or two.

In a rough sense, my interests lie in providing both accurate and efficient simulations in both feasibility/design analysis settings for clients as well as improving the speed and efficiency of the underlying numerical models through source code development and examining new technologies, and determining the proper way to apply them for our workflows, which lately has included a lot of conversations around cloud computing.

Looking forward to getting to know this group and learning from your experiences.

Great to see this initiative, @rsignell!

I think it would be a great idea to reach out to Wyatt Gorman and Joseph Schoonover to learn more about similar efforts taking place on Google Cloud Platform. They recently shared this awesome Codelabs notebook automating the process of spinning up the WRF CONUS/12km benchmark simulation on Google’s HPC offering → Run the WRF Weather Forecasting Model with Fluid Numerics' Slurm-GCP

1 Like