AMS Town Hall - "Weather in the clouds: leveraging public clouds for scalable operational meteorology"

Hello everyone,

I’m hosting a Town Hall meeting at the American Meteorological Society’s Annual Meeting this year on Tuesday, January 14 entitled “Weather in the clouds: leveraging public clouds for scalable operational meteorology”. I’ve attached our extended abstract below this message, but the main thrust is that we’ve gotten together representatives from Google, Amazon, Microsoft, and NOAA for a community discussion about all aspects of operational meteorology on the cloud - from running different types of models and analyses on elastic compute to dealing with new data storage media and technologies.

One of our participants, Kevin Jorissen from Amazon, asked if we might be able to get a Pangeo community member to share some of the story of how Pangeo came to be and how it envisions the cloud to be transformative in the research community. We think that would be a nice book-end to the story that Irene Parker from NOAA/NESDIS is going to share as a panelist on what the potential of the cloud can really be for our community.

Is there anyone in the community attending the AMS who would be available to participate informally in the panel discussion? We can’t fit you on stage (we’re already packed), but what we were hoping to do is provide you dedicated time (up to 5 minutes) to come to the microphone in the room and share the Pangeo story and the impact that it has had on your research as a motivation for what the cloud can do. The Town Hall is at 12:15-1:15 PM on Tuesday, January 14.

Happy to chat offline with anyone who is interested!

Extended Abstract:

The advent of the commercial cloud has presented the broader Weather Enterprise with a unique opportunity - a chance to greatly reduce the cost and complexity of incubating, building, and operationalizing new forecast tools and data products to better serve the public. But fully realizing the potential afforded by this opportunity will require paradigm shifts: changes in how we collaborate (especially in how we transition research to operations), changes in who we collaborate with (including stakeholders and industries outside of our own), changes in how we educate and train our workforce, and changes in the tools of our trade, just to name a few. Understanding, anticipating, and acting on these changes will thus likely be a key theme the AMS will need to embrace as it enters its next century.

As the AMS’ centennial approaches, our community has already been wrestling with the challenges of the commercial cloud for several years. Early stakeholders have pioneered how to adapt new technologies to power big data workflows, and some have even grappled with the challenges of porting HPC applications such as numerical weather production to cloud-based platforms. An even larger community has already successfully leveraged the cloud to rapidly build and serve new commercial products at a fraction of the cost that prior technologies would permit. But despite these early successes, there are still many challenges with adopting the cloud, especially when it comes to porting legacy applications, and even more room for developing best practices for building new ones.

The panelists in our Town Hall include representatives from the private sector - including cloud vendors - as well as their public sector counterparts who are working together to embrace and solve these challenges in their applications from AI to EPIC. As they share their perspectives and experience on what trends in the cloud could be key enablers for the weather community, we invite the community to join us and help brainstorm what barriers might be preventing them from adopting the cloud for their own work. We will also feature a long, open Q&A session for the community and experts to come together, start a dialogue, and ultimately find potential workarounds or solutions to these barriers.

Please join us and help set the course for the cloud-optimized weather enterprise of the future!

2 Likes

This sounds like a good opportunity. I am planning to be at AMS for about 36 hours, arriving on the afternoon of the 14th, so I would not be able to attend the panel under my current itinerary.

If no one else is available to talk about Pangeo, I could look into changing my train ticket to get there earlier in the day.

Hi all, just a bump here. The invitation is open to any member of the community, no matter how junior or senior you might be.

It looks like there is no one else who can do this. But it’s also too late for me to change my train ticket. I leave from Penn station at 12:30pm.

A couple of ideas.

  • I could try to join remotely. I’ll have reasonable internet on the train. But of course this is risky.
  • I could pre-record a screencast or video.
  • I could write something and someone else could share it.

Hey Ryan, no worries. If you would be willing to share a paragraph or two or even a slide, I would be happy to present that to the Town Hall (I have an overview deck with a slide from each participant).

Ok, I’ll try to get you something by tomorrow morning.

@darothen - I just sent you some Google slides via email.

1 Like