Reading and Analyzing Mission HDF5 Files in the Cloud
We are excited to have JP Swinski and Rachel Wegener present at the next ESIP Cloud Computing Cluster meeting on HDF5 Files in the Cloud.
Topic: Reading and Analyzing Mission HDF5 Files in the Cloud
When: Monday August 28th, 10:00-11:00 am PT / 1:00-2:00 pm ET / 7pm CEST
Where: Find joining information on ESIP Community Calendar
Speakers: JP Swinski (Goddard Space Flight Center) and Rachel Wegener (PhD student at University of Maryland)
Abstract
HDF5 is a flexible scientific data format which is used for many high-value Earth observation datasets, such as the ICESat-2 standard data products. However, the HDF5 library and tools were designed for local file systems, and to date have not performed well in cloud environments. The ICESat-2 program funded the development of an on-demand data service called SlideRule. SlideRule needed performant access to HDF5 files in S3, and in response, developed the HDF5 Cloud-Optimized Read-Only Library (h5coro). JP will present the key features of h5coro that enable performant cloud access to HDF5 data. Rachel Wegener will present an in progress effort to integrate h5coro as an xarray backend, and will discuss the need for more robust data formats for Icesat-2 point cloud data.
- 5-10 minutes - Announcements
- 30 minutes - Presentation
- 20 minutes - Q@A
Bonus Round
If you are available to stick around after the H5 in the Cloud session to talk about your takeaways from the summer meeting or ideas for the cluster, please join us in doing so.