Title: “Building Scalable Mosaic Workflows with Flyte/UnionAI, GDAL, and Xarray”
Invited Speaker: Len Strnad (ORCID: 0009-0009-8479-8235)
When: Wednesday, March 12, 2025 at 4 PM EST
Where: Launch Meeting - Zoom
Abstract:
We demonstrate the use of Flyte, deployed by UnionAI, as a workflow orchestration system for constructing large-scale Xarray mosaics using GDAL’s new GTI driver. We highlight the benefits of this orchestration system, including its integration with Kubernetes Dask, which seamlessly connects with Xarray and Zarr. Additionally, we explore the advantages of GDAL’s GTI driver and key configuration considerations. We present various parallelization strategies, offering insights into their effectiveness across different scenarios. Using GLAD’s ARD dataset—pre-tiled to EPSG:4326 globally and temporally stackable—we showcase how ingest and mosaic workflows can be combined to create an end-to-end Xarray dataset, ready for scientific computation.
Agenda:
- ~15 minutes - Showcase presentation
- 10 - 30 minutes - Discussion
- 15 - 30 minutes - Community check-in