Title: “Cubed: Bounded-Memory Serverless Array Processing in Xarray”
Invited Speaker: Tom Nicholas (ORCID: 0000-0002-2176-0530), Tom White
When: Wednesday, Nov 15, 12PM EST
Where: Launch Meeting - Zoom
Abstract:
Cubed is an alternative framework for distributed processing of large arrays. Designed to respect memory constraints at all times, Cubed generalises the model of Rechunker to execute any numpy-like array operation. By using Zarr as persistent storage between steps, and Lithops as an abstraction layer, Cubed can run in a serverless fashion on a range of Cloud platforms. We will show how Cubed has been integrated with Xarray and demonstrate its performance on a tricky array workload.
- 20 minutes - Community Showcase
- 40 minutes - Showcase discussion/Community check-ins