My name is Wes Floyd and I’m a Product Manager at Protocol Labs creators of IPFS, Filecoin) working on a new open source project, Bacalhau, to build a “lowest cost, open, and reproducible” compute platform for researchers. Think of it as a simple, collaborative way to do the “big data processing” part of your pipeline at much lower cost.
For example, we’re focused on helping migrate workloads like the EUREC4A project to IPFS/Filecoin for lowest cost storage and Bacalhau for lowest cost, open data pipeline processing. Here is a video overview of our mission
Can I get your feedback on whether this type of open compute platform would be helpful for the Pangeo (and Pangeo Forge) community? If so, we could start orienting our development efforts to serve as a backend, e.g. a Pangeo Forge Bakery (similar to AWS, Azure, and Google Cloud)