Hi Pangeo community! I’m the product manager for the open source Frictionless Data project and I’m hoping to chat with some climate data researchers (and connect with Pangeo in general!). Sarah Gibson suggested that I reach out to y’all Frictionless Data creates open code and tooling for making research data easier to use (document metadata, validate data types, apply data standards), and we are seeking climate data collaborations. We’re hosting 2 brainstorming sessions next week where we’ll conceptualize ways of cleaning and standardizing open climate data, creating more reproducible and efficient methods of consuming and analyzing that data, and focus on ways to put this data into the hands of those that can truly drive change. Here’s more info (& sign up): Frictionless Planet – Save the Date – Open Knowledge Foundation blog Let me know if you have any questions or want to chat more!
Welcome to the forum Lilly! This sounds like a great effort that is highly aligned with our goals in Pangeo. “Cleaning and standardizing open climate data” and “reproducible and efficient methods for consuming and analyzing data” are two of the main activities this community has been focused on intensely for the past few years!
A summary of some of the best practices that we have developed was published in this paper:
Our newest project, Pangeo Forge, aims to operationalize some of the lessons learned in an open-source data pipeline service. That is described here:
I’ll mention your workshop at our weekly meeting today–or feel free to stop by yourself! Thanks again for sharing.
As maintainer of Intake, I have previously talked with Frictionless in the past, since the two projects have some overlap and may be able to work together or learn from one-another. Intake https://intake.readthedocs.io/ , is a python data catalog and access package and used in several places throughout pangeo.