Hey there, I’ve been working on a kerchunk cookbook and realized that I mention the use of fsspec quite a bit, but I don’t think there are any Pythia foundation materials or cookbooks on it.
Would an fsspec or general filesystem/cloud access cookbook or foundation text be useful for Pythia?
I’ve used fsspec a bit, but am by no means an expert, so I would love to help out on it, but not sure if I’m the right person to lead it.
Anyhow, just wondering if anyone else has been thinking of working on something similar.
2 Likes
This is a great idea. Each time I need to interact with the fsspec ecosystem I am able to find the docs for the individual components but I struggle when trying to understand how the parts come together. I’m usually highly dependent on responses gleaned from particular GitHub issues or Gitter discussions but those don’t have the persistent relevance of a foundational educational document.
And if you have a working knowledge of fsspec , especially from experience of working with those tools as a practitioner, than your foundation/cookboook would be very valuable!
Yes, please go ahead and start this. Since fsspec has use far beyond pangeo, there is an argument that pythia isn’t the best place for it, but maybe it doesn’t matter. Also very useful would be extra structure and overviews in the fsspec documentation itself, because very likely that’s where users will end up first.