@martindurant has proposed a simple ReferenceFileSystem JSON format where you can specify the URL for each chunk. So you don’t need to write a giant NetCDF file. A collection will work nicely.
This opens up exciting possibilities.
One is creating many different virtual datasets from the same collection of files. For example, time series at specific forecast taus from a collection of overlapping forecast data.
Another is switching the file type from Zarr to TileDB and the user accessing the ReferenceFileSystem wouldn’t even know!
So using the ReferenceFileSystem approach not only gives us the opportunity to access files like NetCDF and GRIB2 in a cloud-optimized way, but like Intake, it provides a layer of abstraction on top that could shield users from format changes below the hood.