I’m new to this topic, and am very glad to see focus on global grids.
I’m coming from the lidar space, so I’m more interested in the 3D aspect than rasters and am curious where things have gone since the code sprint.
Is there much thought going into the 3D aspect of grids?
Most of the work I’ve seen focuses on the surface directly, and discretizing into a 2D grid across the surface.
I’m not as technical as the group here, but understand the concepts, and would be grateful for a pointer in the ‘best’ direction for a distortion-free 3D DGGS which would be optimal for Spatio-Temporal 3D datasets.
Thanks for all your work! Looking good so far, especially for the raster realms.
Hello there.
I’ve read this thread with great interest as I’m working on DGGS study and demonstrator, focused on raster data convestion to DGG systems (mostly on Sentinel 2 data)
We explore different approach to store and analyze raster data with DGGS implementations (rHEALPIX, H3, etc.) and the challenges related to the storage, conversion, handling.
No doubt that DGGS are game changers, for data indexation but also for ARD (Analysis Ready Data) in datacubes. Software ecoystem is not so mature but things are evolving.
Looking forward to exchanging on this interesting topic with you. Thanks @allixender for your commitment.
Cheers.
am curious where things have gone since the code sprint.
Development has slowed down a bit, but the best way to catch up is the repository. We’re planning to have regular meetings to pick this up again (see this topic), so that might also be a good way.
Is there much thought going into the 3D aspect of grids?
Not that I’m aware of, but if you have an index implementation (a library that has vectorized bindings for converting to / from cell ids) I’m sure we can figure out how to get these to work (eventually).
I’d like to share some updates that we came across the last weeks, months even, following the codesprint at the BIDS conference in November last year:
What @keewis just eluded to above is the prototype work on Xarray-xdggs:
We have a talk and and article accepted for the FOSS4G Europe 2024 conference, July 01-07 in Tartu, Estonia. There will ba classic free and open.source geospatial, but also an increasing touch of EO. We will certainly discuss DGGS again (like last year in Kosovo at the FOSS4G as well).
At the Open Geospatial Consortium (OGC) member meeting the DGGS standards and domain working groups where presenting case studies and their way forward in standardising “semantics” and API etc. One particular nice info to know is that ESA is continuing their investigation/experimenting with DGGS for satellite data.
Like @keewis mentioned as well, we will start small bi-weekly Xarray-xdggs dev meetings, where we discuss API design and specific library backends, i.e. HEALPIX, DGGRID, H3 …
This is also especially exciting, as different stakeholders currently have different preferences on which DGGS type to use, and custom implementations of converting vector and raster data into DGGS are developed. One example here, in a recent chat with devs at Landcare Research they showed me their tools, which align various input data into a harmonised H3 grid, with intermediate storing in (Geo)Parquet:
There could well be synergies eventually, because this is a classic Python implementation using rasterio, xarray, rioxarray and h3 and h3pandas. H3ronpy from above uses an underlying Rust/H3 implementation that might be more performant.
The ocean and climate science folks have actually historically huge experiences with re-gridding, especially when need to do that on very large scale or even global scale, where need to stay in WGS84 (lat-lon degrees space) and don’t have projected coordiniates in metres for areas available. Calculating overlaps on a sphere is tough, on ellipsoid even more so. @benbovy has started making Google’s S2 more accessible in Python with pys2index and Spherely, and while this is based on the sphere, S2 has very robust maths behind this. This could be a very We could consider turning a raster into a DGGS grid as a type of re-gridding. Experiments around the Pangeo tools xagg, xESMF and pyinterp will also provide valuable insights into dealing universally with gridded and thus DGGS gridded data as well.
Beyond that, other cool DGGS related things I came across recently:
GitHub - alxmrs/xarray-sql: An experiment to query Xarray datasets with SQL - not directly DGGS related, but querying an Xarray DataSet or DataArray via SQL semantics would also be super cool to close the gap between dataframe or database table analytics and Xarray, and having DGGS cell-id indexed 1D data, will super easy to analyse with SQL based query semantics as well
GitHub - sahrk/DGGRID: A command-line application that generates and manipulates icosahedral discrete global grids. while not posted as a release on Github, Kevin Sahr has again updated the DGGRID software, with improvements on internal structure (towards more flexible library usage) and ISEA7H fixes, the current version is 8.1b. I have also started to prepare a ReadTheDocs template for the user manual. I hope to have time to publish the docs and update the feedstock for conda-forge soon
GitHub - am2222/webDggrid: A WASM version of DGGRID - Majid Hojati has done the almost natural progression needed for more adoption, and released a demo of a WASM version of DGGRID. Being able to use WebAssembly compiled DGGRID in browser and NodeJs JavaScript could be a game changer. Well done.
Hi @nicolasCS thanks for joining here. I have a hunch that potentially there might be a chance to catch up and complement research and development efforts
Amazing thread here! Just a quick note that one of the H3 dev leads has a rather excellent H3 plugin for duckdb: GitHub - isaacbrodsky/h3-duckdb: Bindings for H3 to DuckDB . This works really nicely in concert with geoparquet + duckdb’s other spatial features. As @allixender notes, these indices are particularly effective for efficient aggregation of point data, and with duckdb this can be done on remotely streamed and larger-than-ram data.
@keewis made a great intro on using DGGS through xarray xdggs at the Pangeo showcase talk. Xdggs is now in a stage where you can use it fairly robustly with HEALPIX and H3. Other integrations like for DGGRID are developed as separate plugins.
It is wonderful to see the CF-convention considering HEALPix grid!!
I hope that this would later open up other DGGS grids to be easily being defined in CF-convention so that we have standardisation of input/ouput to NetCDF, ZARR (GEO ZARR spec …)
IF CF-convetion can take care of DGGS specification (like OGC-DGGS standards ), and GEO ZARR follows CF-convention, Pangeo will have perfect life with expressing DGGS. ( multidimensional , multivariable …)
