Advice for scalable raster-vector extraction?

*falls out of chair*

2 Likes

Wow, thanks @dbaston , that’s phenomenal to me! Just ran on my machine as well, works perfectly. And such concise code too. this is game-changing for me!

That looks like a pretty nifty trick - so with no need for reproject - will that work similarly but faster for the ‘load the data’ and exact_extract?

Hi @cboettig

I realize I never circled back on this. The poor performance comes from a few polygons that end up (erroneously) spanning the globe when reprojected:

The algorithm developer in me finds it fun to improve performance in these cases, but of course getting a wrong result more quickly isn’t actually helpful. I’ve created a ticket at Check input layer CRS · Issue #108 · isciences/exactextract · GitHub to track this for now.

Thanks @dbaston , good catch! definitely explains the exactextract taking longer on that one polygon and probably why the other methods mentioned above tended to crash when not restricted to CONUS. I still don’t have a great intuition why the performance of this pure gdal approach with exactextract is so much faster and also lighter on RAM while running on a single core than each of the above alternatives are even on many threads, but I’m definitely grateful!