Although this wasn’t our primary design goal, the chunk-key indirection and content-addressable-storage scheme we implemented in Arraylake solves this problem pretty well.
We’ll be giving an overview of our solution this coming Wednesday at the Pangeo showcase.