I’m curious if anyone here has interfaced with plotfile data from any AMReX projects. I’m on a project where we are generating TB-scale data files that are formatted in the native AMReX format, and I’m hoping to reformat these files to Zarr so that we can more easily analyze the data. While the codes are capable of generating data on dynamic grids, my data has a fixed static grid, and there are also some static refinement zones, though I think I could ignore these refinement zones as a starting point. Speaking to AMReX team members, they recommend trying one of two things
The yt-project can read in native AMReX data, so I could maybe interface with the data through there
Using pyamrex, which is a newer library that I haven’t yet had the chance to read up on
If anyone has taken a stab at a problem like this in the past, it’d be great to chat. Thanks!
In case any AMReX users find my post, here’s a short writeup of my efforts. I wrote some really rough code that can be used to directly read plt/ files into a numpy array or a Dask array, and I also applied this code to write out Zarr stores. I ended up copying over a bunch of code from the yt AMReX reader to get this to work. At a high level, it works by reading many small outputs of binary data though numpy memory mapping and then stitching them together. My intent with this reader was to apply it to output from AMR-Wind simulations, so I made some key assumptions that might not work with your code. Importantly, I’m reading data from each Level into its own store, but in the future, maybe something like ome-zarr could be used to bundle things more nicely. Happy to chat more if folks want to mess around with this more.