# Xarray to Zarr Parallel Writes with Dask Distributed

**URL:** <https://discourse.pangeo.io/t/xarray-to-zarr-parallel-writes-with-dask-distributed/2611>\
**Category:** Data\
**Created:** [July 25, 2022, 8:07pm UTC](https://discourse.pangeo.io/t/xarray-to-zarr-parallel-writes-with-dask-distributed/2611 "2022-07-25T20:07:41Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![jgreen](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/jgreen/32/1683_2.png) [@jgreen](https://discourse.pangeo.io/u/jgreen)\
**Post date:** [July 26, 2022, 2:14pm UTC](https://discourse.pangeo.io/t/xarray-to-zarr-parallel-writes-with-dask-distributed/2611/6 "2022-07-26T14:14:51Z")

</div>

Thanks @willirath, I appreciate the information! The reason I am using `intake.open_netcdf` is because I found it very difficult to get the `xr.open_dataset(...)` engines to work with Google Cloud object storage. The `intake` module has proven much easier in this regard.

I have the NetCDF4 chunks internally stored at 100MB apiece, and from a [previous post](https://discourse.pangeo.io/t/unclear-behavior-of-netcdf4-files-loaded-with-intake-xarray/2529/7) I know that the Dask array chunks must be configured to exactly match – or be exact multiples – of the NetCDF internal chunks for an efficient parallel read. So, the chunksize I am specifying for the Dask array comes from `intake.open_netcdf(<URI>).to_dask().<DATA VARIABLE>.encoding['chunksizes']`

I will try the write using the mixture of threads/processes and implement the heuristic, I’m eager to see what new information I can gather from this. You really cleared things up for me!

---

_[View the full topic](https://discourse.pangeo.io/t/xarray-to-zarr-parallel-writes-with-dask-distributed/2611)._