# Extremly slow write to S3 bucket with xarray.Dataset.to\_zarr

**URL:** <https://discourse.pangeo.io/t/extremly-slow-write-to-s3-bucket-with-xarray-dataset-to-zarr/2262>\
**Category:** Data\
**Created:** [February 24, 2022, 4:13am UTC](https://discourse.pangeo.io/t/extremly-slow-write-to-s3-bucket-with-xarray-dataset-to-zarr/2262 "2022-02-24T04:13:14Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![rabernat](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/rabernat/32/22_2.png) [@rabernat](https://discourse.pangeo.io/u/rabernat)\
**Post date:** [February 24, 2022, 3:31pm UTC](https://discourse.pangeo.io/t/extremly-slow-write-to-s3-bucket-with-xarray-dataset-to-zarr/2262/2 "2022-02-24T15:31:22Z")

</div>

Thanks for sharing Masha! Writing a dataset of this size should take seconds, not hours.

@martindurant - do you have any idea what might be happening here? We have had similar reports of very slow writes with xarray → zarr → s3fs → s3, e.g.

> [@Any tips on avoiding high AWS S3 cost when storing Zarr's with lots of objects](https://discourse.pangeo.io/t/any-tips-on-avoiding-high-aws-s3-cost-when-storing-zarrs-with-lots-of-objects/1977):
>
> Hello, We are generating Zarr stores and push final data products to AWS S3 bucket. We were hit by unexpected high cost (~3K to store 40 data products) from AWS S3 to store our test data sets. When using (250, 10, 10) chunking for (date, x, y) dimensions, we ended up with average ~3-4M objects per single Zarr store, thus the high S3 cost. Our normal way of operation is to generate the data set locally and to push it to the S3 bucket. When new data that contributes to the data set becomes avai…

@mlukis has provided a nice minimal reproducible example we could use for debugging this.

---

_[View the full topic](https://discourse.pangeo.io/t/extremly-slow-write-to-s3-bucket-with-xarray-dataset-to-zarr/2262)._
