# Xarray to\_zarr interp\_like via dask distributed - anyone done this before?

**URL:** <https://discourse.pangeo.io/t/xarray-to-zarr-interp-like-via-dask-distributed-anyone-done-this-before/2492>\
**Category:** Science\
**Created:** [May 29, 2022, 8:14am UTC](https://discourse.pangeo.io/t/xarray-to-zarr-interp-like-via-dask-distributed-anyone-done-this-before/2492 "2022-05-29T08:14:19Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![RichardScottOZ](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/richardscottoz/32/752_2.png) [@RichardScottOZ](https://discourse.pangeo.io/u/RichardScottOZ)\
**Post date:** [May 29, 2022, 8:14am UTC](https://discourse.pangeo.io/t/xarray-to-zarr-interp-like-via-dask-distributed-anyone-done-this-before/2492/1 "2022-05-29T08:14:19Z")

</div>

Basically something like this, to turn some old 800m datasets into 400m facsimiles:-  
Load a low\_res xarray zarr dataset, interpolate to higher resolution.

```auto
def simplezarr3(creds):
    with worker_client() as client:
        for lr in lowRes:
            print("loading test lores:", lr)
            s3_path = 's3://bananasplits.zarr/'
            client_kwargs={'region_name': 'us-west-2'}

            s3 = s3fs.S3FileSystem(anon=False, key=access_key, secret=secret_key, client_kwargs=client_kwargs)
            store = s3fs.S3Map(root=s3_path, s3=s3, check=False)
            loRes = xr.open_zarr(store=store)

            with rasterio.Env(session):
                for index, basedir in enumerate(exploracorn_hiRes):    
                    s3dir = 's3://hiresSample/'
                    s3dir = s3dir + 'cog/results-' + basedir + '/'

                    basefile = 'Fleagle.tif'
                    s3file = s3dir + basefile
                    print(s3file)

                    commod = rioxarray.open_rasterio(s3file, chunks=(1,4096,4096))
                    commod = commod.expand_dims(time = [basedir])

                    hiRes = loRes.interp_like(commod)

                    s3_path_write = 's3://bananasplits/hires/' + lr + '.zarr/'
                    s3 = s3fs.S3FileSystem(anon=False, key=access_key, secret=secret_key, client_kwargs=client_kwargs)
                    writestore = s3fs.S3Map(root=s3_path_write, s3=s3, check=False)

                    hiRes.to_zarr(store=writestore, mode='w')

    return True

....

future = client.submit(simplezarr3, creds)

```

However, running this just seems to write the first chunk of the zarr array for each variable - so is the wrong and just writing out from one worker, or does something odd happen with dask/zarr xarray interps?

Get the metadata ok, but only the 0.0.0 chunk in each variable - when there should be several.

---

<div class="post-metadata">

**Author:** ![RichardScottOZ](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/richardscottoz/32/752_2.png) [@RichardScottOZ](https://discourse.pangeo.io/u/RichardScottOZ)\
**Post date:** [May 30, 2022, 3:32am UTC](https://discourse.pangeo.io/t/xarray-to-zarr-interp-like-via-dask-distributed-anyone-done-this-before/2492/2 "2022-05-30T03:32:11Z")

</div>

Now it is quite likely the interp didn’t work, became all nan [e.g. rectangular grid, only the country chunk in the middle has data] - which would be a pretty easy and small array to have compressed chunks of, whatever the size?

I think possibly for this reason have used rioxarray reproject match before - which reads into memory - so might have to go back to looking at other methods - warped vrt to rioxarray is possible I think in previous discussions - would have to pass the vrt via the client.

Barring the fire up a bunch of large machines to do it possibility - which only works up to a certain scale anyway.
