# Using xhistogram to bin measurements at particular stations

**URL:** <https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365>\
**Category:** Science\
**Created:** [March 30, 2022, 1:26pm UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365 "2022-03-30T13:26:56Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dan\_Jones](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/dan_jones/32/1559_2.png) [@Dan\_Jones](https://discourse.pangeo.io/u/Dan_Jones)\
**Post date:** [March 30, 2022, 1:26pm UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365/1 "2022-03-30T13:26:56Z")

</div>

Hi all,

I have a collection of Argo profiles, which as probably all of you know are collections of temperature and salinity measurements over depth at particular latitude/longitude points.

I’ve been using xhistogram to calculate, for example, mean surface temperatures over a selected lat/lon grid. This works really well, and I’m glad to have this tool.

However, some of my values are discrete (e.g. integer labels). I’d like to calculate quantities like the median and the mode over these discrete values. Is it possible to group my profiles based on which lat/lon bin they belong to, with the intention of calculating median and mode over those groups?

Apologies if I’ve missed something obvious. I’ve spent some time looking at the docs and experimenting, but I haven’t found a solution yet. Thanks in advance for any help or clarification you can provide. 🙂

---

<div class="post-metadata">

**Author:** ![rabernat](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/rabernat/32/22_2.png) [@rabernat](https://discourse.pangeo.io/u/rabernat)\
**Post date:** [March 30, 2022, 2:24pm UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365/2 "2022-03-30T14:24:46Z")

</div>

Thanks for the question Dan and welcome back to the forum! 🤗

I am trying to understand what you aim to do, but I’m not quite there yet. Are you saying you’d like to apply a custom reduction using xhistogram, rather than just sum / mean?

Perhaps you could write some pseudo python code to express what you _wish_ would work but does not actually work? That would give some more concreteness to the discussion.

---

<div class="post-metadata">

**Author:** ![Dan\_Jones](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/dan_jones/32/1559_2.png) [@Dan\_Jones](https://discourse.pangeo.io/u/Dan_Jones)\
**Post date:** [March 30, 2022, 3:08pm UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365/3 "2022-03-30T15:08:13Z")

</div>

Thanks for your rapid reply and your warm greeting, Ryan! 😊 Essentially, yes - I’d like to calculate something with histogram other than the mean. I’ll give your suggestion a shot. At present, I have a DataArray that looks like this:

 ![Screen Shot 2022-03-30 at 15.32.42-page-001](https://canada1.discourse-cdn.com/flex030/uploads/pangeo/original/2X/5/59b68522b292ee6164fa8531d1b97d798cdc95c8.jpeg)

The profiles are unevenly distributed in latitude and longitude, and the labels are discrete values assigned by an unsupervised classification algorithm. I’d like to be able to calculate the most common label value in each 1°x1° bin on a lat-lon grid.

Perhaps it would look something like this:

```auto
import numpy as np
from xhistogram.xarray import histogram
import matplotlib.pyplot as plt

# define latitude and longitude bins
binsize = 1.0 # 1°x1° bins
lon_bins = np.arange(lon_min, lon_max, binsize)
lat_bins = np.arange(lat_min, lat_max, binsize)

# either 
indices_of_profiles_in_each_lat_lon_bin, hist = histogram(da.lon, 
                                                          da.lat, 
                                                          bins=[lon_bins, lat_bins])

```

where `indices_of_profiles_in_each_lat_lon_bin` is a set of indices that tells me which profiles fall into each lat-lon grid cell. I could then select the profiles in each lat-lon grid cell and calculate the mode of the label values.

**Even better** would be a command that lets me simply do the following. (I’m totally making this code up, no idea if it makes sense…)

```auto
A = da.groupby(['lon_bins', 'lat_bins']).mode()

```

where `mode` returns the most frequently occurring label value in each lat-lon bin. The result `A` would be a 2D object that I could plot as follows:

```auto
plt.pcolormesh(lon_bins, lat_bins, A)

```

The result might be something like this, except that the values would be discrete instead of continuous:

 ![hist_dyn_height_0K](https://canada1.discourse-cdn.com/flex030/uploads/pangeo/original/2X/0/0add12017feb07cca67ba3b40103fbb40439751c.png)

I hope that makes sense - please do let me know if something is unclear. 🤔 Thanks again! 😊

---

<div class="post-metadata">

**Author:** ![rabernat](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/rabernat/32/22_2.png) [@rabernat](https://discourse.pangeo.io/u/rabernat)\
**Post date:** [March 30, 2022, 3:16pm UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365/4 "2022-03-30T15:16:22Z")

</div>

Ok that is much more clear. Thanks for taking the time to write it up.

You’re correct that it is not possible today with xhistogram. The reason is that, as currently implemented, xhistogram relies heavily on the fact that it is easy to just sum up the bin counts from each block of data to reach the total for each. Sum is commutative and associative, so it is trivial to parallelize (and most of the code in xhistogram is about making things play well with dask).

FWIW, the algorithm itself is here and is not that complicated to read.

> <https://github.com/xgcm/xhistogram/blob/master/xhistogram/core.py>

> [@Dan\_Jones](#):
>
> **Even better** would be a command that lets me simply do the following. (I’m totally making this code up, no idea if it makes sense…)
> 
> ```auto
> A = da.groupby(['lon_bins', 'lat_bins']).mode()
> 
> ```

This is a very long-standing open issue in xarray:

> <https://github.com/pydata/xarray/issues/324>
>
> Multi-dimensional grouped operations should be relatively straightforward -- the… main complexity will be writing an N-dimensional concat that doesn't involve repetitively copying data.
> 
> The idea with \`group\_over\` would be to support groupby operations that act on a single element from each of the given groups, rather than the unique values. For example, \`ds.group\_over(\['lat', 'lon'\])\` would let you iterate over or apply to 2D slices of \`ds\`, no matter how many dimensions it has.
> 
> Roughly speaking (it's a little more complex for the case of non-dimension variables), \`ds.group\_over(dims)\` would get translated into \`ds.groupby(\[d for d in ds.dims if d not in dims\])\`. 
> 
> Related: #266

I _think_ that the Flox package by @dcherian supports it. Let’s see what Deepak has to say about this.

> **[GitHub - xarray-contrib/flox: Fast & furious GroupBy operations for...](https://github.com/xarray-contrib/flox)**
>
> Fast & furious GroupBy operations for dask.array. Contribute to xarray-contrib/flox development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![Dan\_Jones](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/dan_jones/32/1559_2.png) [@Dan\_Jones](https://discourse.pangeo.io/u/Dan_Jones)\
**Post date:** [March 30, 2022, 3:40pm UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365/5 "2022-03-30T15:40:45Z")

</div>

Right! Okay, I see now. Thanks for the detailed reply.

---

<div class="post-metadata">

**Author:** ![dcherian](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/dcherian/32/2235_2.png) [@dcherian](https://discourse.pangeo.io/u/dcherian)\
**Post date:** [March 31, 2022, 4:02pm UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365/6 "2022-03-31T16:02:09Z")

</div>

Yes you can do this with flox but it won’t be fast.

I wrote an example for median here: [Custom Aggregations - flox](https://flox.readthedocs.io/en/latest/user-stories/custom-aggregations.html)

Replace `np.median` with `scipy.stats.mode` for mode. This version doesn’t support dask but this kind of statistic is hard to do in parallel.

The documentation on this cool feature is atrocious. So please let me know if you have any questions. PRs to improve the notebook or documentation are also very welcome!

---

<div class="post-metadata">

**Author:** ![rlourenco](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/rlourenco/32/2667_2.png) [@rlourenco](https://discourse.pangeo.io/u/rlourenco)\
**Post date:** [December 16, 2024, 10:11pm UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365/7 "2024-12-16T22:11:53Z")

</div>

> [@dcherian](#):
>
> [Custom Aggregations - flox](https://flox.readthedocs.io/en/latest/user-stories/custom-aggregations.html)

It’s 2024, but I used it today with mode (for categorical aggregation), and it was pretty helpful (thanks, @dcherian )

---

<div class="post-metadata">

**Author:** ![dcherian](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/dcherian/32/2235_2.png) [@dcherian](https://discourse.pangeo.io/u/dcherian)\
**Post date:** [December 16, 2024, 10:37pm UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365/8 "2024-12-16T22:37:28Z")

</div>

`mode` is now built-in, you can use `func="mode"`

`xarray-regrid` recently added `mode` as a regridding method using flox to compute a histogram. You could use that approach for a faster `mode`

> <https://github.com/xarray-contrib/xarray-regrid/blob/1425e1353b4405b4ea734ef37f78cf4538a72949/src/xarray_regrid/methods/flox_reduce.py#L118>

---

<div class="post-metadata">

**Author:** ![rlourenco](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/rlourenco/32/2667_2.png) [@rlourenco](https://discourse.pangeo.io/u/rlourenco)\
**Post date:** [December 17, 2024, 12:32am UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365/9 "2024-12-17T00:32:44Z")

</div>

Great! Thanks for the update 🙂

---

<div class="post-metadata">

**Author:** ![Dan\_Jones](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/dan_jones/32/1559_2.png) [@Dan\_Jones](https://discourse.pangeo.io/u/Dan_Jones)\
**Post date:** [December 28, 2024, 9:29pm UTC](https://discourse.pangeo.io/t/using-xhistogram-to-bin-measurements-at-particular-stations/2365/10 "2024-12-28T21:29:51Z")

</div>

I am glad to hear it! 🥳