# \#machine-learning

**URL:** https://discourse.pangeo.io/tag/machine-learning/7.md

[Latest](https://discourse.pangeo.io/latest.md) · [Categories](https://discourse.pangeo.io/categories.md) · [Tags](https://discourse.pangeo.io/tags.md)

---

## [AI Research Lead at Gaia Research (Haizea Analytics), Canberra, Australia](https://discourse.pangeo.io/t/ai-research-lead-at-gaia-research-haizea-analytics-canberra-australia/5908)

<div class="topic-metadata">

**Author:** [@Pablo\_Larraondo](https://discourse.pangeo.io/u/Pablo_Larraondo)\
**Replies:** 0\
**Last updated:** [November 13, 2026, 6:04am UTC](https://discourse.pangeo.io/t/ai-research-lead-at-gaia-research-haizea-analytics-canberra-australia/5908 "2026-11-13T06:04:48Z")

</div>

Haizea Analytics is hiring an AI Research Lead for Gaia Research, our AI research lab in Canberra, Australia. Role: full-time, based in Canberra (on-site preferred; remote work may be agreed under specific conditions) A…

---

## [Decode GeoTIFF to GPU memory](https://discourse.pangeo.io/t/decode-geotiff-to-gpu-memory/5214)

<div class="topic-metadata">

**Author:** [@weiji14](https://discourse.pangeo.io/u/weiji14)\
**Replies:** 18\
**Last updated:** [November 9, 2026, 2:41pm UTC](https://discourse.pangeo.io/t/decode-geotiff-to-gpu-memory/5214 "2026-11-09T14:41:55Z")

</div>

Sharing this blog post on speeding up Cloud-optimized GeoTIFF (COG) reads using a CUDA GPU library called nvTIFF that I’ve been playing around with for the past month. Hopefully it’ll be useful for folks struggling to us…

---

## [\[JOB\] Simons Center Postdoctoral Fellows in Computational Climate at NYU](https://discourse.pangeo.io/t/job-simons-center-postdoctoral-fellows-in-computational-climate-at-nyu/5829)

<div class="topic-metadata">

**Author:** [@SimonsCenter-Courant](https://discourse.pangeo.io/u/SimonsCenter-Courant)\
**Replies:** 0\
**Last updated:** [October 2, 2026, 6:11pm UTC](https://discourse.pangeo.io/t/job-simons-center-postdoctoral-fellows-in-computational-climate-at-nyu/5829 "2026-10-02T18:11:27Z")

</div>

The Simons Center for Computational Geophysical Flows at NYU’s Courant Institute is seeking full-time Postdoctoral Fellows to drive cutting-edge research in the following areas of interest: multiscale interactions, geoph…

---

## [How are reseachers using AI in Earth Observation workflows today?](https://discourse.pangeo.io/t/how-are-reseachers-using-ai-in-earth-observation-workflows-today/5804)

<div class="topic-metadata">

**Author:** [@rpmenon](https://discourse.pangeo.io/u/rpmenon)\
**Replies:** 1\
**Last updated:** [September 22, 2026, 2:01pm UTC](https://discourse.pangeo.io/t/how-are-reseachers-using-ai-in-earth-observation-workflows-today/5804 "2026-09-22T14:01:13Z")

</div>

Hi everyone, I’m an early-career researcher working primarily with ICESat-2 and other cryosphere datasets, and I’ve been thinking a lot about how AI is changing Earth observation research. I’ve seen several exciting pr…

---

## [Best Practice for Machine Learning with Huge Datasets](https://discourse.pangeo.io/t/best-practice-for-machine-learning-with-huge-datasets/4635)

<div class="topic-metadata">

**Author:** [@yonghan](https://discourse.pangeo.io/u/yonghan)\
**Replies:** 2\
**Last updated:** [June 27, 2026, 2:19am UTC](https://discourse.pangeo.io/t/best-practice-for-machine-learning-with-huge-datasets/4635 "2026-06-27T02:19:57Z")

</div>

Hey folks, I have been playing around with numerical weather forecasting using deep learning for a while. I mainly use the hourly reanalysis ERA5 datasets. The size of ERA5 datasets is huge, especially if you prefer a h…

---

## [Pangeo Showcase: "Cloud Native Data Loaders for Machine Learning Using Zarr and Xarray"](https://discourse.pangeo.io/t/pangeo-showcase-cloud-native-data-loaders-for-machine-learning-using-zarr-and-xarray/4110)

<div class="topic-metadata">

**Author:** [@rsignell](https://discourse.pangeo.io/u/rsignell)\
**Replies:** 7\
**Last updated:** [June 26, 2026, 6:08pm UTC](https://discourse.pangeo.io/t/pangeo-showcase-cloud-native-data-loaders-for-machine-learning-using-zarr-and-xarray/4110 "2026-06-26T18:08:13Z")

</div>

Title: “Cloud Native Data Loaders for Machine Learning Using Zarr and Xarray” Invited Speaker: Joe Hamman (ORCID:0000-0001-7479-8439) When: Wednesday March 20, 12PM EDT Where: Launch Meeting - Zoom Abstr…

---

## [Favorite way to go from netCDF (&xarray) to torch/TF/Jax et al](https://discourse.pangeo.io/t/favorite-way-to-go-from-netcdf-xarray-to-torch-tf-jax-et-al/2663)

<div class="topic-metadata">

**Author:** [@ThomasMGeo](https://discourse.pangeo.io/u/ThomasMGeo)\
**Replies:** 8\
**Last updated:** [June 26, 2026, 6:04pm UTC](https://discourse.pangeo.io/t/favorite-way-to-go-from-netcdf-xarray-to-torch-tf-jax-et-al/2663 "2026-06-26T18:04:56Z")

</div>

Hello! I am researching best practices to load spatial, geoscience data (generally netCDF/Grib via xarray) to various ML packages. Two things kept coming up: xbatcher project: GitHub - pangeo-data/xbatcher: Batch gen…

---

## [Remote Sensing Scientist - Commodities Research](https://discourse.pangeo.io/t/remote-sensing-scientist-commodities-research/5405)

<div class="topic-metadata">

**Author:** [@george](https://discourse.pangeo.io/u/george)\
**Replies:** 0\
**Last updated:** [September 23, 2025, 10:43pm UTC](https://discourse.pangeo.io/t/remote-sensing-scientist-commodities-research/5405 "2025-09-23T22:43:11Z")

</div>

Hi All, First time posting but Pangeo is where I normally end up when searching for answers to my deepest STAC problems. I run a commodities research firm and we’re currently hiring for a remote sensing scientist posit…

---

## [Interested to learn hybrid modeling](https://discourse.pangeo.io/t/interested-to-learn-hybrid-modeling/5398)

<div class="topic-metadata">

**Author:** [@nrchow](https://discourse.pangeo.io/u/nrchow)\
**Replies:** 5\
**Last updated:** [September 16, 2025, 7:01am UTC](https://discourse.pangeo.io/t/interested-to-learn-hybrid-modeling/5398 "2025-09-16T07:01:04Z")

</div>

I am interested to learn about hybrid modeling (machine learning incorporated to conventional physics-based models) but unsure how to start. I have some background in numerical modeling, as in I have experience in prepar…

---

## [Best way to structure satellite data in Zarr for time-based indexing and batching with DataLoader](https://discourse.pangeo.io/t/best-way-to-structure-satellite-data-in-zarr-for-time-based-indexing-and-batching-with-dataloader/5384)

<div class="topic-metadata">

**Author:** [@gdravilas](https://discourse.pangeo.io/u/gdravilas)\
**Replies:** 1\
**Last updated:** [September 6, 2025, 8:38pm UTC](https://discourse.pangeo.io/t/best-way-to-structure-satellite-data-in-zarr-for-time-based-indexing-and-batching-with-dataloader/5384 "2025-09-06T20:38:41Z")

</div>

Hi everyone, I ‘m working on a project with satellite data that spans across multiple timestamps, and I ‘d love some advice on structuring it efficiently. The data For each variable (satellite channels), I have several…

---

## [Machine Learning training with large Xarray Datasets](https://discourse.pangeo.io/t/machine-learning-training-with-large-xarray-datasets/5136)

<div class="topic-metadata">

**Author:** [@ms2hhc](https://discourse.pangeo.io/u/ms2hhc)\
**Replies:** 0\
**Last updated:** [May 15, 2025, 3:49pm UTC](https://discourse.pangeo.io/t/machine-learning-training-with-large-xarray-datasets/5136 "2025-05-15T15:49:34Z")

</div>

Hi all, I’d like to share a common workflow I face in Land Cover mapping tasks and hear your thoughts on how best to handle part of it with Xarray/Dask. The general idea is to use a global land cover product as ground …

---

## [\[CI2025\] Hydrology modelling hackathon](https://discourse.pangeo.io/t/ci2025-hydrology-modelling-hackathon/4972)

<div class="topic-metadata">

**Author:** [@rlourenco](https://discourse.pangeo.io/u/rlourenco)\
**Replies:** 0\
**Last updated:** [March 11, 2025, 1:47pm UTC](https://discourse.pangeo.io/t/ci2025-hydrology-modelling-hackathon/4972 "2025-03-11T13:47:10Z")

</div>

Dear Colleagues, First, I would like to thank everyone who submitted papers to Climate Informatics (https://2025.climateinformatics.com.br), which will take place at FGV-RJ next April. The submissions are currently unde…

---

## [Create batches of random subsets of data scattered across different files](https://discourse.pangeo.io/t/create-batches-of-random-subsets-of-data-scattered-across-different-files/4840)

<div class="topic-metadata">

**Author:** [@HenrikAsmuth](https://discourse.pangeo.io/u/HenrikAsmuth)\
**Replies:** 2\
**Last updated:** [January 29, 2025, 7:31am UTC](https://discourse.pangeo.io/t/create-batches-of-random-subsets-of-data-scattered-across-different-files/4840 "2025-01-29T07:31:49Z")

</div>

Hi all, I am currently dealing with several ML projects with a similar problem: My training data is scattered across 100s of netcdf files and I need batches of random subsets from all those files. E.g., say I have 100 …

---

## [Shuffling and windowing an xarray dataset for machine learning](https://discourse.pangeo.io/t/shuffling-and-windowing-an-xarray-dataset-for-machine-learning/4790)

<div class="topic-metadata">

**Author:** [@ganzk](https://discourse.pangeo.io/u/ganzk)\
**Replies:** 8\
**Last updated:** [January 13, 2025, 7:42pm UTC](https://discourse.pangeo.io/t/shuffling-and-windowing-an-xarray-dataset-for-machine-learning/4790 "2025-01-13T19:42:46Z")

</div>

Hello, In my work I’m struggling with providing data from xarray to a machine learning model. I’m aware of tools like xbatcher in this blog post and this other thread. I run into two main sticking points: Randomly shu…

---

## [Geosoft GDB converstion at scale](https://discourse.pangeo.io/t/geosoft-gdb-converstion-at-scale/4662)

<div class="topic-metadata">

**Author:** [@RichardScottOZ](https://discourse.pangeo.io/u/RichardScottOZ)\
**Replies:** 0\
**Last updated:** [November 6, 2024, 10:54pm UTC](https://discourse.pangeo.io/t/geosoft-gdb-converstion-at-scale/4662 "2024-11-06T22:54:14Z")

</div>

Last weekend notes from some brief looks: GitHub - RichardScottOZ/Geosoft-GDB-Conversion: Work looking at converting Geosoft GDBs to useable open format for analysis, machine learning and AI This is geophysics data \[Mag…

---

## [Xarray slow read on cluster](https://discourse.pangeo.io/t/xarray-slow-read-on-cluster/4604)

<div class="topic-metadata">

**Author:** [@yonghan](https://discourse.pangeo.io/u/yonghan)\
**Replies:** 4\
**Last updated:** [November 3, 2024, 9:19am UTC](https://discourse.pangeo.io/t/xarray-slow-read-on-cluster/4604 "2024-11-03T09:19:36Z")

</div>

Hi there, I am working on a weather forecasting project using the ERA5 datasets. The datasets are stored on a yearly basis in zarr format, with chunking on time dimension (chunk={time:1, lat: -1, lon: -1, pressure\_level…

---

## [Nexrad Level 2 data - Get slices of an area from the radar reflectivity](https://discourse.pangeo.io/t/nexrad-level-2-data-get-slices-of-an-area-from-the-radar-reflectivity/4602)

<div class="topic-metadata">

**Author:** [@vishnukadiyala](https://discourse.pangeo.io/u/vishnukadiyala)\
**Replies:** 2\
**Last updated:** [October 15, 2024, 8:13pm UTC](https://discourse.pangeo.io/t/nexrad-level-2-data-get-slices-of-an-area-from-the-radar-reflectivity/4602 "2024-10-15T20:13:15Z")

</div>

I am a computer science student and want to build some machine learning application for precipitation rate estimation using machine learning. For proof of concept I decided to predict precipitation rate at one particular…

---

## [Assess the value of data by measuring the cost of information](https://discourse.pangeo.io/t/assess-the-value-of-data-by-measuring-the-cost-of-information/4598)

<div class="topic-metadata">

**Author:** [@alxmrs](https://discourse.pangeo.io/u/alxmrs)\
**Replies:** 0\
**Last updated:** [October 12, 2024, 4:18am UTC](https://discourse.pangeo.io/t/assess-the-value-of-data-by-measuring-the-cost-of-information/4598 "2024-10-12T04:18:58Z")

</div>

I’ve been talking to Jed Sundwall recently, sometimes about open data. I just wrote him about an idea that we both agreed may benefit from a wider discussion. Here it is, in full. Subject: Assess the value of data by me…

---

## [Research Leads/Senior Research Associates in Environment and Sustainability @The Alan Turing Institute](https://discourse.pangeo.io/t/research-leads-senior-research-associates-in-environment-and-sustainability-the-alan-turing-institute/4424)

<div class="topic-metadata">

**Author:** [@acocac](https://discourse.pangeo.io/u/acocac)\
**Replies:** 0\
**Last updated:** [August 14, 2024, 1:12pm UTC](https://discourse.pangeo.io/t/research-leads-senior-research-associates-in-environment-and-sustainability-the-alan-turing-institute/4424 "2024-08-14T13:12:46Z")

</div>

Hi All, The Alan Turing Institute, the UK’s national institute for data science and artificial intelligence, is recruiting two senior research roles with mid-September closing dates. The roles will support the Environm…

---

## [Efficient data selection without for loops](https://discourse.pangeo.io/t/efficient-data-selection-without-for-loops/4371)

<div class="topic-metadata">

**Author:** [@suryadheeshjith](https://discourse.pangeo.io/u/suryadheeshjith)\
**Replies:** 3\
**Last updated:** [July 16, 2024, 12:40pm UTC](https://discourse.pangeo.io/t/efficient-data-selection-without-for-loops/4371 "2024-07-16T12:40:08Z")

</div>

Hello, I am looking for an efficient way to extract data with multiple slices in a list. The best way to show this is with an example - # Create the data array data = xr.DataArray( np.random.rand(5, 6), dims=\["t…

---

## [Webinar: How NOAA Open Data Dissemination Weather Forecasts Are Boosting Grid Operation](https://discourse.pangeo.io/t/webinar-how-noaa-open-data-dissemination-weather-forecasts-are-boosting-grid-operation/4295)

<div class="topic-metadata">

**Author:** [@emfdavid](https://discourse.pangeo.io/u/emfdavid)\
**Replies:** 0\
**Last updated:** [June 17, 2024, 10:22pm UTC](https://discourse.pangeo.io/t/webinar-how-noaa-open-data-dissemination-weather-forecasts-are-boosting-grid-operation/4295 "2024-06-17T22:22:02Z")

</div>

Camus Energy is co-presenting with NODD at this Energy Systems Integration Group (ESIG) webinar. It is intended to introduce the electric grid industry ESIG member community to some of the tools used in the Pangeo commun…

---

## [CZI & Sloan RFI: Pathways to AI-enabled research](https://discourse.pangeo.io/t/czi-sloan-rfi-pathways-to-ai-enabled-research/4207)

<div class="topic-metadata">

**Author:** [@weiji14](https://discourse.pangeo.io/u/weiji14)\
**Replies:** 1\
**Last updated:** [May 7, 2024, 7:56pm UTC](https://discourse.pangeo.io/t/czi-sloan-rfi-pathways-to-ai-enabled-research/4207 "2024-05-07T19:56:53Z")

</div>

The Open Science program at Chan Zuckerberg Initiative and Alfred P. Sloan Foundation Technology Program are requesting community input about existing applications of AI to transform scientific work that are consistent w…

---

## [Pangeo Machine Learning Ecosystem 2023 diagram](https://discourse.pangeo.io/t/pangeo-machine-learning-ecosystem-2023-diagram/3742)

<div class="topic-metadata">

**Author:** [@weiji14](https://discourse.pangeo.io/u/weiji14)\
**Replies:** 1\
**Last updated:** [April 3, 2024, 6:49pm UTC](https://discourse.pangeo.io/t/pangeo-machine-learning-ecosystem-2023-diagram/3742 "2024-04-03T18:49:06Z")

</div>

Spent a whole day working on this diagram of the Pangeo Machine Learning (ML) ecosystem for a talk next week (18 Oct 2023), and I’d like to solicit some feedback from any Pangeo Machine Learning practicioners! In par…

---

## [Climatematch Academy 2024 - Application Open!](https://discourse.pangeo.io/t/climatematch-academy-2024-application-open/4089)

<div class="topic-metadata">

**Author:** [@courtneydean33](https://discourse.pangeo.io/u/courtneydean33)\
**Replies:** 1\
**Last updated:** [March 24, 2024, 5:58am UTC](https://discourse.pangeo.io/t/climatematch-academy-2024-application-open/4089 "2024-03-24T05:58:14Z")

</div>

Climatematch Academy: Computational Tools for Climate Science applications are open for students and TAs! This course is a wide-reaching, inclusive, and approachable program aimed to introduce computational methods for …

---

## [Blog post: cloud native data loaders for machine learning using zarr and xarray](https://discourse.pangeo.io/t/blog-post-cloud-native-data-loaders-for-machine-learning-using-zarr-and-xarray/4103)

<div class="topic-metadata">

**Author:** [@jhamman](https://discourse.pangeo.io/u/jhamman)\
**Replies:** 0\
**Last updated:** [March 14, 2024, 5:09pm UTC](https://discourse.pangeo.io/t/blog-post-cloud-native-data-loaders-for-machine-learning-using-zarr-and-xarray/4103 "2024-03-14T17:09:44Z")

</div>

We just published a blog post outlining a cloud native architecture for training ML models using Xarray and Zarr. Title: cloud native data loaders for machine learning using zarr and xarray Link: Cloud native data load…

---

## [PhD student in Machine Learning with Applications in Weather Forecasting/Climate Science, Linköping, Sweden](https://discourse.pangeo.io/t/phd-student-in-machine-learning-with-applications-in-weather-forecasting-climate-science-linkoping-sweden/3915)

<div class="topic-metadata">

**Author:** [@freli005](https://discourse.pangeo.io/u/freli005)\
**Replies:** 0\
**Last updated:** [December 14, 2023, 1:55pm UTC](https://discourse.pangeo.io/t/phd-student-in-machine-learning-with-applications-in-weather-forecasting-climate-science-linkoping-sweden/3915 "2023-12-14T13:55:48Z")

</div>

We are looking for a PhD student in Machine Learning at Linköping University, Sweden. The research topic is machine learning for spatio-temporal modeling, prediction, and reasoning. The focus is on generic method develop…

---

## [Machine Learning Engineer Job hire](https://discourse.pangeo.io/t/machine-learning-engineer-job-hire/3801)

<div class="topic-metadata">

**Author:** [@Bhavika\_Sharma](https://discourse.pangeo.io/u/Bhavika_Sharma)\
**Replies:** 0\
**Last updated:** [October 27, 2023, 11:36am UTC](https://discourse.pangeo.io/t/machine-learning-engineer-job-hire/3801 "2023-10-27T11:36:39Z")

</div>

We are hiring! If you’re a Machine Learning Engineer looking for an opportunity to use your skills in a meaningful way, come join our team! https://openclimatefix.homerun.co/machine-learning-engineer/en About the job …

---

## [How to run code using GPU on Pangeo? saying "libdevice not found at ./libdevice.10.bc"](https://discourse.pangeo.io/t/how-to-run-code-using-gpu-on-pangeo-saying-libdevice-not-found-at-libdevice-10-bc/3672)

<div class="topic-metadata">

**Author:** [@leiyan90](https://discourse.pangeo.io/u/leiyan90)\
**Replies:** 13\
**Last updated:** [September 21, 2023, 12:20am UTC](https://discourse.pangeo.io/t/how-to-run-code-using-gpu-on-pangeo-saying-libdevice-not-found-at-libdevice-10-bc/3672 "2023-09-21T00:20:50Z")

</div>

I am learning to run a CNN model using GPU on pangeo. The code is okay for CPU, but when I transfer to GPU, there is a an error: 2023-09-10 19:25:40.201197: I tensorflow/compiler/xla/service/service.cc:169\] XLA service …

---

## [Geospatial Data Science opportunities](https://discourse.pangeo.io/t/geospatial-data-science-opportunities/3476)

<div class="topic-metadata">

**Author:** [@Koigi](https://discourse.pangeo.io/u/Koigi)\
**Replies:** 5\
**Last updated:** [August 22, 2023, 9:50pm UTC](https://discourse.pangeo.io/t/geospatial-data-science-opportunities/3476 "2023-08-22T21:50:58Z")

</div>

Hello, I’m looking for guidance on how to land geospatial data science remote opportunities. What skills should I focus on? How can I get a foot into this roles? Kindly advice. Regards

---

## [PostDoc position @Yale: Machine Learning + Ocean Modeling](https://discourse.pangeo.io/t/postdoc-position-yale-machine-learning-ocean-modeling/3613)

<div class="topic-metadata">

**Author:** [@shishirdash](https://discourse.pangeo.io/u/shishirdash)\
**Replies:** 0\
**Last updated:** [August 10, 2023, 1:55pm UTC](https://discourse.pangeo.io/t/postdoc-position-yale-machine-learning-ocean-modeling/3613 "2023-08-10T13:55:28Z")

</div>

The Eisaman Lab for Carbon Removal and Clean Technologies in the Department of Earth & Planetary Sciences and the Yale Center for Natural Carbon Capture (YCNCC) at Yale University is seeking a highly motivated postdoctor…

[Next page](https://discourse.pangeo.io/tag/machine-learning/7.md?match_all_tags=true&page=1&tags%5B%5D=machine-learning)
