# Cloud

**URL:** https://discourse.pangeo.io/c/cloud/7.md

[Latest](https://discourse.pangeo.io/latest.md) · [Categories](https://discourse.pangeo.io/categories.md) · [Tags](https://discourse.pangeo.io/tags.md)

---

## [Pangeo in the cloud](https://discourse.pangeo.io/t/pangeo-in-the-cloud/25)

<div class="topic-metadata">

**Author:** [@hetland](https://discourse.pangeo.io/u/hetland)\
**Replies:** 0

</div>

Pangeo environments in the cloud Here we discuss best practices for setting up Pangeo environments in the cloud.

---

## [I want advice for Smooth Cloud Workflow Setup](https://discourse.pangeo.io/t/i-want-advice-for-smooth-cloud-workflow-setup/5288)

<div class="topic-metadata">

**Author:** [@Elenana](https://discourse.pangeo.io/u/Elenana)\
**Replies:** 0\
**Last updated:** [July 18, 2025, 8:44am UTC](https://discourse.pangeo.io/t/i-want-advice-for-smooth-cloud-workflow-setup/5288 "2025-07-18T08:44:54Z")

</div>

Hi everyone, I am starting to explore cloud-based workflows for working with scientific datasets such as climate and environmental data. I have read through some of the Pangeo documentation and browsed tutorials but I …

---

## [Online Resources from CNG2025](https://discourse.pangeo.io/t/online-resources-from-cng2025/5116)

<div class="topic-metadata">

**Author:** [@NickHodgskin](https://discourse.pangeo.io/u/NickHodgskin)\
**Replies:** 7\
**Last updated:** [July 8, 2025, 11:11pm UTC](https://discourse.pangeo.io/t/online-resources-from-cng2025/5116 "2025-07-08T23:11:32Z")

</div>

Hello, I am interested in the talks from the Cloud Native Geospatial (CNG) Conference 2025 . I’m wondering if there are any online resources (or plans to release online recordings) from the talks at the event as, unfort…

---

## [I want tips on setting up a scalable Pangeo cloud environment](https://discourse.pangeo.io/t/i-want-tips-on-setting-up-a-scalable-pangeo-cloud-environment/5173)

<div class="topic-metadata">

**Author:** [@Sofiaya](https://discourse.pangeo.io/u/Sofiaya)\
**Replies:** 1\
**Last updated:** [June 4, 2025, 12:19am UTC](https://discourse.pangeo.io/t/i-want-tips-on-setting-up-a-scalable-pangeo-cloud-environment/5173 "2025-06-04T00:19:49Z")

</div>

Hey everyone, I am new to Pangeo & trying to set up a cloud environment that can handle large geoscience datasets efficiently. I have been exploring some tools & workflows but still feel a bit lost on where to start wit…

---

## [Rioxarray reading COG (from Earth Engine) with bottom \> top](https://discourse.pangeo.io/t/rioxarray-reading-cog-from-earth-engine-with-bottom-top/5158)

<div class="topic-metadata">

**Author:** [@jakenotjay](https://discourse.pangeo.io/u/jakenotjay)\
**Replies:** 1\
**Last updated:** [May 22, 2025, 5:10pm UTC](https://discourse.pangeo.io/t/rioxarray-reading-cog-from-earth-engine-with-bottom-top/5158 "2025-05-22T17:10:16Z")

</div>

I noticed this when I was running a pad\_xy on my dataset expanding its dimensions to fit a bounding box which I’ll be filling with other data (I expect a dataset filling the coordinates, with NA), this looks like so: bo…

---

## [Duration of Pangeo GCS bucket](https://discourse.pangeo.io/t/duration-of-pangeo-gcs-bucket/5122)

<div class="topic-metadata">

**Author:** [@annabelwade](https://discourse.pangeo.io/u/annabelwade)\
**Replies:** 1\
**Last updated:** [May 9, 2025, 6:52am UTC](https://discourse.pangeo.io/t/duration-of-pangeo-gcs-bucket/5122 "2025-05-09T06:52:54Z")

</div>

Hello, We have been using a Pangeo GCS bucket for a while and want to know how long it is expected to be available, in light of recent changes to the Pangeo hub. Is there a duration or timeline for how long the GCS stor…

---

## [Proposal: Open source guidebook for scientific cloud computing infrastructure](https://discourse.pangeo.io/t/proposal-open-source-guidebook-for-scientific-cloud-computing-infrastructure/4980)

<div class="topic-metadata">

**Author:** [@maxrjones](https://discourse.pangeo.io/u/maxrjones)\
**Replies:** 3\
**Last updated:** [March 13, 2025, 11:27pm UTC](https://discourse.pangeo.io/t/proposal-open-source-guidebook-for-scientific-cloud-computing-infrastructure/4980 "2025-03-13T23:27:11Z")

</div>

@TomNicholas and @martindurant’s questions at today’s Pangeo Showcase reminded me of this old, small issue I wrote proposing work on an open source guidebook for scientific cloud computing infrastructure. I don’t have an…

---

## [Unable to sign up for Pangeo](https://discourse.pangeo.io/t/unable-to-sign-up-for-pangeo/3974)

<div class="topic-metadata">

**Author:** [@Zhangxiangyu98](https://discourse.pangeo.io/u/Zhangxiangyu98)\
**Replies:** 7\
**Last updated:** [February 17, 2025, 8:12pm UTC](https://discourse.pangeo.io/t/unable-to-sign-up-for-pangeo/3974 "2025-02-17T20:12:22Z")

</div>

Hi! I’m interested in signing up for Pangeo, but it’s been a while since I filled out the form, and I haven’t received any updates. Has the signup process moved to a different link? I would appreciate any information you…

---

## [How to sign up for an account of Pangeo?](https://discourse.pangeo.io/t/how-to-sign-up-for-an-account-of-pangeo/4904)

<div class="topic-metadata">

**Author:** [@huanglongyu](https://discourse.pangeo.io/u/huanglongyu)\
**Replies:** 1\
**Last updated:** [February 17, 2025, 8:11pm UTC](https://discourse.pangeo.io/t/how-to-sign-up-for-an-account-of-pangeo/4904 "2025-02-17T20:11:57Z")

</div>

Hi everyone, I am a new and want to learn how to use pangeo. I have filled out google form for several days, but I still receive the reply. How should I do for the next?

---

## [Seeking advice on file format for an API](https://discourse.pangeo.io/t/seeking-advice-on-file-format-for-an-api/4883)

<div class="topic-metadata">

**Author:** [@ags\_pangeo](https://discourse.pangeo.io/u/ags_pangeo)\
**Replies:** 2\
**Last updated:** [February 10, 2025, 5:20pm UTC](https://discourse.pangeo.io/t/seeking-advice-on-file-format-for-an-api/4883 "2025-02-10T17:20:20Z")

</div>

I’m building out an internal API that will serve temperature data at various altitudes on a predetermined grid (x deg lat x y deg long x z km alt). I have data at cadence of n minutes for this grid. I’ve worked with thi…

---

## [Can't log in to Pangeo Google Cloud hub](https://discourse.pangeo.io/t/cant-log-in-to-pangeo-google-cloud-hub/4208)

<div class="topic-metadata">

**Author:** [@cspencerjones](https://discourse.pangeo.io/u/cspencerjones)\
**Replies:** 9\
**Last updated:** [December 18, 2024, 9:59am UTC](https://discourse.pangeo.io/t/cant-log-in-to-pangeo-google-cloud-hub/4208 "2024-12-18T09:59:42Z")

</div>

I’ve lost access to the pangeo Google Cloud hub. When I try to access it I get the message “Looks like you have NOT been added to the list of allowed users for this hub. Please contact the hub administrators.” I understa…

---

## [Converting Weather Radar Raw Data into Analysis-Ready Cloud-Optimized (ARCO) Datasets](https://discourse.pangeo.io/t/converting-weather-radar-raw-data-into-analysis-ready-cloud-optimized-arco-datasets/4562)

<div class="topic-metadata">

**Author:** [@aladinor](https://discourse.pangeo.io/u/aladinor)\
**Replies:** 6\
**Last updated:** [October 17, 2024, 1:54am UTC](https://discourse.pangeo.io/t/converting-weather-radar-raw-data-into-analysis-ready-cloud-optimized-arco-datasets/4562 "2024-10-17T01:54:34Z")

</div>

Hi Pangeo Community, I’m excited to share a work-in-progress project called raw2zarr, a Python package developed to convert weather radar raw data into an Analysis-Ready Cloud-Optimized (ARCO) format. This project aims …

---

## [Icechunk: A new cloud-native transactional storage engine for Zarr](https://discourse.pangeo.io/t/icechunk-a-new-cloud-native-transactional-storage-engine-for-zarr/4610)

<div class="topic-metadata">

**Author:** [@rabernat](https://discourse.pangeo.io/u/rabernat)\
**Replies:** 1\
**Last updated:** [October 16, 2024, 1:42pm UTC](https://discourse.pangeo.io/t/icechunk-a-new-cloud-native-transactional-storage-engine-for-zarr/4610 "2024-10-16T13:42:27Z")

</div>

Hi everybody! Our team at Earthmover is excited to announce a brand new open source project called Icechunk. Here’s our launch blog post: Icechunk is an open-source library and specification for the storage of multid…

---

## [Seeking Advice on Optimizing Pangeo Workflows](https://discourse.pangeo.io/t/seeking-advice-on-optimizing-pangeo-workflows/4453)

<div class="topic-metadata">

**Author:** [@ryanrichards](https://discourse.pangeo.io/u/ryanrichards)\
**Replies:** 0\
**Last updated:** [August 27, 2024, 6:39am UTC](https://discourse.pangeo.io/t/seeking-advice-on-optimizing-pangeo-workflows/4453 "2024-08-27T06:39:11Z")

</div>

Hi everyone, I hope you’re all doing well! I’m currently working on some projects using Pangeo, and I’m looking for some tips and best practices to optimize workflows and improve performance. Whether it’s about handlin…

---

## [Advice for scalable raster-vector extraction?](https://discourse.pangeo.io/t/advice-for-scalable-raster-vector-extraction/4129)

<div class="topic-metadata">

**Author:** [@cboettig](https://discourse.pangeo.io/u/cboettig)\
**Replies:** 24\
**Last updated:** [May 15, 2024, 4:21am UTC](https://discourse.pangeo.io/t/advice-for-scalable-raster-vector-extraction/4129 "2024-05-15T04:21:33Z")

</div>

I’m trying to use rioxarray+dask to compute the mean value inside each polygon given a raster layer (‘zonal statistics’) following the advice linked in the rioxarray docs (Example - Zonal Statistics — geocube 0.5.1 docum…

---

## [Tool for validating geo data/services moved to the cloud?](https://discourse.pangeo.io/t/tool-for-validating-geo-data-services-moved-to-the-cloud/4118)

<div class="topic-metadata">

**Author:** [@briannapagan](https://discourse.pangeo.io/u/briannapagan)\
**Replies:** 6\
**Last updated:** [March 22, 2024, 7:03pm UTC](https://discourse.pangeo.io/t/tool-for-validating-geo-data-services-moved-to-the-cloud/4118 "2024-03-22T19:03:54Z")

</div>

At the data center I work at within NASA, and I am sure throughout the other NASA data centers, we either have or are currently developing lots of tooling that is validating data once it is moved to either the cloud, mov…

---

## [No longer able to set \`dask\_gateway.Gateway.cluster\_options()\` manually from user-end](https://discourse.pangeo.io/t/no-longer-able-to-set-dask-gateway-gateway-cluster-options-manually-from-user-end/3851)

<div class="topic-metadata">

**Author:** [@ofk123](https://discourse.pangeo.io/u/ofk123)\
**Replies:** 5\
**Last updated:** [February 6, 2024, 1:53pm UTC](https://discourse.pangeo.io/t/no-longer-able-to-set-dask-gateway-gateway-cluster-options-manually-from-user-end/3851 "2024-02-06T13:53:47Z")

</div>

Looking for a way around this. Usually from dask\_gateway import Gateway g = Gateway() options = g.cluster\_options() followed by options.worker\_cores = 1 works from the local hub. (Really appreciate this). This seem…

---

## [Cloud use cases](https://discourse.pangeo.io/t/cloud-use-cases/4007)

<div class="topic-metadata">

**Author:** [@betolink](https://discourse.pangeo.io/u/betolink)\
**Replies:** 0\
**Last updated:** [February 1, 2024, 7:10pm UTC](https://discourse.pangeo.io/t/cloud-use-cases/4007 "2024-02-01T19:10:48Z")

</div>

We are working on a cloud cookbook for Openscapes and we would like to include some examples of cases when the cloud was not the solution (cost? complexity?) if you know or work with workloads that are not meant for the …

---

## [Blog Post: How we’re making it easier to work with open climate data](https://discourse.pangeo.io/t/blog-post-how-we-re-making-it-easier-to-work-with-open-climate-data/3960)

<div class="topic-metadata">

**Author:** [@norlandrhagen](https://discourse.pangeo.io/u/norlandrhagen)\
**Replies:** 0\
**Last updated:** [January 9, 2024, 10:21pm UTC](https://discourse.pangeo.io/t/blog-post-how-we-re-making-it-easier-to-work-with-open-climate-data/3960 "2024-01-09T22:21:00Z")

</div>

Hi all! I wanted to share a recent blog post we wrote at CarbonPlan. We used a bunch of tools in the Pangeo Ecosystem, such as Kerchunk, Xarray-Datatree and Coiled/Dask to speed-up part of our work on modeling extreme h…

---

## [Passing gdal config to rioxarray functions?](https://discourse.pangeo.io/t/passing-gdal-config-to-rioxarray-functions/3936)

<div class="topic-metadata">

**Author:** [@cboettig](https://discourse.pangeo.io/u/cboettig)\
**Replies:** 2\
**Last updated:** [December 22, 2023, 9:02pm UTC](https://discourse.pangeo.io/t/passing-gdal-config-to-rioxarray-functions/3936 "2023-12-22T21:02:29Z")

</div>

I’m unclear how to go about passing GDAL configuration to rioxarray functions. For instance, I’d like to use xarray.open\_mfdataset to open up a list of URLs lazily by leveraging the GDAL virtual filesystem. If I first …

---

## [Processing Terabyte-Scale NASA Cloud Datasets with Coiled](https://discourse.pangeo.io/t/processing-terabyte-scale-nasa-cloud-datasets-with-coiled/3825)

<div class="topic-metadata">

**Author:** [@jrbourbeau](https://discourse.pangeo.io/u/jrbourbeau)\
**Replies:** 1\
**Last updated:** [November 27, 2023, 9:37pm UTC](https://discourse.pangeo.io/t/processing-terabyte-scale-nasa-cloud-datasets-with-coiled/3825 "2023-11-27T21:37:04Z")

</div>

Hi All, I’ve been working with groups that process NASA data (folks like @asteiker @betolink @andypbarrett @jinbow) to help move their workloads to the cloud. It turns out a common pattern for lots of workloads is a big…

---

## [Pangeo Forge bakeries](https://discourse.pangeo.io/t/pangeo-forge-bakeries/3630)

<div class="topic-metadata">

**Author:** [@matbro](https://discourse.pangeo.io/u/matbro)\
**Replies:** 21\
**Last updated:** [October 19, 2023, 3:14pm UTC](https://discourse.pangeo.io/t/pangeo-forge-bakeries/3630 "2023-10-19T15:14:03Z")

</div>

Hiya amazing Pangeo folk, I’m working at UKCEH (UK Centre for Ecology and Hydrology, quite a mouthful…!) and a group of us are getting to grips with object/cloud storage and parallel cloud compute of big/huge datasets. A…

---

## [Introducing Xee](https://discourse.pangeo.io/t/introducing-xee/3736)

<div class="topic-metadata">

**Author:** [@alxmrs](https://discourse.pangeo.io/u/alxmrs)\
**Replies:** 3\
**Last updated:** [October 10, 2023, 4:11pm UTC](https://discourse.pangeo.io/t/introducing-xee/3736 "2023-10-10T16:11:17Z")

</div>

Hey Pangeo Community! Today is my 31st birthday. The big birthday gift I’m giving to myself is the release of a project I’ve been working on for some time: Xee. Xee is an Xarray extension for Google Earth Engine. My …

---

## [Dask Kubernetes Setup Issue](https://discourse.pangeo.io/t/dask-kubernetes-setup-issue/3730)

<div class="topic-metadata">

**Author:** [@lei](https://discourse.pangeo.io/u/lei)\
**Replies:** 2\
**Last updated:** [October 5, 2023, 12:34am UTC](https://discourse.pangeo.io/t/dask-kubernetes-setup-issue/3730 "2023-10-05T00:34:23Z")

</div>

Cloud Dask Kubernetes Setup Issue Hello everyone! I’ve been a user of Pangeo Cloud and its distributed computing capabilities through Dask. The last time I used it (about a year ago), everything was working perfectly. …

---

## [Data Provider Strategies for Hosting Different Cloud-Optimized Data Formats](https://discourse.pangeo.io/t/data-provider-strategies-for-hosting-different-cloud-optimized-data-formats/3588)

<div class="topic-metadata">

**Author:** [@briannapagan](https://discourse.pangeo.io/u/briannapagan)\
**Replies:** 8\
**Last updated:** [October 2, 2023, 12:38pm UTC](https://discourse.pangeo.io/t/data-provider-strategies-for-hosting-different-cloud-optimized-data-formats/3588 "2023-10-02T12:38:22Z")

</div>

Thinking out-loud after attending ESIP and the USGEO Data Management Working Cloud Cloud Recommendations meeting this week - the question of which is the ‘best’ cloud optimized data format keeps coming up. It’s repeated …

---

## [Blog post: Processing a 250 TB dataset with Xarray, Dask, and Coiled](https://discourse.pangeo.io/t/blog-post-processing-a-250-tb-dataset-with-xarray-dask-and-coiled/3668)

<div class="topic-metadata">

**Author:** [@scharlottej13](https://discourse.pangeo.io/u/scharlottej13)\
**Replies:** 0\
**Last updated:** [September 5, 2023, 4:57pm UTC](https://discourse.pangeo.io/t/blog-post-processing-a-250-tb-dataset-with-xarray-dask-and-coiled/3668 "2023-09-05T16:57:26Z")

</div>

Hi! I wanted to share a blog post building off of @dcherian’s work to process the national water model retrospective dataset: Processing a 250 TB dataset with Coiled, Dask, and Xarray | by Sarah Johnson | Coiled | Sep, …

---

## [LLC4320 SSU/SSV data at Pangeo cloud mismatched](https://discourse.pangeo.io/t/llc4320-ssu-ssv-data-at-pangeo-cloud-mismatched/3661)

<div class="topic-metadata">

**Author:** [@Isotr0py](https://discourse.pangeo.io/u/Isotr0py)\
**Replies:** 5\
**Last updated:** [September 4, 2023, 4:45pm UTC](https://discourse.pangeo.io/t/llc4320-ssu-ssv-data-at-pangeo-cloud-mismatched/3661 "2023-09-04T16:45:52Z")

</div>

Since access to LLC4320 from ECCO portal is too slow, I’m using LLC4320 data at pangeo cloud for my research. However, it seems that the SSU/SSV variable of LLC4320 is wrongly mismatched at Pacific Ocean. U/V from Pang…

---

## [Question about how to best store self-written functions, when using dask\_gateway](https://discourse.pangeo.io/t/question-about-how-to-best-store-self-written-functions-when-using-dask-gateway/3651)

<div class="topic-metadata">

**Author:** [@ofk123](https://discourse.pangeo.io/u/ofk123)\
**Replies:** 2\
**Last updated:** [August 29, 2023, 3:12pm UTC](https://discourse.pangeo.io/t/question-about-how-to-best-store-self-written-functions-when-using-dask-gateway/3651 "2023-08-29T15:12:06Z")

</div>

I have several notebooks, using LocalCluster, and Gateway in a couple of them, and reuse some functions across notebooks. If a dask.delayed-object is defined to use a function imported from a .py-file in the /home/jovyan…

---

## [Best practice to store and load data-columns of equal-length from GCS (data not on a regular grid)](https://discourse.pangeo.io/t/best-practice-to-store-and-load-data-columns-of-equal-length-from-gcs-data-not-on-a-regular-grid/3591)

<div class="topic-metadata">

**Author:** [@ofk123](https://discourse.pangeo.io/u/ofk123)\
**Replies:** 1\
**Last updated:** [August 28, 2023, 12:39am UTC](https://discourse.pangeo.io/t/best-practice-to-store-and-load-data-columns-of-equal-length-from-gcs-data-not-on-a-regular-grid/3591 "2023-08-28T00:39:42Z")

</div>

Hi! I thought to ask on here, if am I missing out on even faster ways to load data. From several zarr-stores that each store 5 data-columns of equal length, I want to load 4 of them into dask-worker-memory (columnlengt…

---

## [Statement of Need: Integrating JupyterBook and JupyterHubs via CI](https://discourse.pangeo.io/t/statement-of-need-integrating-jupyterbook-and-jupyterhubs-via-ci/2705)

<div class="topic-metadata">

**Author:** [@rabernat](https://discourse.pangeo.io/u/rabernat)\
**Replies:** 17\
**Last updated:** [August 25, 2023, 11:08pm UTC](https://discourse.pangeo.io/t/statement-of-need-integrating-jupyterbook-and-jupyterhubs-via-ci/2705 "2023-08-25T23:08:59Z")

</div>

Context This post is an action item that emerged from a recent meeting organized by @maxrjones and attended by @jbednar, @clyne, @ktyle, and myself. The topic was how to integrate and harmonize the different “notebook ex…

[Next page](https://discourse.pangeo.io/c/cloud/7.md?page=1)