How to see all the public repos in arraylake programmatically?

Data
1

In Old format, no problem!: Cloud-optimizing the GOES-16 archive as Virtual Zarr | Earthmover it shows how to easily access:

import arraylake as al
import xarray as xr

client = al.Client()
client.login()

repo = client.get_repo("earthmover-public/goes-16")
session = repo.readonly_session("main")
ds = xr.open_zarr(session.store, group="ABI-L2-MCMIPF/post-2023-04-19")
ds

However, I’m curious whether there are other repos besides goes-16, so I try:

import arraylake as al

client = al.Client()
client.login()

client.list_repos("earthmover-public")

But I get permission error.

Is there a way to list all the public datasets on Marketplace | Arraylake?

3 Likes
2

Thanks for the feedback here @ahuang11! I agree this is a bit confusing. earthmover-public is just one of the organizations within the platform. There is currently no public view of an organization’s repos for non-org members.

There is currently no Python Client or CLI support for exploring the marketplace. This is a real gap.

However, it IS supported by the Arraylake MCP, which has been my main way of interacting with…well, just about everything! :laughing:

claude> list all the listing on the Earthmover data marketplace

  Called Arraylake - Production

The results are large — let me read through the file to extract the listings.

  Ran 4 shell commands, searched memories

Here are all 53 listings on the Earthmover Data Marketplace, grouped by provider:

---
ClimateUncertaintyLab (4)

- BCD-ME 1° bias-corrected (QM) temperature time series — free
- BCD-ME 0.25° bias-corrected (QM), downscaled temperature statistics — free
- BCD-ME 0.25° bias-corrected (QDM) and downscaled temperature statistics — free
- BCD-ME 1° bias-corrected (QDM) temperature time series — free

National Oceanography Centre (1)

- NOC Near-Present Day 1/12° Ocean Sea-Ice Hindcast — free

OGS (1)

- ARCO-OCEAN — free

Spire (2)

- saifs-s2s-hindcast — free
- Spire AI-S2S — paid

beyond-weather (1)

- ECMWF AIFS global long-range wind model — paid

Brightband (6)

- ECMWF IFS 15-day Forecast (open) — free
- ECMWF IFS Initial Conditions (open) — free
- ECMWF IFS Initial Conditions (low latency) — paid
- ECMWF IFS 15-day Forecast (low latency) — paid
- MLWP Forecast Archive — free
- ECMWF IFS Ensemble Initial Conditions — free

CarbonPlan (1)

- open-climate-risk — free

CTrees (1)

- Global aboveground biomass (AGB), 100m — free

Dynamical (11)

- ECMWF AIFS ENS forecast — free
- DWD ICON-EU forecast, 5 day — free
- ECMWF AIFS Single forecast — free
- NOAA MRMS CONUS analysis, hourly — free
- NOAA GFS analysis — free
- NOAA HRRR analysis — free
- NOAA HRRR forecast, 48 hour — free
- ECMWF IFS ENS forecast, 15 day, 0.25 degree — free
- NOAA GFS forecast — free
- NOAA GEFS analysis — free
- NOAA GEFS forecast, 35 day — free

Eagle Rock / Cal-Adapt (1)

- Cal-Adapt: Data Catalog (WRF) — free

Earthmover Public (4)

- NOAA GOES-16 MCMIPF — free
- ERA5 — free
- ERA5 (Daily Updates) — paid
- ERA5 Surface — free

OU SubC (1)

- SubC Forecasts — free

Planette (1)

- Joro S2S Global Forecasts — paid

Prescient Weather (4)

- Historical ECMWF Sub-Seasonal Ensemble Archive — paid
- Historical ECMWF Sub-Seasonal Reforecast Archive — paid
- Historical ECMWF HRES Archive — paid
- Historical ECMWF ENS Archive — paid

Reflective (2)

- GeoMIP MIROC-ES2H G6-1.5K Simulations — free
- GeoMIP CESM2-WACCM G6-1.5K Simulations — free

Sylvera (2)

- Biomass Atlas — paid
- Biomass Atlas Sample — free

Taylor Geospatial (4)

- Meta Canopy Height Maps v2 — free
- FTW Global Sentinel-2 Mosaics — free
- Global Fields of The World (FTW) — free
- AlphaEarth Foundations Satellite Embedding Dataset — free

UW-Madison ALIVE (1)

- ALIVE_hourly — free

Zeus AI (5)

- GEO-IR: Merged Geostationary Infrared (Hourly Updates) — paid
- GEO-IR: Merged Geostationary Infrared — free
- GEO-GEO Stereo Wind Retrievals — free
- Helios Global Solar Irradiance Forecast - 6 hour / 6 km — paid
- Helios Solar Irradiance - Historical (Americas) — free

---
53 total listings: 38 free, 15 paid. The marketplace skews heavily toward weather/climate data, with Dynamical being the largest single provider (11 listings).

Our action item from this is to add similar capabilities to the CLI and Python SDK.

1 Like
3

Yes, if it can be added to the Python SDK that’d be awesome!