# Accessing the new Element84 S2 (L1C) stac catalogue

**URL:** <https://discourse.pangeo.io/t/accessing-the-new-element84-s2-l1c-stac-catalogue/3499>\
**Category:** Data\
**Created:** [June 7, 2023, 8:35pm UTC](https://discourse.pangeo.io/t/accessing-the-new-element84-s2-l1c-stac-catalogue/3499 "2023-06-07T20:35:24Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![akpetty](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/akpetty/32/806_2.png) [@akpetty](https://discourse.pangeo.io/u/akpetty)\
**Post date:** [June 7, 2023, 8:35pm UTC](https://discourse.pangeo.io/t/accessing-the-new-element84-s2-l1c-stac-catalogue/3499/1 "2023-06-07T20:35:24Z")

</div>

Hey folks, I was excited to see the new Element84 Sentinel-2 STAC catalogue which now includes L1C data: [https://www.element84.com/earth-search/](https://www.element84.com/earth-search/). I was following this [guide](https://odc-stac.readthedocs.io/en/latest/notebooks/stac-load-e84-aws.html) on loading data from the previous (v0) catalogue which works great, but when I upgrade this to try and work with the new v1 catalogue I’m getting Access Denied issues on compute. I’ve tried various AWS configs (configure\_rio) but without joy so wondering if anyone else has taken a look yet!?

---

<div class="post-metadata">

**Author:** ![philvarner](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/philvarner/32/2289_2.png) [@philvarner](https://discourse.pangeo.io/u/philvarner)\
**Post date:** [June 8, 2023, 7:04pm UTC](https://discourse.pangeo.io/t/accessing-the-new-element84-s2-l1c-stac-catalogue/3499/2 "2023-06-08T19:04:46Z")

</div>

This is likely because all of the asset URLs are incorrect for the L1C Items – the bucket was accidentally set to the sentinel-s2-l2a bucket instead of the sentinel-s2-l1c bucket. For example, this asset href on the first result I got: “s3://sentinel-s2-l2a/tiles/33/X/WB/2023/6/8/0/B02.jp2” should be “s3://sentinel-s2-l1c/tiles/33/X/WB/2023/6/8/0/B02.jp2”. The workaround is to rewrite the URL to fix the bucket name if you can. We’re working on fixing this, but it’s likely several weeks out at least.

---

<div class="post-metadata">

**Author:** ![gadomski](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/gadomski/32/995_2.png) [@gadomski](https://discourse.pangeo.io/u/gadomski)\
**Post date:** [June 8, 2023, 7:40pm UTC](https://discourse.pangeo.io/t/accessing-the-new-element84-s2-l1c-stac-catalogue/3499/3 "2023-06-08T19:40:26Z")

</div>

Here’s an issue that is likely part of the OP’s problem: [Collection sentinel-2-l2c items have asset hrefs that reference sentinel-s2-l2a bucket · Issue #3 · Element84/earth-search · GitHub](https://github.com/Element84/earth-search/issues/3)

(note: @philvarner pointed me to this issue, so I just thought I’d link it in to this conversation as well)

---

<div class="post-metadata">

**Author:** ![akpetty](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/akpetty/32/806_2.png) [@akpetty](https://discourse.pangeo.io/u/akpetty)\
**Post date:** [June 15, 2023, 6:12pm UTC](https://discourse.pangeo.io/t/accessing-the-new-element84-s2-l1c-stac-catalogue/3499/4 "2023-06-15T18:12:02Z")

</div>

Ah gotcha, yep that looks to be the likely culprit! Thanks for that. Interesting that the earthsearch:s3\_path is still listed as l1c though?

[EDIT] I’m not in much of a rush on this so I could just wait for the change and maybe play with L2A instead for now, but I did try and access with a simple replace string approach but still getting access denied:

import rasterio  
with rasterio.open(S2\_items[0][‘assets’][‘red’][‘href’].replace(“l2a”, “l1c” )) as dataset:  
rasterio.plot.show(dataset)

---

<div class="post-metadata">

**Author:** ![weiji14](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/weiji14/32/389_2.png) [@weiji14](https://discourse.pangeo.io/u/weiji14)\
**Post date:** [June 16, 2023, 3:26am UTC](https://discourse.pangeo.io/t/accessing-the-new-element84-s2-l1c-stac-catalogue/3499/5 "2023-06-16T03:26:49Z")

</div>

Hi @akpetty! Here’s some code that might help. Some of my team members were working on L1C data recently (you should know Lilly 😉) and was facing the same issue, so you’re in luck!

First, import some libraries

```python
import os

import pystac_client
import rioxarray
import stackstac

```

Next, set up the STAC query and patch the href URLs of the STAC assets. This is an example using [stackstac](https://github.com/gjoseph92/stackstac):

```python
client = pystac_client.Client.open(url="https://earth-search.aws.element84.com/v1/")
search = client.search(
    collections="sentinel-2-l1c",
    bbox=[-20.7, 64.5, -19.5, 64.8], # xmin, ymin, xmax, ymax
    datetime="2023-02-01/2023-02-28",
)
stac_items = search.items()
stac_item = next(stac_items) # <Item id=S2B_27WWM_20230228_0_L1C>

for stac_asset in stac_item.assets.values():
    stac_asset.href = stac_asset.href.replace(
        "s3://sentinel-s2-l2a/", "s3://sentinel-s2-l1c/"
    )

```

At this point, you should be able to read the metadata (even without authentication).

```python
dataarray = stackstac.stack(items=stac_item, dtype="float16", resolution=10)
print(dataarray)

```

produces

```python
<xarray.DataArray 'stackstac-40b6166a40b446f90241b2283a6022d7' (time: 1,
                                                                band: 14,
                                                                y: 10980,
                                                                x: 10980)>
dask.array<fetch_raster_window, shape=(1, 14, 10980, 10980), dtype=float16, chunksize=(1, 1, 1024, 1024), chunktype=numpy.ndarray>
Coordinates: (12/39)
  * time (time) datetime64[ns] 2023-02-28T13:03:...
    id (time) <U24 'S2B_27WWM_20230228_0_L1C'
  * band (band) <U8 'blue' 'cirrus' ... 'visual'
  * x (x) float64 5e+05 5e+05 ... 6.098e+05
  * y (y) float64 7.2e+06 7.2e+06 ... 7.09e+06
    processing:software object {'sentinel2-to-stac': '0.1.0'}
    ... ...
    raster:bands (band) object [{'nodata': 0, 'data_type...
    gsd (band) object 10 60 60 10 ... 20 20 None
    common_name (band) object 'blue' 'cirrus' ... None
    center_wavelength (band) object 0.49 1.3735 ... 2.19 None
    full_width_half_max (band) object 0.098 0.075 ... 0.242 None
    epsg int64 32627
Attributes:
    spec: RasterSpec(epsg=32627, bounds=(499980, 7090200, 609780, 7200...
    crs: epsg:32627
    transform: | 10.00, 0.00, 499980.00|\n| 0.00,-10.00, 7200000.00|\n| 0.0...
    resolution: 10

```

Now this is where it gets tricky, you’ll need to set up the AWS requester pays somehow. There’s probably a couple of ways, but one way I have it setup is to edit the `~/.aws/credentials` file, and have three lines like this:

```auto
[default]
aws_access_key_id = ABCDEFGHIJKLMNOPQRST
aws_secret_access_key = MnOpQrStUvWxYz1a2B3c4D5e6f7G8h9IjKlMnOpQ

```

Now you can set some environment variables and plot the Sentinel L1C data:

```python
os.environ["AWS_REQUEST_PAYER"] = "requester"
os.environ["AWS_PROFILE"] = "default"
da_rgb = dataarray.sel(band=["red", "green", "blue"]).squeeze()[:, :100, :100] # get subset
da_rgb.astype("int").plot.imshow(rgb="band", robust=True)

```

produces

![image](https://canada1.discourse-cdn.com/flex030/uploads/pangeo/original/2X/3/3d8c259a155050dca88f6c03b9438a01d0d619f1.png)

Notes:

- I had this running on the [CryoCloud Hub](https://hub.cryointhecloud.com) at AWS us-west-2, might need to `pip install stackstac` first if running there too.
- According to [https://element84.com/blog/introducing-earth-search-v1-new-datasets-now-available](https://element84.com/blog/introducing-earth-search-v1-new-datasets-now-available), the `sentinel-s2-l1c` collection is now named `sentinel-2-l1c`, but I couldn’t get the new one to work for some reason, getting `RuntimeError: Error opening 's3://sentinel-2-l1c/tiles/27/W/WM/2023/2/28/0/B04.jp2': RasterioIOError("'/vsis3/sentinel-2-l1c/tiles/27/W/WM/2023/2/28/0/B04.jp2' does not exist in the file system, and is not recognized as a supported dataset name.")`. Might be some delay in the renaming?

---

<div class="post-metadata">

**Author:** ![akpetty](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/akpetty/32/806_2.png) [@akpetty](https://discourse.pangeo.io/u/akpetty)\
**Post date:** [June 20, 2023, 6:27pm UTC](https://discourse.pangeo.io/t/accessing-the-new-element84-s2-l1c-stac-catalogue/3499/6 "2023-06-20T18:27:26Z")

</div>

Thanks @weiji14, I had a feeling I should have reached out to you directly! I copied exactly your example and am also running on CryoCloud but unfortunately I am still getting Access Denied issues, so maybe there’s something else going on my end. It seems like my credentials are being used now at the very least (when I tried a fake ID I got a different error!) so I’ll keep exploring…

---

<div class="post-metadata">

**Author:** ![weiji14](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/weiji14/32/389_2.png) [@weiji14](https://discourse.pangeo.io/u/weiji14)\
**Post date:** [June 20, 2023, 10:24pm UTC](https://discourse.pangeo.io/t/accessing-the-new-element84-s2-l1c-stac-catalogue/3499/7 "2023-06-20T22:24:16Z")

</div>

Yeah, we should catch up sometime 😃 Anyways, on the Access Denied part, I think the CryoCloud Hub has set up requester pays for the USGS Landsat S3 bucket (see [Requester pays fix needed · Issue #52 · CryoInTheCloud/hub-image · GitHub](https://github.com/CryoInTheCloud/hub-image/issues/52)), but you may need a different access key (i.e. your personal or institutional one) for this particular Element84 bucket. The documentation around this isn’t particularly good, I found [Downloading objects in Requester Pays buckets - Amazon Simple Storage Service](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ObjectsinRequesterPaysBuckets.html) which might be helpful if you want to test things out on the CLI first though if you want to make sure that your credentials work.

---

<div class="post-metadata">

**Author:** ![akpetty](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/akpetty/32/806_2.png) [@akpetty](https://discourse.pangeo.io/u/akpetty)\
**Post date:** [June 21, 2023, 2:21am UTC](https://discourse.pangeo.io/t/accessing-the-new-element84-s2-l1c-stac-catalogue/3499/8 "2023-06-21T02:21:32Z")

</div>

OK finally got it working hurrah, had to create and use a new personal access key instead of the access key created on my institutional account, so maybe our NASA admins have blocked requester\_pay buckets (or I messed up something, I’ve pinged them an email). Anyway thanks so much for your help @weiji14, couldn’t have got there without that input. Excited to show you what I’m working on once I’ve polished it up.

Definitely agree people have created some amazing resources out there to explain the stac side of all this, but that final AWS link is pretty unclear.