Filtering out clouds in Sentinel 2

Data
1

Following my related post, I’m trying to use data from S2 and filter out cloudy pixels.

There’s a level 1 cloud mask product. But this means loading in something else as well as your desired Level 2 product. There’s a scene classification map (SCL) in the L2 product, which seems to include such information. But for interpreting its values, I’ve only seen references in this sentinel hub FAQ article and, for example, this gis.stackexchange post. If I go to the ESA Sentinel MSI page, I go round and round in circles and utterly fail to find a canonical description of the values in the SCL asset. I’ve found references to a paper (J. Louis, A. Charantonis & B. Berthelot, “Cloud Detection for Sentinel-2”, Proceedings of ESA Living Planet Symposium (2010)), which I’ve failed to find, but searching did lead me to this paper, which has a bar chart that clearly knows the labels for the SCL codes.

Hallelujah, I’ve found something vaguely official in the Sen2Cor Configuration and User Manual and S2 MPC Level 2A Input Output Data Definition documents that describe the meaning of the 12 values. But this was harder to find than the plans announcing the demolition of Arthur Dent’s house! I can only think I’m missing some fundamental knowledge of the data.
How hard can this be?

Also, there’s reference in the doc to probability of cloud and probability of snow and it implies these are L2 products. But are they not included in the COGs, and you only get them if you run sen2cor?

By the way, whoever decided to name remote/distributed computing as “cloud” has done nothing to help web searches on this topic! Just saying. :wink:

2

scm_classes

I mean. This isn’t much to ask for …

3

I am sorry it was not available anywhere else! That is a picture from my internship 14 years ago!
Happy you ended up finding what you needed.

3 Likes
4

We did make an algorithm that could provide probability of clouds, but I think both approaches would, nowadays benefit from a re-implementation given the improvement of machine learning techniques.

5

The Sentinel-2 Australian bare earth map has some work along those lines.

https://ecat.ga.gov.au/geonetwork/srv/eng/catalog.search#/metadata/146125

Look for Dale Roberts work.

6

Table 4 in Baetens et al. (2019) is also helpful to decide on which values to include in a valid/not-valid binary class. E.g. given an xarray.Dataset including the SCL layer as a data variable:

import xarray as xr

valid = ((ds.SCL == 2) |  # dark area pixels
         (ds.SCL > 3) &   # vegetation, bare soils, water, unclassified
         (ds.SCL <= 7) |
         (ds.SCL == 11)   # snow/ice
         )

ds_masked = xr.where(valid, ds, 0)
7

As an aside, if the CF-compliant attributes are set you have access to (possibly) nicer syntax: Flag Variables - cf_xarray documentation

1 Like