Shadows erroneously detected in valleys and over lakes

[olegzhelezn]

When using the QAI band created by L2 processing with standard settings, I experienced a rather common cloud shadow commission errors illustrated below. The errors are more common in mountainous areas and over water.

1. Shadow detected over lake and adjacent (very bright!) area, rather far away from the original cloud:

2. Similar, but shadow extends >8 km from the edge of the cloud:

3. Shadow covers the entire mountain valley, although in reality it is mostly bright:

Is it possible to adjust the shadow detection algorithm to minimise such occurrences?
For example, include some customisable parameters in L2 processing, limiting the maximum distance from cloud to cloud shadow? Or increasing the importance of bright pixels in areas considered as possible shadows?

[davidfrantz]

Hi Oleg,

this is a complicated matter.

Note that limiting the cast shadow range might come with unwanted side effects. The potential offset can be substantial and depends on cloud height and sun zenith.

1. Cloud base heights can be anywhere between 200m - 12000m. This can result in a wide range of possible displacements. See this quick plot, X = cloud base height [m], Y = displacement [m], black line = summer image (July), red line = winter image (January)

By restricting displacement, one could miss the shadows of high clouds, and it wouldn't be consistent across seasons. The better way would be to restrict potential cloud height (at least better seasonal consistency) - but you would still miss shadows of high clouds. I can add such a parameter if you wish.

2. in FORCE, every locally dark patch is matched with a potential shadow location if the match similarity is large enough. This was implemented to make the shadow masking more aggressive and be more sure to catch the actual shadow (producer's accuracy in favor of user's accuracy). This is different from the original Fmask. There, only the best match is used along with an early-exit strategy, and this often results in spatially incomplete shadow detections. Additionally, it has an even higher risk that the best match ends up in a body of water (if between cloud and actual shadow). Of course, there would be possibilities to include an early-stopping functionality like in the original Fmask - or to give you control over the match similarity threshold (though both bear the risk that the shadow ends up in the water instead of the actual shadow). Let me know if you would like to have that - it might take a bit to implement though.

Best,
David

[katjakowalski]

Hi David,
thank you for thinking of some potential solutions. We discussed this a bit and thought that the possibility for early exit/match similarity threshold could be the more robust fix. We also checked in the European datacube and I think it is not so widespread or at least we don't have such big effects because there are more images compared to the boreal. If it is relatively straight forward to add a potential cloud height parameter that would also be much appreciated in the meantime.
Best,
Katja