Skip to content

Conversation

@robertplatt-mo
Copy link

@robertplatt-mo robertplatt-mo commented Jan 26, 2026

EPPT-2991 Add _zero_mask_values method to CubeCombiner with preserve_mask_values flag.

Description

During development of https://metoffice.atlassian.net/browse/EPPT-2991 we identified a need for mask handling where the masked data could be treated as zero to enable calculations to take place without the masked area persisting into downstream datasets. Adding the _zero_mask_values method to CubeCombiner enables this behaviour when the preserve_mask_values option is set to False.

Testing:

Duplicated the test_with_mask test to create test_process_add_preserve_mask_true and test_process_add_preserve_mask_false which test the existing behaviour and then the _zero_mask_values behaviour with the add operator. Repeated the above for the multiply operator.

Upon realising that the existing masked_add operator currently retains the mask wherever the input masks are overlapping, added a test for masked_add to demonstrate this behaviour and for the Test_process tests to demonstrate the opposite when _zero_mask_values is used.

  • Ran tests and they passed OK
  • Added new tests for the new feature(s)

CLA

  • If a new developer, signed up to CLA

@robertplatt-mo robertplatt-mo marked this pull request as draft January 26, 2026 10:24
@robertplatt-mo robertplatt-mo force-pushed the EPPT2991_Add_option_to_cube_comebine_method_to_zero_masked_datasets branch from 98bb47c to dc565d3 Compare January 26, 2026 11:59
@cpelley
Copy link
Contributor

cpelley commented Jan 26, 2026

CI failure addressed in #2287

@robertplatt-mo robertplatt-mo force-pushed the EPPT2991_Add_option_to_cube_comebine_method_to_zero_masked_datasets branch from dc565d3 to 4113a03 Compare January 26, 2026 12:10
@robertplatt-mo robertplatt-mo self-assigned this Jan 26, 2026
Warning:
The masked_add function will retain the mask wherever the input masks
are overlapping.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added these warnings because I did some digging in to the masked_add function to see whether it was a distinct method that still requires it's own function or whether it can be handled under the _zero_mask_values method when preserve_mask_values=False. The methods do differ but I'm not entirely convinced this is intentional.

I suspect that this overlapping issue wasn't a concern in the original work (masking land and sea) because when masks are created to distinguish land and sea they are most likely created from the same source coastline (so overlaps should not happen). That said I'm struggling to think of a use case for a scenario where we retain only overlapping masks rather than allowing them to be visible as zeros. Should we be raising a task to deprecate this function and use the new method instead or is there a use case I'm unaware of that can be included in the docstrings?

The background to the masked_add function was task EPPT1305 and the work was on behalf of the stability adjusted wind gust wf. According to page 6 of the linked workflow plan that work may ultimately not have been needed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Anzerkhan27 Did you have a position on this? I'm primarily wondering if the masked_add behaviour is in use anywhere and/or whether it could/should be deprecated in favour of the new approach which handles masked values on any of the operations.

@robertplatt-mo robertplatt-mo marked this pull request as ready for review January 26, 2026 12:55
Copy link
Contributor

@Anzerkhan27 Anzerkhan27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve with one minor type hint correction needed. The implementation is clean, well-tested, and maintains backward compatibility with the default preserve_mask_values=True

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants