Zonal Coastal FIM Dockerized Lambda [Atlantic Gulf ANA and MRF NBM 10 Day Max Only]

[GeorgeRought-NOAA]

This is the first implementation of the new barycentric method for Coastal FIM developed by Arash Rad and Mike Johnson. The zonal fim repository can be found here: https://github.com/owp-spatial/zonal-fim/tree/main.

This initial test is only for the Atlantic gulf, for ANA and MRF NBM 10 Day Max forecasts, and limited to just inundation extent.

Additions

• hv-vpp-ti-process-zonal-coastal-fim-dev step function
• hv-vpp-dev-zonal-coastal-fim-georgerought docker image based lambda function
• hv-zonal-coastal-fim-test elastic file system for DuckDB storage
• hv_vpp_zonal_coastal_fim_dev elastic container repository (Image tag 2.2 is latest as of writing this)
• zonal fim required components from repository
• ana_zonal_coastal_inundation_extent_dev_noaa experimental service
• mrf_nbm_10day_max_zonal_coastal_inundation_extent_noaa experimental service
• new supporting tables in viz db ingest/publish schemas

Removals

• Commented out 3 day, 5 day, psurge options from old coastal MRF NBM 10 Day service to isolate testing. Intent is to roll out additional services if zonal coastal fim initial deployment is promising.

Changes

• Updated the ANA and MRF NBM 10 Day initialize pipeline .yml files for coastal FIM to work with zonal coastal FIM.
• Updated initialize pipeline lambda_function.py to uncomment the aforementioned netcdf types
• Added new "zonal_coastal" fim_type to the Coastal vs Inland Inundation Choice State in hv-vpp-GeorgeRought-viz-pipeline

Testing

1. The docker-based lambda function has been tested in isolation on test event json payloads on a netcdf from NOMADS. It has also been added to a separate version of the viz processing pipeline step function as a nested step function to test the zonal fim step function within the context of the main pipeline.
2. A comparison to SCHISM FIM was made by processing a netcdf with zonal coastal FIM that SCHISM FIM has already processed. The SCHISM FIM results were stored on S3, and ingested into a web map alongside the zonal coastal FIM results for comparison. The web map can be found here: https://eks-maps-testing.water.noaa.gov/server/home/item.html?id=fadea9a5a6f14db194d07712a7aaa697

Screenshots

Notes

• Names listed above are temporary development names and will be standardized in deployment.

Todos

Checklist

• PR has an informative and human-readable title
• Changes are limited to a single goal (no scope creep)
• Code can be automatically merged (no conflicts)
• Code follows project standards (link if applicable)
• Passes all existing automated tests
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future todos are captured in comments
• Visually tested in supported browsers and devices (see checklist below 👇)
• Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
• Reviewers requested with the Reviewers tool ➡️

Testing checklist

Target Environment support

• Windows
• Linux
• Browser

Accessibility

• Keyboard friendly
• Screen reader friendly

Other

• Is useable without CSS
• Is useable without JS
• Flexible from small to large screens
• No linting errors or warnings
• JavaScript tests are passing

[groutr]

This might be a more efficient way to construct the dataframe of shapes.

import operator
from shapely.geometry import shape
#------------------------
image_depth = src.read(1, out_dtype='float32', masked=True)
masked = image_depth > 0
raster_shapes = map(operator.itemgetter(0), features.shapes(masked.data, mask=~masked.mask, transform=transform))
gdf_polygons = gpd.GeoDataFrame(crs=raster_crs, geometry=list(map(shape, raster_shapes)))

[groutr]

The line masked = image_depth > 0 above should read masked = np.ma.masked_less_equal(image_depth, 0)

[shawncrawley]

A few more things and we should be good to go!

All resolved