ECCC Grib Consumer

[brettedw]

Closes #5021

Main Changes

• Adds aio-pika as amqp consumer
• Adds new deployment for an eccc-consumer
  • wps-weather now has it's own workflow that runs conditionally on changes to wps-weather package
  • Configurable to change weather models and run hours
  • Currently consuming/downloading 00/12 runs of GDPS/RDPS
• Runs wps-weather tests on pull request
  • Does not include these results in code coverage
• Adds pagination to S3Client with tests
• Includes cleanup cronjob that runs once per day and is configurable via env vars (currently set to 7 days)
  • Data can grow quickly, a single filtered run of RDPS & GDPS is ~3GB
  • Currently storing data in wxmap test bucket

Other

• Hooks up wps-wf1 tests to run on pull request + codecov
• Fixes a couple failing wf1 tests

Test Links:

Landing Page
MoreCast
Percentile Calculator
C-Haines
FireCalc
FireCalc bookmark
Auto Spatial Advisory (ASA)
HFI Calculator
SFMS Insights
Fire Watch

[codecov]

Codecov Report

❌ Patch coverage is 66.66667% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.81%. Comparing base (75f2eed) to head (b3522d0).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...kages/wps-shared/src/wps_shared/utils/s3_client.py	67.34%	11 Missing and 5 partials ⚠️
backend/packages/wps-api/src/app/jobs/rdps_sfms.py	25.00%	9 Missing ⚠️
...end/packages/wps-shared/src/wps_shared/utils/s3.py	89.47%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5098      +/-   ##
==========================================
- Coverage   75.20%   69.81%   -5.40%     
==========================================
  Files         592      410     -182     
  Lines       28770    17856   -10914     
  Branches     2108     2000     -108     
==========================================
- Hits        21637    12466    -9171     
+ Misses       6493     4765    -1728     
+ Partials      640      625      -15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[dgboss]

Amazing work!

[conbrad]

Looking really good!

[sonarqubecloud]

Quality Gate failed

Failed conditions
1 Security Hotspot

See analysis details on SonarQube Cloud

[conbrad]

Nice work!!!