Skip to content

test(infrastructure): add Go output contract tests for root Terraform module#571

Open
bindsi wants to merge 8 commits intomainfrom
feature/go-tests
Open

test(infrastructure): add Go output contract tests for root Terraform module#571
bindsi wants to merge 8 commits intomainfrom
feature/go-tests

Conversation

@bindsi
Copy link
Copy Markdown
Member

@bindsi bindsi commented Apr 27, 2026

This PR completes the final sub-issue of the Go/Terratest umbrella (#290) by adding a fast, credential-free contract test that catches drift between the root Terraform module's declared output blocks and the typed InfraOutputs struct end-to-end tests rely on. The test runs in under one second, requires no terraform init and no Azure credentials, and uses terraform-docs to extract declared outputs at test time.

It also closes #335 by adding go to the CodeQL language matrix so the new Go code is scanned alongside JavaScript/TypeScript and Python from day one.

Description

The new contract pattern lives under infrastructure/terraform/e2e/ and has three layers. A reusable testutil package shells out to terraform-docs json to discover declared outputs, then validates them against a required-key set produced by reflection over output:"..." struct tags. A typed InfraOutputs struct in outputs.go defines that contract for the root module's 27 outputs. A thin TestTerraformOutputsContract entry point wires the two together and is the first real test in the previously placeholder e2e package.

CI wiring lives in .github/workflows/go-tests.yml: a pinned Install terraform-docs step (v0.21.0, SHA256 verified) runs between Setup Go and Run Go tests. A small developer helper, run-contract-tests.sh, provides a one-command local entry point with preflight checks for go and terraform-docs. The *.go row in .github/copilot-instructions.md now cites the helper alongside the existing lint and test commands.

Test infrastructure

The contract is intentionally required-subset-of-declared. Adding a new output to outputs.tf without updating InfraOutputs leaves the test green; removing a declared output that the struct still references fails with an actionable message naming outputs.tf. Both directions were probed during validation.

  • New testutil package exposing GetTerraformDeclaredOutputs, ValidateOutputContract, ValidateTerraformContract, and GetOutputKeysFromStruct — reused unchanged by future per-deployment contracts.
  • New InfraOutputs struct mirroring the root module's 27 outputs, each tagged with output:"<name>"; RequiredOutputKeys() extracts the tag set via reflection.
  • New TestTerraformOutputsContract in contract_test.go — replaces the placeholder e2e.go file as the package's anchor.
  • Unit tests in testutil/contract_test.go cover value/pointer receivers, empty struct, untagged struct, and contract-validation success paths so the Codecov go flag reports meaningful coverage on the helper.
  • go.mod / go.sum now require github.com/stretchr/testify v1.11.1.

CI and tooling

  • New Install terraform-docs step in go-tests.yml, pinned to v0.21.0 with explicit SHA256 verification matching the literal already used by terraform-docs-check.yml. Installs to /usr/local/bin/ via sudo mv on ubuntu-latest.
  • New executable run-contract-tests.sh with set -euo pipefail, a -v/--verbose flag, and clear preflight messages when go or terraform-docs is missing.
  • .github/copilot-instructions.md *.go Quick Reference row updated to list the new helper and call out the terraform-docs dependency.
  • .github/workflows/codeql-analysis.yml matrix gains go (closes ci(security): add Go to CodeQL language matrix #335); CodeQL autobuild discovers infrastructure/terraform/e2e/go.mod automatically.

Documentation table-alignment catch-up

A pre-existing format:tables drift in six markdown files (data-pipeline/arc/README.md, data-pipeline/setup/README.md, docs/data-pipeline/acsa-setup.md, docs/security/README.md, docs/security/workflow-permissions.md, scripts/README.md) is included as whitespace-only edits, balanced 101 insertions / 101 deletions. No content change.

Type of Change

  • ✨ New feature (non-breaking change adding functionality)
  • 📚 Documentation update
  • 🐛 Bug fix (non-breaking change fixing an issue)
  • 💥 Breaking change (fix or feature causing existing functionality to change)
  • 🏗️ Infrastructure change (Terraform/IaC)
  • ♻️ Refactoring (no functional changes)

Component(s) Affected

  • infrastructure/terraform/prerequisites/ - Azure subscription setup
  • infrastructure/terraform/ - Terraform infrastructure
  • infrastructure/setup/ - OSMO control plane / Helm
  • workflows/ - Training and evaluation workflows
  • training/ - Training pipelines and scripts
  • docs/ - Documentation

Testing Performed

  • Terraform plan reviewed (no unexpected changes)
  • Terraform apply tested in dev environment
  • Training scripts tested locally with Isaac Sim
  • OSMO workflow submitted successfully
  • Smoke tests passed (smoke_test_azure.py)

Additional validation specific to this PR (all passed locally):

  • npm run test:go — 10/10 tests pass across e2e and testutil packages
  • ./infrastructure/terraform/e2e/run-contract-tests.sh — end-to-end helper exercise green in <1s
  • npm run lint:yaml, npm run lint:md, npm run spell-check — all clean
  • shellcheck infrastructure/terraform/e2e/run-contract-tests.sh — clean
  • Contract drift probes — extra declared outputs tolerated; missing declared output fails with the expected outputs.tf pointer

Documentation Impact

  • No documentation changes needed
  • Documentation updated in this PR
  • Documentation issue filed

Bug Fix Checklist

Not applicable — this is a feature PR adding new tests.

  • Linked to issue being fixed
  • Regression test included, OR
  • Justification for no regression test:

Checklist

Related Issues

Closes #290
Closes #335 (CodeQL Go language matrix)

Notes

The single commit on this branch has a malformed message (no newlines between body bullets, so the entire body collapses into the subject line). Recommend amending to the conventional-commit format before push:

test(infrastructure): add Go contract tests for root Terraform outputs (#290)

* add testutil package with terraform-docs-driven contract validation
* add InfraOutputs struct with 27 tagged fields mirroring outputs.tf
* add TestTerraformOutputsContract entry point (<1s, zero Azure creds)
* wire pinned terraform-docs install into go-tests.yml
* include whitespace-only table-alignment catch-up in 6 docs READMEs

🤖 - Generated by Copilot

Follow-up

Tracked in the implementation plan's Planning Log:

  • WI-01 — G-005 deployment tests (real terraform apply against Azure)
  • WI-02 — Contract tests for Terraform submodules
  • WI-03 — Per-deployment contract tests (VPN, DNS, automation)
  • WI-04 — Flip go-tests.yml soft-fail to hard-fail once stable on CI
  • WI-05 — Composite action for Install terraform-docs to dedupe with terraform-docs-check.yml
  • WI-06 / ci(security): add Go to CodeQL language matrix #335 — Add go to the CodeQL language matrix once Go LOC exceeds ~500

bindsi and others added 2 commits April 27, 2026 09:44
…contract tests for Terraform outputs without deployment- create utility functions for validating output contracts- include scripts for running tests and validating outputs- update documentation for new testing procedures
…#564)

Bumps the python-dependencies group with 3 updates:
[hypothesis](https://github.com/HypothesisWorks/hypothesis),
[ruff](https://github.com/astral-sh/ruff) and
[matplotlib](https://github.com/matplotlib/matplotlib).

Updates `hypothesis` from 6.152.1 to 6.152.3
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/HypothesisWorks/hypothesis/releases">hypothesis's
releases</a>.</em></p>
<blockquote>
<h2>Hypothesis for Python - version 6.152.3</h2>
<p>The &quot;hypothesis-urandom&quot; backend now reads from
&quot;/dev/urandom&quot; with
buffering disabled, which improves the control of those hooking
&quot;/dev/urandom&quot; to change or read Hypothesis's random
decisions.</p>
<p><em><a
href="https://hypothesis.readthedocs.io/en/latest/changelog.html#v6-152-3">The
canonical version of these notes (with links) is on
readthedocs.</a></em></p>
<h2>Hypothesis for Python - version 6.152.2</h2>
<p>This release further improves printing of generated values, building
on the changes in  version 6.151.11.</p>
<p>Principle changes:</p>
<ul>
<li>
<p>In many cases where we would have printed a complex expression
producing a value, we now print the repr (or a pretty-printed
version of it).</p>
</li>
<li>
<p>Additionally, in some cases where we would print a complex
expression that involved a lambda, we are now able to simplify that
expression into a more readable one.</p>
</li>
</ul>
<p><em><a
href="https://hypothesis.readthedocs.io/en/latest/changelog.html#v6-152-2">The
canonical version of these notes (with links) is on
readthedocs.</a></em></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/HypothesisWorks/hypothesis/commit/609de04108945b1ee711a514c3bfa520f75b71e9"><code>609de04</code></a>
Bump hypothesis-python version to 6.152.3 and update changelog</li>
<li><a
href="https://github.com/HypothesisWorks/hypothesis/commit/902f1baddcbc0059eee61b3518a07489935e28ef"><code>902f1ba</code></a>
Merge pull request <a
href="https://redirect.github.com/HypothesisWorks/hypothesis/issues/4720">#4720</a>
from Liam-DeVoe/urandom-disable-buffering</li>
<li><a
href="https://github.com/HypothesisWorks/hypothesis/commit/da81118e4cf06651c63ade29892fe45562a4afbc"><code>da81118</code></a>
claude: open /dev/urandom with buffering=0 in URandomProvider</li>
<li><a
href="https://github.com/HypothesisWorks/hypothesis/commit/80fada332e4a5cd19e22ac6be9f7dcfea71d9b68"><code>80fada3</code></a>
Merge pull request <a
href="https://redirect.github.com/HypothesisWorks/hypothesis/issues/4714">#4714</a>
from HypothesisWorks/DRMacIver/uv</li>
<li><a
href="https://github.com/HypothesisWorks/hypothesis/commit/634e2be14895c79c88abcf04b7f0ac3aef685023"><code>634e2be</code></a>
Let tox auto-provision tox-uv instead of pinning it in tools.txt</li>
<li><a
href="https://github.com/HypothesisWorks/hypothesis/commit/526556470a396edb8c43033d466ccf7416c58efe"><code>5265564</code></a>
Seed pip into tox-uv envs</li>
<li><a
href="https://github.com/HypothesisWorks/hypothesis/commit/d6caeb86759f52d5d68c4ad9535f5b471b288240"><code>d6caeb8</code></a>
Use uv instead of pyenv for build-time Python installs and tox</li>
<li><a
href="https://github.com/HypothesisWorks/hypothesis/commit/c727ead5923bd738c51116bba50d4c9f06a1ff51"><code>c727ead</code></a>
Bump hypothesis-python version to 6.152.2 and update changelog</li>
<li><a
href="https://github.com/HypothesisWorks/hypothesis/commit/36d74b796957d7ca99927a3456c6179c22b8dd83"><code>36d74b7</code></a>
Merge pull request <a
href="https://redirect.github.com/HypothesisWorks/hypothesis/issues/4711">#4711</a>
from HypothesisWorks/DRMacIver/is-code-owner</li>
<li><a
href="https://github.com/HypothesisWorks/hypothesis/commit/bbc8963637ec87b8f450f09b1db2e375f0be7247"><code>bbc8963</code></a>
Merge pull request <a
href="https://redirect.github.com/HypothesisWorks/hypothesis/issues/4712">#4712</a>
from HypothesisWorks/DRMacIver/fix-build</li>
<li>Additional commits viewable in <a
href="https://github.com/HypothesisWorks/hypothesis/compare/hypothesis-python-6.152.1...hypothesis-python-6.152.3">compare
view</a></li>
</ul>
</details>
<br />

Updates `ruff` from 0.15.11 to 0.15.12
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/astral-sh/ruff/releases">ruff's
releases</a>.</em></p>
<blockquote>
<h2>0.15.12</h2>
<h2>Release Notes</h2>
<p>Released on 2026-04-24.</p>
<h3>Preview features</h3>
<ul>
<li>Implement <code>#ruff:file-ignore</code> file-level suppressions (<a
href="https://redirect.github.com/astral-sh/ruff/pull/23599">#23599</a>)</li>
<li>Implement <code>#ruff:ignore</code> logical-line suppressions (<a
href="https://redirect.github.com/astral-sh/ruff/pull/23404">#23404</a>)</li>
<li>Revert preview changes to displayed diagnostic severity in LSP (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24789">#24789</a>)</li>
<li>[<code>airflow</code>] Implement
<code>task-branch-as-short-circuit</code> (<code>AIR004</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/23579">#23579</a>)</li>
<li>[<code>flake8-bugbear</code>] Fix
<code>break</code>/<code>continue</code> handling in
<code>loop-iterator-mutation</code> (<code>B909</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24440">#24440</a>)</li>
<li>[<code>pylint</code>] Fix <code>PLC2701</code> for type parameter
scopes (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24576">#24576</a>)</li>
</ul>
<h3>Rule changes</h3>
<ul>
<li>[<code>pandas-vet</code>] Suggest <code>.array</code> as well in
<code>PD011</code> (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24805">#24805</a>)</li>
</ul>
<h3>CLI</h3>
<ul>
<li>Respect default Unix permissions for cache files (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24794">#24794</a>)</li>
</ul>
<h3>Documentation</h3>
<ul>
<li>[<code>pylint</code>] Fix <code>PLR0124</code> description not to
claim self-comparison always returns the same value (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24749">#24749</a>)</li>
<li>[<code>pyupgrade</code>] Expand docs on reusable
<code>TypeVar</code>s and scoping (<code>UP046</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24153">#24153</a>)</li>
<li>Improve rules table accessibility (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24711">#24711</a>)</li>
</ul>
<h3>Contributors</h3>
<ul>
<li><a href="https://github.com/dylwil3"><code>@​dylwil3</code></a></li>
<li><a
href="https://github.com/AlexWaygood"><code>@​AlexWaygood</code></a></li>
<li><a
href="https://github.com/woodruffw"><code>@​woodruffw</code></a></li>
<li><a
href="https://github.com/avasis-ai"><code>@​avasis-ai</code></a></li>
<li><a href="https://github.com/Dev-iL"><code>@​Dev-iL</code></a></li>
<li><a
href="https://github.com/denyszhak"><code>@​denyszhak</code></a></li>
<li><a
href="https://github.com/ShipItAndPray"><code>@​ShipItAndPray</code></a></li>
<li><a
href="https://github.com/anishgirianish"><code>@​anishgirianish</code></a></li>
<li><a
href="https://github.com/augustelalande"><code>@​augustelalande</code></a></li>
<li><a
href="https://github.com/amyreese"><code>@​amyreese</code></a></li>
<li><a
href="https://github.com/majiayu000"><code>@​majiayu000</code></a></li>
</ul>
<h2>Install ruff 0.15.12</h2>
<h3>Install prebuilt binaries via shell script</h3>
<pre lang="sh"><code>curl --proto '=https' --tlsv1.2 -LsSf
https://releases.astral.sh/github/ruff/releases/download/0.15.12/ruff-installer.sh
| sh
</code></pre>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md">ruff's
changelog</a>.</em></p>
<blockquote>
<h2>0.15.12</h2>
<p>Released on 2026-04-24.</p>
<h3>Preview features</h3>
<ul>
<li>Implement <code>#ruff:file-ignore</code> file-level suppressions (<a
href="https://redirect.github.com/astral-sh/ruff/pull/23599">#23599</a>)</li>
<li>Implement <code>#ruff:ignore</code> logical-line suppressions (<a
href="https://redirect.github.com/astral-sh/ruff/pull/23404">#23404</a>)</li>
<li>Revert preview changes to displayed diagnostic severity in LSP (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24789">#24789</a>)</li>
<li>[<code>airflow</code>] Implement
<code>task-branch-as-short-circuit</code> (<code>AIR004</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/23579">#23579</a>)</li>
<li>[<code>flake8-bugbear</code>] Fix
<code>break</code>/<code>continue</code> handling in
<code>loop-iterator-mutation</code> (<code>B909</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24440">#24440</a>)</li>
<li>[<code>pylint</code>] Fix <code>PLC2701</code> for type parameter
scopes (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24576">#24576</a>)</li>
</ul>
<h3>Rule changes</h3>
<ul>
<li>[<code>pandas-vet</code>] Suggest <code>.array</code> as well in
<code>PD011</code> (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24805">#24805</a>)</li>
</ul>
<h3>CLI</h3>
<ul>
<li>Respect default Unix permissions for cache files (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24794">#24794</a>)</li>
</ul>
<h3>Documentation</h3>
<ul>
<li>[<code>pylint</code>] Fix <code>PLR0124</code> description not to
claim self-comparison always returns the same value (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24749">#24749</a>)</li>
<li>[<code>pyupgrade</code>] Expand docs on reusable
<code>TypeVar</code>s and scoping (<code>UP046</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24153">#24153</a>)</li>
<li>Improve rules table accessibility (<a
href="https://redirect.github.com/astral-sh/ruff/pull/24711">#24711</a>)</li>
</ul>
<h3>Contributors</h3>
<ul>
<li><a href="https://github.com/dylwil3"><code>@​dylwil3</code></a></li>
<li><a
href="https://github.com/AlexWaygood"><code>@​AlexWaygood</code></a></li>
<li><a
href="https://github.com/woodruffw"><code>@​woodruffw</code></a></li>
<li><a
href="https://github.com/avasis-ai"><code>@​avasis-ai</code></a></li>
<li><a href="https://github.com/Dev-iL"><code>@​Dev-iL</code></a></li>
<li><a
href="https://github.com/denyszhak"><code>@​denyszhak</code></a></li>
<li><a
href="https://github.com/ShipItAndPray"><code>@​ShipItAndPray</code></a></li>
<li><a
href="https://github.com/anishgirianish"><code>@​anishgirianish</code></a></li>
<li><a
href="https://github.com/augustelalande"><code>@​augustelalande</code></a></li>
<li><a
href="https://github.com/amyreese"><code>@​amyreese</code></a></li>
<li><a
href="https://github.com/majiayu000"><code>@​majiayu000</code></a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/astral-sh/ruff/commit/66f93cf7ed4d36325f35a452e4afa28268fbcd28"><code>66f93cf</code></a>
Bump 0.15.12 (<a
href="https://redirect.github.com/astral-sh/ruff/issues/24815">#24815</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/476a4d02e8e3b6c157ac39979d8b698a1b6baa91"><code>476a4d0</code></a>
[ty] Complete support for more detailed diagnostics on possibly unbound
error...</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/ed669eab30095d6c51fe6cdef6050fb01276bcb3"><code>ed669ea</code></a>
Implement <code>#ruff:file-ignore</code> file-level suppressions (<a
href="https://redirect.github.com/astral-sh/ruff/issues/23599">#23599</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/e73d952e43feb51356ee740c5a973fce81396ff6"><code>e73d952</code></a>
[ty] Include inferred type in <code>invalid-key</code> concise
diagnostic for union/inte...</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/80feb29b31cd98c093316df2e0407b0c70c01b55"><code>80feb29</code></a>
[ty] report only dead annotation-only locals as unused (<a
href="https://redirect.github.com/astral-sh/ruff/issues/24811">#24811</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/0fbf2bc27336a3d17d39af52cf89b78dcda8c7c8"><code>0fbf2bc</code></a>
Drop deprecated license classifier (<a
href="https://redirect.github.com/astral-sh/ruff/issues/24808">#24808</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/43b174cc7f2fcb0080bb1d4843cd4bf6b72bbe27"><code>43b174c</code></a>
[ty] Infer lambda parameter types with <code>Callable</code> type
context (<a
href="https://redirect.github.com/astral-sh/ruff/issues/24317">#24317</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/4f449ae4a2377569330a5ab94799d389357b5a3f"><code>4f449ae</code></a>
[ty] Add error context for intersection types (<a
href="https://redirect.github.com/astral-sh/ruff/issues/24772">#24772</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/5b4e753acb46e96ad408e4904c15308e33efe307"><code>5b4e753</code></a>
[ty] Add support for goto in literal enum member inlay hint (<a
href="https://redirect.github.com/astral-sh/ruff/issues/24792">#24792</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/e7cc76275a758ce1c636ea1c2d091fd576aac794"><code>e7cc762</code></a>
[ty] Add error context for TypedDict assignments (<a
href="https://redirect.github.com/astral-sh/ruff/issues/24790">#24790</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/astral-sh/ruff/compare/0.15.11...0.15.12">compare
view</a></li>
</ul>
</details>
<br />

Updates `matplotlib` from 3.10.8 to 3.10.9
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/matplotlib/matplotlib/releases">matplotlib's
releases</a>.</em></p>
<blockquote>
<h2>v3.10.9</h2>
<p>This is a micro release of the v3.10.x series.
Highlights of this release include:</p>
<ul>
<li>Various minor bug and doc fixes</li>
<li>Security hardening validation of cyclers - Removing eval usage</li>
<li>Security hardening in Latex and PS calls - Removing shell
escapes</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/matplotlib/matplotlib/commit/dd8d78b8dce60b6c8db86132892577a0b9dbe469"><code>dd8d78b</code></a>
REL: v3.10.9</li>
<li><a
href="https://github.com/matplotlib/matplotlib/commit/2fb18915bcfe69a188832c776fe18d88337de9bc"><code>2fb1891</code></a>
REL: Release prep v3.10.9</li>
<li><a
href="https://github.com/matplotlib/matplotlib/commit/d0e923abfa016c04901fe4e315b9d215949f6fc5"><code>d0e923a</code></a>
Merge branch 'v3.10.8-doc' into v3.10.x</li>
<li><a
href="https://github.com/matplotlib/matplotlib/commit/163793248a5fc9f23a560e45551c44351a8bd716"><code>1637932</code></a>
Merge pull request <a
href="https://redirect.github.com/matplotlib/matplotlib/issues/31558">#31558</a>
from meeseeksmachine/auto-backport-of-pr-31556-on-v...</li>
<li><a
href="https://github.com/matplotlib/matplotlib/commit/a83faacb0dbe7edd1bae38e1e715b77b6aaebb84"><code>a83faac</code></a>
Backport PR <a
href="https://redirect.github.com/matplotlib/matplotlib/issues/31556">#31556</a>:
FIX: Inverted PyErr_Occurred check in enum type caster (_...</li>
<li><a
href="https://github.com/matplotlib/matplotlib/commit/a4f57ab0623f9d26be29e0a3b0de904667c7eeb7"><code>a4f57ab</code></a>
Merge pull request <a
href="https://redirect.github.com/matplotlib/matplotlib/issues/31545">#31545</a>
from ksunden/backport-of-pr-31282-on-v3.10.x</li>
<li><a
href="https://github.com/matplotlib/matplotlib/commit/063288d0cc912aa2af5cc1b7e7ca3d228d9f8976"><code>063288d</code></a>
Merge pull request <a
href="https://redirect.github.com/matplotlib/matplotlib/issues/31544">#31544</a>
from ksunden/backport-of-pr-31248-on-v3.10.x</li>
<li><a
href="https://github.com/matplotlib/matplotlib/commit/b2ed1969191a03ec8927f96573664474662ab4c1"><code>b2ed196</code></a>
Backport PR <a
href="https://redirect.github.com/matplotlib/matplotlib/issues/31248">#31248</a>:
SEC: Remove eval() from validate_cycler</li>
<li><a
href="https://github.com/matplotlib/matplotlib/commit/acc60241a70b920eaf04fce41a8cf0a77010fb7d"><code>acc6024</code></a>
Merge pull request <a
href="https://redirect.github.com/matplotlib/matplotlib/issues/31282">#31282</a>
from scottshambaugh/tex_no_shell</li>
<li><a
href="https://github.com/matplotlib/matplotlib/commit/e3fb54163b1ce9dbc1a9e8e0973289dc14e366c2"><code>e3fb541</code></a>
Merge pull request <a
href="https://redirect.github.com/matplotlib/matplotlib/issues/31078">#31078</a>
from meeseeksmachine/auto-backport-of-pr-31075-on-v...</li>
<li>Additional commits viewable in <a
href="https://github.com/matplotlib/matplotlib/compare/v3.10.8...v3.10.9">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marcel Bindseil <marcelbindseil@gmail.com>
@bindsi bindsi requested a review from a team as a code owner April 27, 2026 08:52
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 27, 2026

Dependency Review

The following issues were found:
  • ✅ 0 vulnerable package(s)
  • ✅ 0 package(s) with incompatible licenses
  • ✅ 0 package(s) with invalid SPDX license definitions
  • ⚠️ 4 package(s) with unknown licenses.
See the Details below.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA a4ce5ee.
Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

License Issues

infrastructure/terraform/e2e/go.mod

PackageVersionLicenseIssue Type
github.com/davecgh/go-spew1.1.1NullUnknown License
github.com/pmezard/go-difflib1.0.0NullUnknown License
github.com/stretchr/testify1.11.1NullUnknown License
gopkg.in/yaml.v33.0.1NullUnknown License

OpenSSF Scorecard

PackageVersionScoreDetails
gomod/github.com/davecgh/go-spew 1.1.1 UnknownUnknown
gomod/github.com/pmezard/go-difflib 1.0.0 UnknownUnknown
gomod/github.com/stretchr/testify 1.11.1 UnknownUnknown
gomod/gopkg.in/yaml.v3 3.0.1 UnknownUnknown

Scanned Files

  • infrastructure/terraform/e2e/go.mod

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 27, 2026

Codecov Report

❌ Patch coverage is 93.75000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.67%. Comparing base (6141db4) to head (a4ce5ee).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
infrastructure/terraform/e2e/testutil/contract.go 93.47% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #571      +/-   ##
==========================================
+ Coverage   65.16%   66.67%   +1.50%     
==========================================
  Files         251      264      +13     
  Lines       15597    16659    +1062     
  Branches     2152     2294     +142     
==========================================
+ Hits        10164    11107     +943     
- Misses       5142     5264     +122     
+ Partials      291      288       -3     
Flag Coverage Δ *Carryforward flag
go 93.75% <93.75%> (?)
pester 83.13% <ø> (ø) Carriedforward from 0ca5df9
pytest-data-pipeline 100.00% <ø> (ø) Carriedforward from 0ca5df9
pytest-dataviewer 65.21% <ø> (-1.72%) ⬇️ Carriedforward from 0ca5df9
pytest-dm-tools 100.00% <ø> (ø) Carriedforward from 0ca5df9
pytest-evaluation 99.83% <ø> (?)
pytest-fuzz 4.99% <ø> (+0.09%) ⬆️ Carriedforward from 0ca5df9
pytest-inference 0.00% <ø> (ø) Carriedforward from 0ca5df9
pytest-training 82.14% <ø> (ø) Carriedforward from 0ca5df9
vitest 51.06% <ø> (-1.96%) ⬇️ Carriedforward from 0ca5df9

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
infrastructure/terraform/e2e/outputs.go 100.00% <100.00%> (ø)
infrastructure/terraform/e2e/testutil/contract.go 93.47% <93.47%> (ø)

... and 30 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

- Closes #335

- Adds 'go' alongside javascript-typescript and python

- CodeQL autobuild will discover go.mod once #290 lands on main

🤖 - Generated by Copilot
@github-advanced-security
Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

Copy link
Copy Markdown
Collaborator

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking fantastic — solid contract testing pattern, clean CI wiring, and the terraform-docs SHA verification is a nice touch. A few items to tighten up before merge, mostly around test coverage and documentation cross-references.


Additional documentation cross-references (files not in this diff)

⚠️ docs/contributing/contribution-workflow.md — Terraform modules row (line ~170)

The Terraform modules validation row lists terraform fmt, terraform validate, and terraform plan but doesn't mention that output changes also require npm run test:go to validate the contract. Suggest:

| Terraform modules | `terraform fmt`, `terraform validate`, `terraform plan` output attached to PR, `npm run test:go` (output contract) |

⚠️ docs/contributing/prerequisites.md — Go validation section (line ~190)

The Go block lists lint and test commands but doesn't explain that contract tests validate Terraform outputs or that terraform-docs is required. Suggest adding after the existing Go block:

# Contract tests (validates Terraform outputs against Go struct — requires terraform-docs)
# Run after adding/removing/renaming Terraform outputs
./infrastructure/terraform/e2e/run-contract-tests.sh

💡 docs/contributing/contribution-workflow.md — Go modules row (line ~173)

Contributors who run npm run test:go without terraform-docs installed will get a failure with no guidance from this table. Suggest:

| Go modules | `npm run lint:go` (golangci-lint), `npm run test:go` (`go test`, requires `terraform-docs`) |

Comment thread infrastructure/terraform/e2e/testutil/contract_test.go
Comment thread .github/copilot-instructions.md
Comment thread .github/workflows/go-tests.yml
* refactor ValidateOutputContract to expose findMissingOutputs helper
* add failure-path, exact-match, and empty-declared tests (testutil 58.3% -> 94.6%)
* add minimal-module testdata fixture exercising terraform-docs JSON parsing
* add TestGetTerraformDeclaredOutputs and TestValidateTerraformContract
* fix go-tests.yml cache-dependency-path to track go.sum
* cross-reference npm run test:go from *.tf row in copilot-instructions
* update contribution-workflow.md Terraform/Go rows with contract test command
* document run-contract-tests.sh in prerequisites.md Go validation block

🤖 - Generated by Copilot
@bindsi bindsi requested a review from katriendg April 28, 2026 08:30
bindsi added 2 commits April 28, 2026 09:30
The minimal-module testdata fixture is picked up by tflint --recursive
and triggered terraform_required_version warning. Add a terraform block
matching the repo convention.

🤖 - Generated by Copilot
Copy link
Copy Markdown
Contributor

@rezatnoMsirhC rezatnoMsirhC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for a well-structured PR. The contract pattern is clean, the testutil package is reusable, and the CI wiring is correct. Left two comments.

Comment on lines +18 to +20
func TestTerraformOutputsContract(t *testing.T) {
testutil.ValidateTerraformContract(t, "..", InfraOutputs{}.RequiredOutputKeys())
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The testutil fixture tests (TestGetTerraformDeclaredOutputs, TestValidateTerraformContract) both guard with exec.LookPath("terraform-docs") and call t.Skip() when the tool is absent. This test has no such guard, so running go test ./... on a machine without terraform-docs installed produces a hard failure with an opaque require.NoError message rather than a clean skip. Suggest adding the same guard for consistency:

func TestTerraformOutputsContract(t *testing.T) {
    if _, err := exec.LookPath("terraform-docs"); err != nil {
        t.Skip("terraform-docs not installed; skipping contract test")
    }
    testutil.ValidateTerraformContract(t, "..", InfraOutputs{}.RequiredOutputKeys())
}

Comment on lines +46 to +55

t.Logf("Declared outputs: %v", declared)
t.Logf("Required outputs: %v", required)

missing := findMissingOutputs(declared, required)

if len(missing) > 0 {
t.Errorf("Missing %d required outputs: %v", len(missing), missing)
t.Errorf("Declare these in infrastructure/terraform/outputs.tf or remove from InfraOutputs struct")
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two t.Errorf calls (non-fatal) followed by require.Empty (fatal) report the same condition. The final "output contract validation failed" message is less informative than the preceding errors and adds noise to test output. Suggest consolidating into a single require.Empty with the full actionable message:

func ValidateOutputContract(t *testing.T, declared, required []string) {
    t.Helper()

    t.Logf("Declared outputs: %v", declared)
    t.Logf("Required outputs: %v", required)

    missing := findMissingOutputs(declared, required)
    require.Empty(t, missing,
        "missing %d required outputs: %v -- declare these in infrastructure/terraform/outputs.tf or remove from InfraOutputs struct",
        len(missing), missing)
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(security): add Go to CodeQL language matrix test(infrastructure): Go/Terratest contract and integration tests — tracking issue

6 participants