test(infrastructure): add Go output contract tests for root Terraform module#571
test(infrastructure): add Go output contract tests for root Terraform module#571
Conversation
…contract tests for Terraform outputs without deployment- create utility functions for validating output contracts- include scripts for running tests and validating outputs- update documentation for new testing procedures
…#564) Bumps the python-dependencies group with 3 updates: [hypothesis](https://github.com/HypothesisWorks/hypothesis), [ruff](https://github.com/astral-sh/ruff) and [matplotlib](https://github.com/matplotlib/matplotlib). Updates `hypothesis` from 6.152.1 to 6.152.3 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/HypothesisWorks/hypothesis/releases">hypothesis's releases</a>.</em></p> <blockquote> <h2>Hypothesis for Python - version 6.152.3</h2> <p>The "hypothesis-urandom" backend now reads from "/dev/urandom" with buffering disabled, which improves the control of those hooking "/dev/urandom" to change or read Hypothesis's random decisions.</p> <p><em><a href="https://hypothesis.readthedocs.io/en/latest/changelog.html#v6-152-3">The canonical version of these notes (with links) is on readthedocs.</a></em></p> <h2>Hypothesis for Python - version 6.152.2</h2> <p>This release further improves printing of generated values, building on the changes in version 6.151.11.</p> <p>Principle changes:</p> <ul> <li> <p>In many cases where we would have printed a complex expression producing a value, we now print the repr (or a pretty-printed version of it).</p> </li> <li> <p>Additionally, in some cases where we would print a complex expression that involved a lambda, we are now able to simplify that expression into a more readable one.</p> </li> </ul> <p><em><a href="https://hypothesis.readthedocs.io/en/latest/changelog.html#v6-152-2">The canonical version of these notes (with links) is on readthedocs.</a></em></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/HypothesisWorks/hypothesis/commit/609de04108945b1ee711a514c3bfa520f75b71e9"><code>609de04</code></a> Bump hypothesis-python version to 6.152.3 and update changelog</li> <li><a href="https://github.com/HypothesisWorks/hypothesis/commit/902f1baddcbc0059eee61b3518a07489935e28ef"><code>902f1ba</code></a> Merge pull request <a href="https://redirect.github.com/HypothesisWorks/hypothesis/issues/4720">#4720</a> from Liam-DeVoe/urandom-disable-buffering</li> <li><a href="https://github.com/HypothesisWorks/hypothesis/commit/da81118e4cf06651c63ade29892fe45562a4afbc"><code>da81118</code></a> claude: open /dev/urandom with buffering=0 in URandomProvider</li> <li><a href="https://github.com/HypothesisWorks/hypothesis/commit/80fada332e4a5cd19e22ac6be9f7dcfea71d9b68"><code>80fada3</code></a> Merge pull request <a href="https://redirect.github.com/HypothesisWorks/hypothesis/issues/4714">#4714</a> from HypothesisWorks/DRMacIver/uv</li> <li><a href="https://github.com/HypothesisWorks/hypothesis/commit/634e2be14895c79c88abcf04b7f0ac3aef685023"><code>634e2be</code></a> Let tox auto-provision tox-uv instead of pinning it in tools.txt</li> <li><a href="https://github.com/HypothesisWorks/hypothesis/commit/526556470a396edb8c43033d466ccf7416c58efe"><code>5265564</code></a> Seed pip into tox-uv envs</li> <li><a href="https://github.com/HypothesisWorks/hypothesis/commit/d6caeb86759f52d5d68c4ad9535f5b471b288240"><code>d6caeb8</code></a> Use uv instead of pyenv for build-time Python installs and tox</li> <li><a href="https://github.com/HypothesisWorks/hypothesis/commit/c727ead5923bd738c51116bba50d4c9f06a1ff51"><code>c727ead</code></a> Bump hypothesis-python version to 6.152.2 and update changelog</li> <li><a href="https://github.com/HypothesisWorks/hypothesis/commit/36d74b796957d7ca99927a3456c6179c22b8dd83"><code>36d74b7</code></a> Merge pull request <a href="https://redirect.github.com/HypothesisWorks/hypothesis/issues/4711">#4711</a> from HypothesisWorks/DRMacIver/is-code-owner</li> <li><a href="https://github.com/HypothesisWorks/hypothesis/commit/bbc8963637ec87b8f450f09b1db2e375f0be7247"><code>bbc8963</code></a> Merge pull request <a href="https://redirect.github.com/HypothesisWorks/hypothesis/issues/4712">#4712</a> from HypothesisWorks/DRMacIver/fix-build</li> <li>Additional commits viewable in <a href="https://github.com/HypothesisWorks/hypothesis/compare/hypothesis-python-6.152.1...hypothesis-python-6.152.3">compare view</a></li> </ul> </details> <br /> Updates `ruff` from 0.15.11 to 0.15.12 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/astral-sh/ruff/releases">ruff's releases</a>.</em></p> <blockquote> <h2>0.15.12</h2> <h2>Release Notes</h2> <p>Released on 2026-04-24.</p> <h3>Preview features</h3> <ul> <li>Implement <code>#ruff:file-ignore</code> file-level suppressions (<a href="https://redirect.github.com/astral-sh/ruff/pull/23599">#23599</a>)</li> <li>Implement <code>#ruff:ignore</code> logical-line suppressions (<a href="https://redirect.github.com/astral-sh/ruff/pull/23404">#23404</a>)</li> <li>Revert preview changes to displayed diagnostic severity in LSP (<a href="https://redirect.github.com/astral-sh/ruff/pull/24789">#24789</a>)</li> <li>[<code>airflow</code>] Implement <code>task-branch-as-short-circuit</code> (<code>AIR004</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/23579">#23579</a>)</li> <li>[<code>flake8-bugbear</code>] Fix <code>break</code>/<code>continue</code> handling in <code>loop-iterator-mutation</code> (<code>B909</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/24440">#24440</a>)</li> <li>[<code>pylint</code>] Fix <code>PLC2701</code> for type parameter scopes (<a href="https://redirect.github.com/astral-sh/ruff/pull/24576">#24576</a>)</li> </ul> <h3>Rule changes</h3> <ul> <li>[<code>pandas-vet</code>] Suggest <code>.array</code> as well in <code>PD011</code> (<a href="https://redirect.github.com/astral-sh/ruff/pull/24805">#24805</a>)</li> </ul> <h3>CLI</h3> <ul> <li>Respect default Unix permissions for cache files (<a href="https://redirect.github.com/astral-sh/ruff/pull/24794">#24794</a>)</li> </ul> <h3>Documentation</h3> <ul> <li>[<code>pylint</code>] Fix <code>PLR0124</code> description not to claim self-comparison always returns the same value (<a href="https://redirect.github.com/astral-sh/ruff/pull/24749">#24749</a>)</li> <li>[<code>pyupgrade</code>] Expand docs on reusable <code>TypeVar</code>s and scoping (<code>UP046</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/24153">#24153</a>)</li> <li>Improve rules table accessibility (<a href="https://redirect.github.com/astral-sh/ruff/pull/24711">#24711</a>)</li> </ul> <h3>Contributors</h3> <ul> <li><a href="https://github.com/dylwil3"><code>@dylwil3</code></a></li> <li><a href="https://github.com/AlexWaygood"><code>@AlexWaygood</code></a></li> <li><a href="https://github.com/woodruffw"><code>@woodruffw</code></a></li> <li><a href="https://github.com/avasis-ai"><code>@avasis-ai</code></a></li> <li><a href="https://github.com/Dev-iL"><code>@Dev-iL</code></a></li> <li><a href="https://github.com/denyszhak"><code>@denyszhak</code></a></li> <li><a href="https://github.com/ShipItAndPray"><code>@ShipItAndPray</code></a></li> <li><a href="https://github.com/anishgirianish"><code>@anishgirianish</code></a></li> <li><a href="https://github.com/augustelalande"><code>@augustelalande</code></a></li> <li><a href="https://github.com/amyreese"><code>@amyreese</code></a></li> <li><a href="https://github.com/majiayu000"><code>@majiayu000</code></a></li> </ul> <h2>Install ruff 0.15.12</h2> <h3>Install prebuilt binaries via shell script</h3> <pre lang="sh"><code>curl --proto '=https' --tlsv1.2 -LsSf https://releases.astral.sh/github/ruff/releases/download/0.15.12/ruff-installer.sh | sh </code></pre> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md">ruff's changelog</a>.</em></p> <blockquote> <h2>0.15.12</h2> <p>Released on 2026-04-24.</p> <h3>Preview features</h3> <ul> <li>Implement <code>#ruff:file-ignore</code> file-level suppressions (<a href="https://redirect.github.com/astral-sh/ruff/pull/23599">#23599</a>)</li> <li>Implement <code>#ruff:ignore</code> logical-line suppressions (<a href="https://redirect.github.com/astral-sh/ruff/pull/23404">#23404</a>)</li> <li>Revert preview changes to displayed diagnostic severity in LSP (<a href="https://redirect.github.com/astral-sh/ruff/pull/24789">#24789</a>)</li> <li>[<code>airflow</code>] Implement <code>task-branch-as-short-circuit</code> (<code>AIR004</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/23579">#23579</a>)</li> <li>[<code>flake8-bugbear</code>] Fix <code>break</code>/<code>continue</code> handling in <code>loop-iterator-mutation</code> (<code>B909</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/24440">#24440</a>)</li> <li>[<code>pylint</code>] Fix <code>PLC2701</code> for type parameter scopes (<a href="https://redirect.github.com/astral-sh/ruff/pull/24576">#24576</a>)</li> </ul> <h3>Rule changes</h3> <ul> <li>[<code>pandas-vet</code>] Suggest <code>.array</code> as well in <code>PD011</code> (<a href="https://redirect.github.com/astral-sh/ruff/pull/24805">#24805</a>)</li> </ul> <h3>CLI</h3> <ul> <li>Respect default Unix permissions for cache files (<a href="https://redirect.github.com/astral-sh/ruff/pull/24794">#24794</a>)</li> </ul> <h3>Documentation</h3> <ul> <li>[<code>pylint</code>] Fix <code>PLR0124</code> description not to claim self-comparison always returns the same value (<a href="https://redirect.github.com/astral-sh/ruff/pull/24749">#24749</a>)</li> <li>[<code>pyupgrade</code>] Expand docs on reusable <code>TypeVar</code>s and scoping (<code>UP046</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/24153">#24153</a>)</li> <li>Improve rules table accessibility (<a href="https://redirect.github.com/astral-sh/ruff/pull/24711">#24711</a>)</li> </ul> <h3>Contributors</h3> <ul> <li><a href="https://github.com/dylwil3"><code>@dylwil3</code></a></li> <li><a href="https://github.com/AlexWaygood"><code>@AlexWaygood</code></a></li> <li><a href="https://github.com/woodruffw"><code>@woodruffw</code></a></li> <li><a href="https://github.com/avasis-ai"><code>@avasis-ai</code></a></li> <li><a href="https://github.com/Dev-iL"><code>@Dev-iL</code></a></li> <li><a href="https://github.com/denyszhak"><code>@denyszhak</code></a></li> <li><a href="https://github.com/ShipItAndPray"><code>@ShipItAndPray</code></a></li> <li><a href="https://github.com/anishgirianish"><code>@anishgirianish</code></a></li> <li><a href="https://github.com/augustelalande"><code>@augustelalande</code></a></li> <li><a href="https://github.com/amyreese"><code>@amyreese</code></a></li> <li><a href="https://github.com/majiayu000"><code>@majiayu000</code></a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/astral-sh/ruff/commit/66f93cf7ed4d36325f35a452e4afa28268fbcd28"><code>66f93cf</code></a> Bump 0.15.12 (<a href="https://redirect.github.com/astral-sh/ruff/issues/24815">#24815</a>)</li> <li><a href="https://github.com/astral-sh/ruff/commit/476a4d02e8e3b6c157ac39979d8b698a1b6baa91"><code>476a4d0</code></a> [ty] Complete support for more detailed diagnostics on possibly unbound error...</li> <li><a href="https://github.com/astral-sh/ruff/commit/ed669eab30095d6c51fe6cdef6050fb01276bcb3"><code>ed669ea</code></a> Implement <code>#ruff:file-ignore</code> file-level suppressions (<a href="https://redirect.github.com/astral-sh/ruff/issues/23599">#23599</a>)</li> <li><a href="https://github.com/astral-sh/ruff/commit/e73d952e43feb51356ee740c5a973fce81396ff6"><code>e73d952</code></a> [ty] Include inferred type in <code>invalid-key</code> concise diagnostic for union/inte...</li> <li><a href="https://github.com/astral-sh/ruff/commit/80feb29b31cd98c093316df2e0407b0c70c01b55"><code>80feb29</code></a> [ty] report only dead annotation-only locals as unused (<a href="https://redirect.github.com/astral-sh/ruff/issues/24811">#24811</a>)</li> <li><a href="https://github.com/astral-sh/ruff/commit/0fbf2bc27336a3d17d39af52cf89b78dcda8c7c8"><code>0fbf2bc</code></a> Drop deprecated license classifier (<a href="https://redirect.github.com/astral-sh/ruff/issues/24808">#24808</a>)</li> <li><a href="https://github.com/astral-sh/ruff/commit/43b174cc7f2fcb0080bb1d4843cd4bf6b72bbe27"><code>43b174c</code></a> [ty] Infer lambda parameter types with <code>Callable</code> type context (<a href="https://redirect.github.com/astral-sh/ruff/issues/24317">#24317</a>)</li> <li><a href="https://github.com/astral-sh/ruff/commit/4f449ae4a2377569330a5ab94799d389357b5a3f"><code>4f449ae</code></a> [ty] Add error context for intersection types (<a href="https://redirect.github.com/astral-sh/ruff/issues/24772">#24772</a>)</li> <li><a href="https://github.com/astral-sh/ruff/commit/5b4e753acb46e96ad408e4904c15308e33efe307"><code>5b4e753</code></a> [ty] Add support for goto in literal enum member inlay hint (<a href="https://redirect.github.com/astral-sh/ruff/issues/24792">#24792</a>)</li> <li><a href="https://github.com/astral-sh/ruff/commit/e7cc76275a758ce1c636ea1c2d091fd576aac794"><code>e7cc762</code></a> [ty] Add error context for TypedDict assignments (<a href="https://redirect.github.com/astral-sh/ruff/issues/24790">#24790</a>)</li> <li>Additional commits viewable in <a href="https://github.com/astral-sh/ruff/compare/0.15.11...0.15.12">compare view</a></li> </ul> </details> <br /> Updates `matplotlib` from 3.10.8 to 3.10.9 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/matplotlib/matplotlib/releases">matplotlib's releases</a>.</em></p> <blockquote> <h2>v3.10.9</h2> <p>This is a micro release of the v3.10.x series. Highlights of this release include:</p> <ul> <li>Various minor bug and doc fixes</li> <li>Security hardening validation of cyclers - Removing eval usage</li> <li>Security hardening in Latex and PS calls - Removing shell escapes</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/matplotlib/matplotlib/commit/dd8d78b8dce60b6c8db86132892577a0b9dbe469"><code>dd8d78b</code></a> REL: v3.10.9</li> <li><a href="https://github.com/matplotlib/matplotlib/commit/2fb18915bcfe69a188832c776fe18d88337de9bc"><code>2fb1891</code></a> REL: Release prep v3.10.9</li> <li><a href="https://github.com/matplotlib/matplotlib/commit/d0e923abfa016c04901fe4e315b9d215949f6fc5"><code>d0e923a</code></a> Merge branch 'v3.10.8-doc' into v3.10.x</li> <li><a href="https://github.com/matplotlib/matplotlib/commit/163793248a5fc9f23a560e45551c44351a8bd716"><code>1637932</code></a> Merge pull request <a href="https://redirect.github.com/matplotlib/matplotlib/issues/31558">#31558</a> from meeseeksmachine/auto-backport-of-pr-31556-on-v...</li> <li><a href="https://github.com/matplotlib/matplotlib/commit/a83faacb0dbe7edd1bae38e1e715b77b6aaebb84"><code>a83faac</code></a> Backport PR <a href="https://redirect.github.com/matplotlib/matplotlib/issues/31556">#31556</a>: FIX: Inverted PyErr_Occurred check in enum type caster (_...</li> <li><a href="https://github.com/matplotlib/matplotlib/commit/a4f57ab0623f9d26be29e0a3b0de904667c7eeb7"><code>a4f57ab</code></a> Merge pull request <a href="https://redirect.github.com/matplotlib/matplotlib/issues/31545">#31545</a> from ksunden/backport-of-pr-31282-on-v3.10.x</li> <li><a href="https://github.com/matplotlib/matplotlib/commit/063288d0cc912aa2af5cc1b7e7ca3d228d9f8976"><code>063288d</code></a> Merge pull request <a href="https://redirect.github.com/matplotlib/matplotlib/issues/31544">#31544</a> from ksunden/backport-of-pr-31248-on-v3.10.x</li> <li><a href="https://github.com/matplotlib/matplotlib/commit/b2ed1969191a03ec8927f96573664474662ab4c1"><code>b2ed196</code></a> Backport PR <a href="https://redirect.github.com/matplotlib/matplotlib/issues/31248">#31248</a>: SEC: Remove eval() from validate_cycler</li> <li><a href="https://github.com/matplotlib/matplotlib/commit/acc60241a70b920eaf04fce41a8cf0a77010fb7d"><code>acc6024</code></a> Merge pull request <a href="https://redirect.github.com/matplotlib/matplotlib/issues/31282">#31282</a> from scottshambaugh/tex_no_shell</li> <li><a href="https://github.com/matplotlib/matplotlib/commit/e3fb54163b1ce9dbc1a9e8e0973289dc14e366c2"><code>e3fb541</code></a> Merge pull request <a href="https://redirect.github.com/matplotlib/matplotlib/issues/31078">#31078</a> from meeseeksmachine/auto-backport-of-pr-31075-on-v...</li> <li>Additional commits viewable in <a href="https://github.com/matplotlib/matplotlib/compare/v3.10.8...v3.10.9">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Marcel Bindseil <marcelbindseil@gmail.com>
Dependency ReviewThe following issues were found:
Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. License Issuesinfrastructure/terraform/e2e/go.mod
OpenSSF Scorecard
Scanned Files
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #571 +/- ##
==========================================
+ Coverage 65.16% 66.67% +1.50%
==========================================
Files 251 264 +13
Lines 15597 16659 +1062
Branches 2152 2294 +142
==========================================
+ Hits 10164 11107 +943
- Misses 5142 5264 +122
+ Partials 291 288 -3
*This pull request uses carry forward flags. Click here to find out more.
🚀 New features to boost your workflow:
|
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
katriendg
left a comment
There was a problem hiding this comment.
This is looking fantastic — solid contract testing pattern, clean CI wiring, and the terraform-docs SHA verification is a nice touch. A few items to tighten up before merge, mostly around test coverage and documentation cross-references.
Additional documentation cross-references (files not in this diff)
docs/contributing/contribution-workflow.md — Terraform modules row (line ~170)
The Terraform modules validation row lists terraform fmt, terraform validate, and terraform plan but doesn't mention that output changes also require npm run test:go to validate the contract. Suggest:
| Terraform modules | `terraform fmt`, `terraform validate`, `terraform plan` output attached to PR, `npm run test:go` (output contract) |docs/contributing/prerequisites.md — Go validation section (line ~190)
The Go block lists lint and test commands but doesn't explain that contract tests validate Terraform outputs or that terraform-docs is required. Suggest adding after the existing Go block:
# Contract tests (validates Terraform outputs against Go struct — requires terraform-docs)
# Run after adding/removing/renaming Terraform outputs
./infrastructure/terraform/e2e/run-contract-tests.sh💡 docs/contributing/contribution-workflow.md — Go modules row (line ~173)
Contributors who run npm run test:go without terraform-docs installed will get a failure with no guidance from this table. Suggest:
| Go modules | `npm run lint:go` (golangci-lint), `npm run test:go` (`go test`, requires `terraform-docs`) |* refactor ValidateOutputContract to expose findMissingOutputs helper * add failure-path, exact-match, and empty-declared tests (testutil 58.3% -> 94.6%) * add minimal-module testdata fixture exercising terraform-docs JSON parsing * add TestGetTerraformDeclaredOutputs and TestValidateTerraformContract * fix go-tests.yml cache-dependency-path to track go.sum * cross-reference npm run test:go from *.tf row in copilot-instructions * update contribution-workflow.md Terraform/Go rows with contract test command * document run-contract-tests.sh in prerequisites.md Go validation block 🤖 - Generated by Copilot
The minimal-module testdata fixture is picked up by tflint --recursive and triggered terraform_required_version warning. Add a terraform block matching the repo convention. 🤖 - Generated by Copilot
rezatnoMsirhC
left a comment
There was a problem hiding this comment.
Thanks for a well-structured PR. The contract pattern is clean, the testutil package is reusable, and the CI wiring is correct. Left two comments.
| func TestTerraformOutputsContract(t *testing.T) { | ||
| testutil.ValidateTerraformContract(t, "..", InfraOutputs{}.RequiredOutputKeys()) | ||
| } |
There was a problem hiding this comment.
The testutil fixture tests (TestGetTerraformDeclaredOutputs, TestValidateTerraformContract) both guard with exec.LookPath("terraform-docs") and call t.Skip() when the tool is absent. This test has no such guard, so running go test ./... on a machine without terraform-docs installed produces a hard failure with an opaque require.NoError message rather than a clean skip. Suggest adding the same guard for consistency:
func TestTerraformOutputsContract(t *testing.T) {
if _, err := exec.LookPath("terraform-docs"); err != nil {
t.Skip("terraform-docs not installed; skipping contract test")
}
testutil.ValidateTerraformContract(t, "..", InfraOutputs{}.RequiredOutputKeys())
}|
|
||
| t.Logf("Declared outputs: %v", declared) | ||
| t.Logf("Required outputs: %v", required) | ||
|
|
||
| missing := findMissingOutputs(declared, required) | ||
|
|
||
| if len(missing) > 0 { | ||
| t.Errorf("Missing %d required outputs: %v", len(missing), missing) | ||
| t.Errorf("Declare these in infrastructure/terraform/outputs.tf or remove from InfraOutputs struct") | ||
| } |
There was a problem hiding this comment.
Two t.Errorf calls (non-fatal) followed by require.Empty (fatal) report the same condition. The final "output contract validation failed" message is less informative than the preceding errors and adds noise to test output. Suggest consolidating into a single require.Empty with the full actionable message:
func ValidateOutputContract(t *testing.T, declared, required []string) {
t.Helper()
t.Logf("Declared outputs: %v", declared)
t.Logf("Required outputs: %v", required)
missing := findMissingOutputs(declared, required)
require.Empty(t, missing,
"missing %d required outputs: %v -- declare these in infrastructure/terraform/outputs.tf or remove from InfraOutputs struct",
len(missing), missing)
}
This PR completes the final sub-issue of the Go/Terratest umbrella (#290) by adding a fast, credential-free contract test that catches drift between the root Terraform module's declared
outputblocks and the typedInfraOutputsstruct end-to-end tests rely on. The test runs in under one second, requires noterraform initand no Azure credentials, and uses terraform-docs to extract declared outputs at test time.It also closes #335 by adding
goto the CodeQL language matrix so the new Go code is scanned alongside JavaScript/TypeScript and Python from day one.Description
The new contract pattern lives under infrastructure/terraform/e2e/ and has three layers. A reusable testutil package shells out to
terraform-docs jsonto discover declared outputs, then validates them against a required-key set produced by reflection overoutput:"..."struct tags. A typed InfraOutputs struct in outputs.go defines that contract for the root module's 27 outputs. A thinTestTerraformOutputsContractentry point wires the two together and is the first real test in the previously placeholdere2epackage.CI wiring lives in .github/workflows/go-tests.yml: a pinned
Install terraform-docsstep (v0.21.0, SHA256 verified) runs betweenSetup GoandRun Go tests. A small developer helper, run-contract-tests.sh, provides a one-command local entry point with preflight checks forgoandterraform-docs. The*.gorow in .github/copilot-instructions.md now cites the helper alongside the existing lint and test commands.Test infrastructure
The contract is intentionally required-subset-of-declared. Adding a new output to outputs.tf without updating InfraOutputs leaves the test green; removing a declared output that the struct still references fails with an actionable message naming outputs.tf. Both directions were probed during validation.
GetTerraformDeclaredOutputs,ValidateOutputContract,ValidateTerraformContract, andGetOutputKeysFromStruct— reused unchanged by future per-deployment contracts.InfraOutputsstruct mirroring the root module's 27 outputs, each tagged withoutput:"<name>";RequiredOutputKeys()extracts the tag set via reflection.TestTerraformOutputsContractin contract_test.go — replaces the placeholder e2e.go file as the package's anchor.github.com/stretchr/testify v1.11.1.CI and tooling
/usr/local/bin/viasudo mvonubuntu-latest.set -euo pipefail, a-v/--verboseflag, and clear preflight messages whengoorterraform-docsis missing.*.goQuick Reference row updated to list the new helper and call out theterraform-docsdependency.go(closes ci(security): add Go to CodeQL language matrix #335); CodeQL autobuild discovers infrastructure/terraform/e2e/go.mod automatically.Documentation table-alignment catch-up
A pre-existing
format:tablesdrift in six markdown files (data-pipeline/arc/README.md, data-pipeline/setup/README.md, docs/data-pipeline/acsa-setup.md, docs/security/README.md, docs/security/workflow-permissions.md, scripts/README.md) is included as whitespace-only edits, balanced 101 insertions / 101 deletions. No content change.Type of Change
Component(s) Affected
infrastructure/terraform/prerequisites/- Azure subscription setupinfrastructure/terraform/- Terraform infrastructureinfrastructure/setup/- OSMO control plane / Helmworkflows/- Training and evaluation workflowstraining/- Training pipelines and scriptsdocs/- DocumentationTesting Performed
planreviewed (no unexpected changes)applytested in dev environmentsmoke_test_azure.py)Additional validation specific to this PR (all passed locally):
npm run test:go— 10/10 tests pass acrosse2eandtestutilpackages./infrastructure/terraform/e2e/run-contract-tests.sh— end-to-end helper exercise green in <1snpm run lint:yaml,npm run lint:md,npm run spell-check— all cleanshellcheck infrastructure/terraform/e2e/run-contract-tests.sh— cleanDocumentation Impact
Bug Fix Checklist
Not applicable — this is a feature PR adding new tests.
Checklist
Related Issues
Closes #290
Closes #335 (CodeQL Go language matrix)
Notes
The single commit on this branch has a malformed message (no newlines between body bullets, so the entire body collapses into the subject line). Recommend amending to the conventional-commit format before push:
Follow-up
Tracked in the implementation plan's Planning Log:
terraform applyagainst Azure)soft-failto hard-fail once stable on CIInstall terraform-docsto dedupe with terraform-docs-check.ymlgoto the CodeQL language matrix once Go LOC exceeds ~500