refactor(agents): migrate dependabot AW review to workflow_run trigger#612
Open
refactor(agents): migrate dependabot AW review to workflow_run trigger#612
Conversation
- Switch trigger from pull_request_target to workflow_run gated on PR Validation completion on main - Filter on workflow_run.actor.login == 'dependabot[bot]' (replacing pull_request_target bots:/roles: allowlists) - Hydrate PR_VALIDATION_CONCLUSION from workflow_run payload and PR_VALIDATION_FAILING_CHECKS via checks.listForRef - Tighten persona verdict rubric so non-success conclusions map to COMMENT with caution banner - Replace persona check-run API walk with resolver-supplied env vars - Regenerate aw-dependabot-pr-review.lock.yml 🤖 - Generated by Copilot Co-authored-by: Copilot <copilot@github.com>
…dabot branches - change workflow_run branches from main to dependabot/** - clarify workflow execution context for Dependabot PRs 🔧 - Generated by Copilot
Contributor
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. OpenSSF Scorecard
Scanned Files
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #612 +/- ##
==========================================
- Coverage 77.27% 77.27% -0.01%
==========================================
Files 272 272
Lines 18140 18137 -3
Branches 2452 2467 +15
==========================================
- Hits 14018 14015 -3
Misses 3698 3698
Partials 424 424
*This pull request uses carry forward flags. Click here to find out more. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The
aw-dependabot-pr-reviewagentic workflow used to fire onpull_request_target, which meant the resolver step captured a snapshot ofPR Validationwhile it was stillpendingorin_progress:*, and the advisory review was posted before the orchestrator ever finished. PR #608 was the canonical example: the review correctly applied the Isaac Sim numpy 2.x ABI guard, but its CI banner quoted a stalein_progress:in_progressconclusion.This PR migrates the workflow to
workflow_runkeyed onPR Validationcompleted, reads the orchestrator's terminal conclusion straight fromcontext.payload.workflow_run.conclusion, and pre-resolves failing per-surface check-runs once in the resolver step. The persona rubric is rewritten to consume those env vars and to map every terminal conclusion explicitly -pendingandin_progress:*branches are gone because they are now unreachable.Related to #579.
Type of Change
Component(s) Affected
infrastructure/terraform/prerequisites/- Azure subscription setupinfrastructure/terraform/- Terraform infrastructureinfrastructure/setup/- OSMO control plane / Helmworkflows/- Training and evaluation workflowstraining/- Training pipelines and scriptsdocs/- DocumentationChanges
Workflow trigger and resolver
pull_request_targetwithworkflow_runonworkflows: ["PR Validation"],types: [completed],branches: ["dependabot/**"]. Thebranches:filter onworkflow_runmatches the triggering run'shead_branch(not the base), sodependabot/**is the only value that fires for Dependabot PRs — usingmainhere was the #583 regression fixed in #584. The workflow-levelif:filters onworkflow_run.event == 'pull_request',workflow_run.actor.login == 'dependabot[bot]', and a whitelist of seven terminal conclusions.on.bots: ["dependabot[bot]"]andon.roles: [admin, maintainer, write]at the top level — gh-aw'spre_activationguard checks the triggering actor againston.bots/on.rolesindependently of the workflowif:, so dropping these would resurrect the #585 / #586User permission 'none'activation block.checks: readtopermissions:for server-side check-run enumeration; existingcontents,pull-requests, andactionsscopes are unchanged.context.payload.workflow_run, prefersworkflow_run.pull_requests[0], and falls back tosearch.issuesAndPullRequestskeyed onhead_shafor the fork case. Both paths re-hydrate viapulls.getsobodyanddraftare reliable.listWorkflowRunsForRepolookup.PR_VALIDATION_CONCLUSIONnow reads directly fromrun.conclusion, which undertypes: [completed]is always one ofsuccess,failure,cancelled,timed_out,neutral,skipped, oraction_required.PR_VALIDATION_FAILING_CHECKS— JSON array of{name, html_url, conclusion}fromchecks.listForRef(ref=pr.head.sha)filtered to completed non-success/non-neutral/non-skipped runs.PR_BODY— PR body hydrated server-side so the agent does not depend on the integrity-filtered MCP read of the PR.PR_DEPENDABOT_SKIP_REASON:not-a-pr-runandpr-resolution-failed, alongside the existingnot-dependabot/draft.safe-outputs:submit-pull-request-review.target→${{ env.PR_NUMBER }}add-comment.target→${{ env.PR_NUMBER }}(wastriggering, which is undefined underworkflow_run)create-pull-request-review-comment.target→"*"Persona verdict rubric
.github/agents/dependabot-pr-reviewer.agent.md. The persona is told the workflow runs afterPR Validationreaches a terminal conclusion, and is explicitly forbidden from callingchecks.listForReforcommits/{sha}/check-runs— it readsPR_VALIDATION_FAILING_CHECKSfrom the environment instead.success+ no static concern + no sticky high-risk trigger →APPROVE-eligible, citing the orchestrator conclusion plus an emptyPR_VALIDATION_FAILING_CHECKS.failure | cancelled | timed_out | action_required→COMMENT; body MUST quote every entry fromPR_VALIDATION_FAILING_CHECKS(nameplushtml_url).neutral | skipped | unknownorPR_DEPENDABOT_SKIP_REASON == 'pr-resolution-failed'→COMMENTwith a> [!CAUTION]banner: Deterministic CI signal unavailable ({conclusion}); review is advisory only.numpy2.x bump still keeps the verdict atCOMMENTand forces the⚠️ Maintainer review recommendedbanner regardless of CI conclusion.Workflow documentation and lock files
aw-dependabot-pr-review.mdto describe theworkflow_runexecution model, the gh-aw compiler's auto-injected fork-PR exclusion andrepository.idguard, and the new env-var contract.github/gh-aw-actions/setupv0.68.3→v0.71.1in.github/aw/actions-lock.json(SHAba90f21…→239aec4…), picked up by recompilation..github/workflows/aw-dependabot-pr-review.lock.ymlvia the gh-aw compiler — diff reflects the trigger swap, the new env vars, and the setup-action SHA bump. No hand edits.Testing Performed
planreviewed (no unexpected changes)applytested in dev environmentsmoke_test_azure.py)Documentation Impact
Bug Fix Checklist
Not a bug fix — this is a refactor of an agentic-workflow trigger surface.
Checklist
Related Issues
Related to #579
Notes
min-integritytounapprovedwas rejected on prompt-injection grounds; the resolver-side hydration is the chosen mitigation.workflow_runruns in default-branch context, which means changes to the AW workflow itself cannot be exercised by a Dependabot PR — this is the secure-by-design tradeoff documented in the GitHub Security Lab "preventing pwn requests" guide and aligns with the gh-awworkflow_runrecommendation.Follow-up Tasks
PR Validationruns against the same head SHA — confirm that only the latest completed run drives the advisory review.safe-outputs.submit-pull-request-reviewandadd-commentpost successfully underworkflow_run— thetarget: ${{ env.PR_NUMBER }}overrides are the #588 / #589 mitigation; a Not in pull request context skip insafe_outputswould mean the env var did not resolve.