From d7f5e37c9155d8f497e7c45aa48e61b4b761a86c Mon Sep 17 00:00:00 2001 From: Safia Abdalla Date: Wed, 25 Feb 2026 11:51:57 -0800 Subject: [PATCH] harden review-pr prompt to prevent agent from calling gh pr review Add explicit prohibitions in the System Context, Final Step, and Quality Checklist sections of the prompt to ensure the agent never uses gh pr review, gh pr comment, or any gh command that posts to GitHub. All review output must go to review.json only. Co-Authored-By: Oz --- .github/workflows/review-pr.yml | 5 ++++- examples/review-pr.yml | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/.github/workflows/review-pr.yml b/.github/workflows/review-pr.yml index e84f5f3..a9d8212 100644 --- a/.github/workflows/review-pr.yml +++ b/.github/workflows/review-pr.yml @@ -95,6 +95,7 @@ jobs: - Working directory is PR branch. - You have access to the full source code, but you should primarily analyze and comment on the diff below - **Restrict feedback to code modified in this PR and its immediate context**. Do not comment on untouched files or lines (GitHub will reject these comments). If you must mention issues in untouched code, add them to the summary instead + - **CRITICAL: DO NOT run \`gh pr review\`, \`gh pr comment\`, or any \`gh\` command that posts to GitHub. You must NOT submit any GitHub review or comment directly. All output must be written to \`review.json\` only—the Post Review step will handle publishing.** ## PR Context @@ -187,8 +188,9 @@ jobs: ### 7. Final Step - Create \`review.json\` with your drafted content - - Validate JSON with \`jq\`. If this fails, update the \`review.json\` file with the correct JSON. Do not attempt to use \`rm\` + - Validate JSON with \`jq\`. If this fails, update the \`review.json\` file with the correct JSON. Do not attempt to use \`rm\` - Ensure line numbers are correct and match the diff + - **DO NOT run \`gh pr review\`, \`gh pr comment\`, \`gh api\`, or any other command that posts content to GitHub. Your only output is \`review.json\`. Stop after validating the file.** --- @@ -246,6 +248,7 @@ jobs: - [ ] Comment spans are ≤10 lines (\`line - start_line <= 10\`) - [ ] Summary includes issue counts and recommendation - [ ] JSON is valid, use \`jq\` to validate + - [ ] No \`gh pr review\`, \`gh pr comment\`, or other GitHub-posting commands were run **Goal**: Provide actionable, high-signal feedback to help the author improve the code. Be direct and specific. `; diff --git a/examples/review-pr.yml b/examples/review-pr.yml index 33ac7ee..4573076 100644 --- a/examples/review-pr.yml +++ b/examples/review-pr.yml @@ -87,6 +87,7 @@ jobs: - Working directory is PR branch. - You have access to the full source code, but you should primarily analyze and comment on the diff below - **Restrict feedback to code modified in this PR and its immediate context**. Do not comment on untouched files or lines (GitHub will reject these comments). If you must mention issues in untouched code, add them to the summary instead + - **CRITICAL: DO NOT run \`gh pr review\`, \`gh pr comment\`, or any \`gh\` command that posts to GitHub. You must NOT submit any GitHub review or comment directly. All output must be written to \`review.json\` only—the Post Review step will handle publishing.** ## PR Context @@ -179,8 +180,9 @@ jobs: ### 7. Final Step - Create \`review.json\` with your drafted content - - Validate JSON with \`jq\`. If this fails, update the \`review.json\` file with the correct JSON. Do not attempt to use \`rm\` + - Validate JSON with \`jq\`. If this fails, update the \`review.json\` file with the correct JSON. Do not attempt to use \`rm\` - Ensure line numbers are correct and match the diff + - **DO NOT run \`gh pr review\`, \`gh pr comment\`, \`gh api\`, or any other command that posts content to GitHub. Your only output is \`review.json\`. Stop after validating the file.** --- @@ -238,6 +240,7 @@ jobs: - [ ] Comment spans are ≤10 lines (\`line - start_line <= 10\`) - [ ] Summary includes issue counts and recommendation - [ ] JSON is valid, use \`jq\` to validate + - [ ] No \`gh pr review\`, \`gh pr comment\`, or other GitHub-posting commands were run **Goal**: Provide actionable, high-signal feedback to help the author improve the code. Be direct and specific. `;