Skip to content

feat(workflows): add weekly GitHub code scanning automation#1495

Open
rezatnoMsirhC wants to merge 15 commits intomainfrom
feat/1329-gh-security-weekly-scanning-workflow
Open

feat(workflows): add weekly GitHub code scanning automation#1495
rezatnoMsirhC wants to merge 15 commits intomainfrom
feat/1329-gh-security-weekly-scanning-workflow

Conversation

@rezatnoMsirhC
Copy link
Copy Markdown
Contributor

@rezatnoMsirhC rezatnoMsirhC commented Apr 30, 2026

feat(workflows): add weekly GitHub code scanning automation

Description

Added a weekly GitHub code scanning backlog automation that runs every Monday at 03:00 UTC. The implementation is split into a reusable workflow_call scan workflow (gh-code-scanning.yml), a reusable issue-creation workflow (create-gh-code-scanning-issues.yml), and a thin schedule orchestrator (weekly-gh-code-scanning.yml), keeping the logic composable and the scheduler minimal.

The scan workflow fetches code scanning alerts using the existing gh-code-scanning skill's Get-CodeScanningAlerts.ps1 script and uploads them as a workflow artifact. The issue-creation workflow downloads the artifact, then iterates over unique rules to create labeled security issues — updating existing open issues with the current alert count when a matching title is found.

Workflow Architecture

  • Added gh-code-scanning.yml: reusable workflow_call workflow. Runs on ubuntu-latest, scopes permissions to contents: read and security-events: read at both workflow and job level. Uses SHA-pinned actions/checkout with persist-credentials: false. Uploads a gh-code-scanning-alerts artifact.
  • Added create-gh-code-scanning-issues.yml: reusable workflow_call workflow. Downloads the alerts artifact, iterates over unique rules, and creates or updates security-labeled backlog issues.
  • Added weekly-gh-code-scanning.yml: schedule orchestrator. Triggers on cron: '0 3 * * 1' and workflow_dispatch. Delegates to gh-code-scanning.yml then create-gh-code-scanning-issues.yml via uses:. Sets cancel-in-progress: false to prevent overlapping scan runs.

Backlog Issue Management

  • Issues are deduplicated by title search ("[Security] ${RULE_DESC}" in:title) against open issues before creation.
  • When an existing open issue is found, gh issue edit refreshes the body with the current alert count and gh issue comment posts a dated weekly re-check note.
  • Each issue body includes an HTML comment marker (automation:security-scan:${RULE_ID}) for traceability.

Related Issue(s)

Closes #1329

Related issues created during research and implementation

Issue Title
#1498 Add analyses error detection to the weekly code scanning workflow
#1510 feat(workflows): add gh-aw agentic code scanning issue creation workflow

Issues created from test run

Issue Rule Severity Tool Occurrences
#1499 py/empty-except note CodeQL 23
#1500 actions/code-injection/medium medium CodeQL 2
#1501 BranchProtectionID high Scorecard 1
#1502 py/redundant-comparison warning CodeQL 1

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Note for AI Artifact Contributors:

  • Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
  • Skills: Must include both bash and PowerShell scripts. See Skills.
  • Model Versions: Only contributions targeting the latest Anthropic and OpenAI models will be accepted. Older model versions (e.g., GPT-3.5, Claude 3) will be rejected.
  • See Agents Not Accepted and Model Version Requirements.

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Sample Prompts (for AI Artifact Contributions)

User Request:

Execution Flow:

Output Artifacts:

Success Indicators:

For detailed contribution requirements, see:

Testing

Automated validation results:

  • Markdown linting (npm run lint:md): passed — 0 errors across 196 files
  • Spell checking (npm run spell-check): passed — 0 issues across 300 files
  • Frontmatter validation (npm run lint:frontmatter): passed — 536 files, 0 errors
  • Skill structure validation (npm run validate:skills): passed — 19 skills, 0 errors (1 pre-existing warning unrelated to this PR)
  • Link validation (npm run lint:md-links): passed
  • PowerShell analysis (npm run lint:ps): passed — all files clean

Security analysis:

  • No sensitive data, secrets, or credentials introduced in the diff.
  • actions/checkout is SHA-pinned to de0fac2e4500dabe0009e67214ff5f5447ce83dd.
  • Permissions are scoped to minimum required at both workflow and job level.
  • persist-credentials: false set on checkout step.

Diff-based assessments:

  • Three new files added; no existing files modified or deleted.
  • No breaking changes: no public surfaces removed.
  • No AI artifact files changed.

Note

Add manual testing descriptions when applicable.

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable) (N/A — no test framework for GHA workflows)

AI Artifact Contributions

  • Used /prompt-analyze to review contribution (N/A — no AI artifacts changed)
  • Addressed all feedback from prompt-builder review (N/A — no AI artifacts changed)
  • Verified contribution follows common standards and type-specific requirements (N/A — no AI artifacts changed)

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps

- Create weekly-security-scanning.yml on Monday 03:00 UTC schedule
- Reuse gh-code-scanning skill script; no duplication to scripts/security/
- Deduplicate backlog issues via automation marker per unique rule
- Detect scan analysis failures and file labeled ci-scanning-failure issue

🔒 - Generated by Copilot
- Delete weekly-security-scanning.yml (replaced by two new files)
- Create gh-security-scanning.yml as workflow_call-only reusable
- Create weekly-gh-security-scanning.yml as thin schedule orchestrator

♻️ - Generated by Copilot
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.70%. Comparing base (57ea279) to head (ab37e93).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1495      +/-   ##
==========================================
+ Coverage   87.44%   87.70%   +0.26%     
==========================================
  Files          68       67       -1     
  Lines       10335    10313      -22     
==========================================
+ Hits         9037     9045       +8     
+ Misses       1298     1268      -30     
Flag Coverage Δ
pester 85.18% <100.00%> (+0.37%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...h-code-scanning/scripts/Get-CodeScanningAlerts.ps1 97.50% <100.00%> (+0.72%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…le workflow

- Add create-gh-security-scanning-issues.yml as workflow_call reusable
- Strip issue-creation steps from gh-security-scanning.yml; add artifact upload
- Update weekly orchestrator to call both workflows via needs:

🔒 - Generated by Copilot
- Remove owner and repo inputs from gh-security-scanning.yml
- Remove owner and repo inputs from create-gh-security-scanning-issues.yml
- Drop with: blocks from weekly orchestrator jobs

🔧 - Generated by Copilot
…apply Q1/Q2/Q3 fixes

- Remove deferred Check for scan analysis errors step (Q1)
- Rename all three workflow files to gh-code-scanning naming (Q2)
- Add else branch to issue loop: gh issue edit + gh issue comment (Q3)

🔧 - Generated by Copilot
@rezatnoMsirhC rezatnoMsirhC changed the title feat(workflows): add weekly GitHub security scanning automation feat(workflows): add weekly GitHub code scanning automation May 1, 2026
- Add on: pull_request targeting main with opened/synchronize/reopened/ready_for_review types
- Update concurrency to cancel stale PR runs, preserve scheduled run behavior

🧪 - Generated by Copilot
- Extend PS grouper: sentinel filter, AffectedPaths, HasFilePaths, AlertUrl, FindingDescription
- Rewrite bash step: bulleted linked paths, conditional repo-level section
- Fix labels, title severity, dedup to body marker, grammar, enriched body

🔒 - Generated by Copilot
- rename SamplePaths to AffectedPaths in JSON example and prose
- add HasFilePaths, AlertUrl, FindingDescription to example and field list
- fix BranchProtectionID example to show empty AffectedPaths array
- add GroupedJson alias and Key fields API path disambiguation note

📝 - Generated by Copilot
…e finding

- add --title and --add-label flags to gh issue edit on existing issues
- remove duplicate FindingDescription from repo-level PATHS_SECTION

🔒 - Generated by Copilot
…back field

- add Severity (rule.severity) field to PS script grouped output
- fall back to Severity when SecuritySeverity is null
- omit [SEVERITY] title bracket and severity body line when both are null

🔧 - Generated by Copilot
- rename SamplePaths references to AffectedPaths in three tests
- update no-file test to assert empty array and HasFilePaths false

🧪 - Generated by Copilot
@rezatnoMsirhC rezatnoMsirhC marked this pull request as ready for review May 1, 2026 20:26
@rezatnoMsirhC rezatnoMsirhC requested a review from a team as a code owner May 1, 2026 20:26
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review — feat(workflows): add weekly GitHub code scanning automation

The overall architecture is well-conceived — splitting the scan, issue creation, and scheduling into three composable reusable workflows is the right pattern. The improvements to Get-CodeScanningAlerts.ps1 (richer output fields, HasFilePaths guard, AlertUrl, FindingDescription) are a clear improvement, and the updated tests correctly track the renamed fields. The request-changes verdict is driven by three concrete issues that must be addressed before merging.


🔗 Issue Alignment

Linked issue #1329 could not be read (integrity policy), but the PR description is comprehensive and the implementation matches the declared scope. No scope creep detected. ✅


📋 PR Template Compliance

Check Status Note
Description filled in Detailed and well-structured
Related Issue(s) Closes #1329
Type of Change checked New feature + GitHub Actions workflow
Testing section All automated checks listed with results
"Documentation is updated" ⚠️ SKILL.md was substantively updated; this checkbox should be checked
Security Considerations section ⚠️ The PR template includes a ## Security Considerations block with three checkboxes that is absent from this PR description. Please add it and check the applicable items.
npm run plugin:generate and npm run docs:test ⚠️ These two required automated checks appear in the template but are absent from the PR checklist. Mark them N/A if they don't apply, or run and record results.

🔍 Coding Standards

Bash script conventions: The run: block in create-gh-code-scanning-issues.yml is missing set -euo pipefail (see inline comment, line 39). The bash conventions file explicitly requires this at the top of every bash script block.

Workflow permissions: security-events: read appears in create-gh-code-scanning-issues.yml at both the workflow and job level but is never used (see inline comment, line 14). All other permission declarations follow least-privilege correctly.

SHA-pinned actions: All three new workflows correctly pin SHA-referenced actions. ✅


🛡️ Code Quality and Security

❌ Orphaned file with broken field reference — create-gh-security-scanning-issues.yml

This is the primary blocking issue. A fourth new workflow file, create-gh-security-scanning-issues.yml, was added but:

  1. It reads .SamplePaths (the old field name), which was renamed to .AffectedPaths by Get-CodeScanningAlerts.ps1 in this same PR (see inline comment, line 44 of that file). Every issue body created by this workflow would have an empty paths section.
  2. Its deduplication logic searches by title ("[Security] ${RULE_DESC}" in:title) rather than by the HTML-comment marker in the body ("${MARKER}" in:body) — diverging from the strategy in create-gh-code-scanning-issues.yml.
  3. It is not referenced by weekly-gh-code-scanning.yml or any other orchestrator. It is unreachable dead code.

Action required: Either remove create-gh-security-scanning-issues.yml entirely, or update it to use AffectedPaths, align its deduplication logic, and wire it into the scheduler.

⚠️ Shell expansion inside ISSUE_BODY double-quoted string

ISSUE_BODY is assigned via a double-quoted string that embeds $(...) subshell expressions populated with API-sourced values such as FINDING_DESC. Bash performs word-splitting and command substitution inside double-quoted variable assignments, so a FINDING_DESC value containing $(cmd) would execute cmd. The risk is low given the source (GitHub API), but it violates the principle of defense-in-depth. See inline comment on line 81.


✅ Action Items

  1. Remove or fix create-gh-security-scanning-issues.yml — the file is broken (stale field) and unreachable. This is the primary blocker.
  2. Add set -euo pipefail to the run: block in create-gh-code-scanning-issues.yml.
  3. Mitigate shell expansion in the ISSUE_BODY construction in create-gh-code-scanning-issues.yml.
  4. Remove security-events: read from create-gh-code-scanning-issues.yml (both workflow and job level).
  5. Check the "Documentation is updated" checkboxSKILL.md was updated.
  6. Add the missing ## Security Considerations section from the PR template.
  7. Address npm run plugin:generate and npm run docs:test in the automated checks checklist.

Note

🔒 Integrity filter blocked 1 item

The following item were blocked because they don't meet the GitHub integrity level.

  • #1329 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by PR Review for issue #1495 · ● 2.8M

COUNT=$(echo "$alert" | jq -r '.Count')
PATHS=$(echo "$alert" | jq -r '.SamplePaths | join(", ")')
MARKER="automation:security-scan:${RULE_ID}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Stale field name — SamplePaths was renamed to AffectedPaths in this PR

This line reads .SamplePaths from the JSON output, but Get-CodeScanningAlerts.ps1 was updated in this same PR to emit AffectedPaths instead. As written this line produces an empty string for every alert.

# Fix: use the new field name
PATHS=$(echo "$alert" | jq -r '.AffectedPaths | join(", ")')

More broadly, this entire file appears to be a leftover from an earlier development iteration. It is not referenced by any orchestrator workflow in this PR, and its deduplication strategy (title search on line 55) diverges from the body-marker approach adopted by create-gh-code-scanning-issues.yml. Please either remove this file or update it to use the current API and wire it into the scheduler.

- name: Create backlog issues for new findings
shell: bash
run: |
while IFS= read -r alert; do
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing strict error handling — set -euo pipefail required

Per the bash script conventions for this repository (.github/instructions/hve-core/bash.instructions.md), every bash run: block must begin with set -euo pipefail to ensure the step fails immediately on any unhandled error. Without it, a failed jq call or a failed gh issue create silently swallows the error and the loop continues, potentially producing incomplete or malformed issue tracking.

      - name: Create backlog issues for new findings
        shell: bash
        run: |
          set -euo pipefail
          while IFS= read -r alert; do

$([ -n "${ALERT_URL}" ] && echo "**Alert:** ${ALERT_URL}")

### What was found
$([ -n "${FINDING_DESC}" ] && echo "${FINDING_DESC}" || echo "See the linked alert for details.")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Shell expansion inside a double-quoted string — potential injection vector

ISSUE_BODY is assembled as a double-quoted string ("..."). Subshell expressions such as $([ -n "\$\{FINDING_DESC}" ] && echo "\$\{FINDING_DESC}") are expanded by bash inside double quotes. If FINDING_DESC (derived from most_recent_instance.message.text in a GitHub API response) contains a pattern like $(malicious_command), bash will execute it during variable assignment.

While the risk is low in practice because GitHub API values are controlled, defense-in-depth recommends preventing expansion by writing the body to a temporary file instead of a shell variable, or by using printf '%s' with a heredoc:

ISSUE_BODY=$(cat <<'BODY_EOF'
<!-- \$\{MARKER} -->
BODY_EOF
printf "**Rule:** \`%s\`\n**Tool:** %s\n...\n" "\$\{RULE_ID}" "\$\{TOOL}" >> /tmp/issue_body_$$.md)

Alternatively, build the body with jq --rawfile or jq -n --arg ... to keep all values properly escaped and isolated from shell expansion.


permissions:
issues: write
security-events: read
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 security-events: read is not used by this job

This workflow downloads a pre-generated artifact and calls gh issue create/edit. It never queries the code-scanning API directly, so security-events: read is unnecessary here. Removing it keeps the permissions footprint at the minimum required (principle of least privilege per the workflow convention instructions).

permissions:
  issues: write
  # Remove: security-events: read

Apply the same change to the job-level permissions block on line 22.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: feat(workflows): add weekly GitHub code scanning automation

Overview

The core design — a composable three-workflow architecture for weekly code scanning automation — is well-structured and largely follows repository conventions. The SKILL.md update, PowerShell script enhancements, and test changes are all solid. However, one blocking issue must be addressed before this can merge.


Issue Alignment ✅

PR closes #1329. The implementation scope (three reusable workflows, skill improvements, enriched alert output) is coherent with the stated goal of automating weekly security backlog management. No significant scope gaps were identified.


PR Template Compliance ⚠️

The Copilot skill checkbox is checked under Type of Change (correct — SKILL.md was modified), but the AI Artifact Contributions checklist entries carry "(N/A — no AI artifacts changed)" annotations. These are contradictory. Additionally, the Sample Prompts section is empty, which should be filled when a SKILL.md change is declared.

Please either:

  • Fill the Sample Prompts section with a brief invocation example for the updated skill, or
  • Uncheck the Copilot skill Type of Change checkbox and note the SKILL.md changes are purely structural/documentation updates.

Code Quality ❌ — Blocking

.github/workflows/create-gh-security-scanning-issues.yml must be removed or corrected.

See the inline comment at line 42 for the full analysis. In summary:

  1. The workflow references .SamplePaths (renamed to .AffectedPaths in this PR) — it will silently produce null in every issue body.
  2. It expects the artifact gh-security-scanning-alerts, but gh-code-scanning.yml only uploads gh-code-scanning-alerts — the download step will fail immediately.
  3. It is not described in the PR description, not called by any orchestrator in this PR, and appears to be an accidental inclusion of an early draft.

Recommended action: Remove this file from the PR.


Coding Standards ✅ (with minor notes)

  • All three new workflows properly SHA-pin their actions using the repository-standard SHAs with semantic version comments. ✅
  • persist-credentials: false is correctly applied on the checkout step. ✅
  • concurrency is set on the orchestrator with cancel-in-progress: false. ✅
  • Top-level and job-level permissions are declared on all workflows. ✅
  • Minor: create-gh-code-scanning-issues.yml includes security-events: read at both permission scopes, but this workflow never reads security events directly. See inline comment at line 14.

Skill and Script Quality ✅ (with minor note)

The Get-CodeScanningAlerts.ps1 enrichment (Severity, AffectedPaths, HasFilePaths, AlertUrl, FindingDescription) is well-implemented and the test suite was updated to match. The GroupedJson alias is a nice ergonomic improvement.

Minor: The sentinel path filter uses a broad substring regex (-notmatch '(?i)no file') that could incorrectly drop legitimate paths. See inline comment at line 91 of the PowerShell script.


Required Actions

# Severity Action
1 🔴 Blocking Remove (or fully update) .github/workflows/create-gh-security-scanning-issues.yml
2 🟡 Minor Tighten the sentinel filter in Get-CodeScanningAlerts.ps1 to an exact string match
3 🟡 Minor Remove security-events: read from create-gh-code-scanning-issues.yml permissions
4 🔵 Info Resolve the PR template contradiction (Copilot skill checkbox vs AI Artifact annotations)

Note

🔒 Integrity filter blocked 1 item

The following item were blocked because they don't meet the GitHub integrity level.

  • #1329 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by PR Review for issue #1495 · ● 2.5M

SEVERITY=$(echo "$alert" | jq -r '.SecuritySeverity // "unspecified"')
TOOL=$(echo "$alert" | jq -r '.Tool')
COUNT=$(echo "$alert" | jq -r '.Count')
PATHS=$(echo "$alert" | jq -r '.SamplePaths | join(", ")')
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Stale field reference — workflow will fail at runtime

This line references .SamplePaths, a field renamed to .AffectedPaths in Get-CodeScanningAlerts.ps1 as part of this same PR. jq silently returns null for missing keys, so PATHS will be the literal string null in every issue body.

Beyond the field name, this file has several additional problems that suggest it is an accidental inclusion of an early draft:

  • Artifact name mismatch: expects gh-security-scanning-alerts (line 10), but gh-code-scanning.yml only uploads gh-code-scanning-alerts. The download step would fail with "no artifact found".
  • Not mentioned in the PR description: the description documents exactly three workflows (gh-code-scanning.yml, create-gh-code-scanning-issues.yml, weekly-gh-code-scanning.yml). This file is a fourth that is unaccounted for.
  • Not called by any orchestrator: weekly-gh-code-scanning.yml only calls create-gh-code-scanning-issues.yml. This workflow is dead code.
  • Stale deduplication logic: searches by title ("[Security] \$\{RULE_DESC}" in:title) rather than the body marker (\$\{MARKER} in:body) used in create-gh-code-scanning-issues.yml, and it never updates existing issues.

Recommended fix: Remove .github/workflows/create-gh-security-scanning-issues.yml from the PR entirely. If a distinct "security scanning" variant is needed in the future, it should be addressed in a separate, fully described PR.


permissions:
issues: write
security-events: read
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 security-events: read is not required by this workflow

This workflow only (1) downloads a pre-uploaded artifact and (2) creates/updates GitHub issues via gh issue CLI commands. Neither operation requires direct access to the GitHub Security Events API.

Per the repository's workflow convention, permissions follow the principle of least privilege. security-events: read should be removed from both the top-level permissions block (here) and the job-level permissions block (line 22).

Suggested fix:

permissions:
  issues: write

$paths = @(
$_.Group |
ForEach-Object { $_.most_recent_instance.location.path } |
Where-Object { $_ -and $_ -notmatch '(?i)no file' } |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Overly broad sentinel filter

The regex -notmatch '(?i)no file' will silently drop any path containing the substring no file (case-insensitive), including legitimate paths such as /services/no-file-cache/handler.ps1 or scripts/nofile-cleanup.py.

Suggested fix: Match the exact sentinel string the GitHub API returns for repo-level alerts:

Where-Object { $_ -and $_ -ne 'no file associated with this alert' }

This is unambiguous, doesn't rely on a substring heuristic, and matches the existing test fixture string verbatim.

Copy link
Copy Markdown
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice addition - splitting this into a workflow_call scan job, a reusable issue-creation job, and a thin schedule orchestrator gives us the right composability, and the per-rule dedup marker (automation:security-scan:${RULE_ID}) plus the weekly count refresh is exactly the deterministic floor a security backlog needs. The skill changes (AffectedPaths, HasFilePaths, AlertUrl, FindingDescription, Severity fallback) are well-scoped and the rendered issues from the test run (#1499#1502) read clearly.

A few items to address before merge - full list and validation results are in the review thread; the only blocker is the orphan create-gh-security-scanning-issues.yml (leftover from the rename, unreferenced, still uses the removed SamplePaths field). Beyond that: the automated label isn't in labels.yml, the (?i)no file filter is too loose, and the new skill output fields aren't unit-tested.

On the agentic-workflow angle (and follow-up #1510): I'd keep this PR exactly as a deterministic floor - predictable, zero LLM cost, easy to reason about, easy to roll back. The places where a gh-aw layer would genuinely add value are on top of this, not in place of it: narrative bodies for repo-level rules like BranchProtectionID where there is no source file to link, dedup-by-root-cause across CodeQL rules that share a fix, auto-closing issues when the next scan shows the alert dismissed or fixed, and linking related findings as sub-issues (the "Issue Arborist" / "Sub-Issue Closer" patterns from githubnext/agentics - https://github.com/githubnext/agentics). Crucially the title format and the automation:security-scan:${RULE_ID} marker should stay deterministic so the agentic layer can match against them safely. Guardrails on the gh-aw side should pin safe-outputs.create-issue to title-prefix: "[Security]" and a fixed label allowlist, cap add-comment per run, and keep the agent itself read-only with the write job gated - i.e., the agent requests, a scoped job decides. That keeps this PR's deterministic guarantees intact while letting the agent earn its keep on the parts that actually need contextual reasoning.

Left some inline comments and sharing a list of low items:

Low / Informational

💡 L1. gh issue edit ... --add-label runs as a second API call

.github/workflows/create-gh-code-scanning-issues.yml

gh issue edit accepts --add-label in the same call as --title/--body. Combining them halves API calls per existing-issue update.

💡 L2. MARKER is also a stable id; consider promoting to a typed field in skill output

The skill could add a MarkerId = "automation:security-scan:${RuleId}" field so consumers do not re-derive it inconsistently. Optional.

💡 L3. Body uses Markdown task list inside an automated message

The "Action Required" checkboxes will never get programmatically updated, and weekly comment refreshes do not reset them. Cosmetic, but consider plain bullet text or a separate "Reviewer checklist" section to set expectations.

@@ -0,0 +1,67 @@
name: Create GitHub Security Scanning Issues
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 B1. Orphan/dead workflow file: create-gh-security-scanning-issues.yml

.github/workflows/create-gh-security-scanning-issues.yml

This 67-line file is a leftover from the rename in commit 2dc55203 feat(workflows): rename gh-security-scanning to gh-code-scanning. It is never referenced anywhere (weekly-gh-code-scanning.yml only calls create-gh-code-scanning-issues.yml), still uses the old SamplePaths field name (which the skill no longer emits), and contradicts the PR description which says "three new workflows added".

If left in place it is unreachable code that future readers will mistake for a second active code path, and any reusable workflow consumer who picks the wrong file will get broken behavior (it expects the old artifact name gh-security-scanning-alerts and the removed SamplePaths field).

Action: Delete .github/workflows/create-gh-security-scanning-issues.yml.

--search "\"${MARKER}\" in:body" \
--state open --json number --jq '.[0].number // empty')

ISSUE_BODY="<!-- ${MARKER} -->
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 M1. Bash here-doc style is fragile and non-idiomatic

.github/workflows/create-gh-code-scanning-issues.yml

The body is built with a multi-line double-quoted assignment whose internal indentation (10 spaces) only happens to render correctly because YAML strips the block-scalar indent first. Any future edit that nudges indentation by one column will silently turn the issue body into a code block. Use a cat <<'EOF' heredoc piped through envsubst, or build the body with printf using explicit \n, so the content is independent of YAML indent.

Suggested rewrite (heredoc):

ISSUE_BODY=$(cat <<EOF
<!-- ${MARKER} -->
## Code Scanning Alert: ${RULE_DESC}

**Rule:** \`${RULE_ID}\`
$([ -n "${SEVERITY}" ] && echo "**Severity:** ${SEVERITY}")
**Tool:** ${TOOL}
...
EOF
)

This also removes the trailing-quote-on-its-own-line pattern that is easy to misalign.

--repo "${OWNER}/${REPO}" \
--title "${ISSUE_TITLE}" \
--body "${ISSUE_BODY}"
gh issue edit "${existing}" \
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 M2. automated label is referenced but missing from labels.yml

.github/workflows/create-gh-code-scanning-issues.yml and .github/workflows/create-gh-code-scanning-issues.yml

Workflows here, in create-stale-docs-issues.yml, and in weekly-security-maintenance.yml all attach automated, but .github/labels.yml only declares automation (line 70). The automated label currently exists in the live repo only because someone created it ad-hoc; a fresh fork or a future sweep will fail issue creation with could not add label. Either:

  • Add automated to labels.yml (preferred - preserves existing convention), or
  • Switch all three workflows to automation.

This is pre-existing technical debt, but it is being entrenched by this PR; fix it here.

$paths = @(
$_.Group |
ForEach-Object { $_.most_recent_instance.location.path } |
Where-Object { $_ -and $_ -notmatch '(?i)no file' } |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 M3. Path filter regex is too loose

.github/skills/github/gh-code-scanning/scripts/Get-CodeScanningAlerts.ps1

Where-Object { $_ -and $_ -notmatch '(?i)no file' }

(?i)no file is an unanchored substring, so any future legitimate path containing the bytes "no file" (case-insensitive - e.g. docs/known-files-list.md? no; path/to/no-file-found.md? yes) gets dropped. Anchor the match to the GitHub sentinel exactly:

Where-Object { $_ -and $_ -ne 'no file associated with this alert' }

default: gh-code-scanning-alerts

permissions:
issues: write
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 M4. Unused security-events: read on issue-creation job

.github/workflows/create-gh-code-scanning-issues.yml

The issue-creation job only reads a workflow artifact and writes issues. security-events: read is granted at both workflow and job scope but the job never calls a security-events API. Drop it (principle of least privilege).

type: string
default: gh-code-scanning-alerts

permissions:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 M5. Reusable workflow does not declare actions: read for download-artifact@v8

.github/workflows/create-gh-code-scanning-issues.yml

Same-run artifact downloads are fine, but actions/download-artifact@v8 documents actions: read as required when artifacts could come from another run. This workflow only consumes its own job's artifact today, so no immediate breakage; still, declaring actions: read future-proofs against run-id parameterization, which is a natural extension of a reusable workflow.

}

It 'Serializes SamplePaths as a JSON array even when only one path exists' {
It 'Serializes AffectedPaths as a JSON array even when only one path exists' {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 M6. New skill output fields are not under unit test

.github/skills/github/gh-code-scanning/tests/Get-CodeScanningAlerts.Tests.ps1

The PR adds Severity, AlertUrl, and FindingDescription to the grouped output. Tests cover only AffectedPaths/HasFilePaths. Add at least one assertion per new field (e.g. $parsed[0].Severity | Should -Be 'warning', $parsed[0].AlertUrl | Should -Match '/security/code-scanning/') so a future regression that drops one of these silently does not also silently degrade the issue body.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(ci): Create weekly code scanning workflows for automated backlog issue creation

3 participants