-
Notifications
You must be signed in to change notification settings - Fork 0
Add automated Python and Docker version update workflow with issue tracking #20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Copilot
wants to merge
9
commits into
main
Choose a base branch
from
copilot/update-python-docker-workflow
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
06b8603
Initial plan
Copilot 8f0b56b
Add Python and Docker update workflow and script
Copilot f933d5f
Address code review feedback: make version detection more flexible
Copilot e926174
Add documentation for Python/Docker update workflow
Copilot 16ffca6
Implement issue-first workflow and address review feedback
Copilot e8dc8ad
Update README to reflect issue-first workflow
Copilot 7099c0a
Address final code review feedback
Copilot 8b949d9
Fix critical workflow and script issues
Copilot 99bec76
Fix false positive detection and workflow cancellation handling
Copilot File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,341 @@ | ||
| name: Update Python and Docker versions | ||
|
|
||
| on: | ||
| # Run every 2 months (first day at 3 AM UTC) | ||
| schedule: | ||
| - cron: '0 3 1 */2 *' | ||
| # Allow manual triggering for testing | ||
| workflow_dispatch: {} | ||
|
|
||
| concurrency: | ||
| group: python-docker-update | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| update-python: | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| issues: write | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Get current Python version | ||
| id: current-version | ||
| run: | | ||
| # Extract current Python version from build-ci-image.yml | ||
| CURRENT_VERSION=$(grep -m1 "python-version:" .github/workflows/build-ci-image.yml | sed -E "s/.*python-version:[[:space:]]*['\"]?([0-9.]+)['\"]?.*/\1/") | ||
| echo "version=$CURRENT_VERSION" >> $GITHUB_OUTPUT | ||
| echo "Using Python $CURRENT_VERSION for workflow execution" | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ steps.current-version.outputs.version }} | ||
|
|
||
|
Comment on lines
+33
to
+37
|
||
| - name: Check for Python updates | ||
| id: check-update | ||
| run: | | ||
| # Run the update script; if it fails, this step (and workflow) will fail. | ||
| set -euo pipefail | ||
| python3 scripts/update_python_version.py | ||
|
|
||
| # Determine whether any tracked files were modified by the script. | ||
| # git diff exits with 1 if there are differences, so disable pipefail for this check | ||
| set +e | ||
| git diff --quiet | ||
| DIFF_EXIT=$? | ||
| set -e | ||
|
|
||
| if [ $DIFF_EXIT -eq 0 ]; then | ||
| echo "update_available=false" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "update_available=true" >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| - name: Get updated Python version | ||
| id: get-version | ||
| if: steps.check-update.outputs.update_available == 'true' | ||
| run: | | ||
| # Extract the new Python version from build-ci-image.yml | ||
| NEW_VERSION=$(grep -m1 "python-version:" .github/workflows/build-ci-image.yml | sed -E "s/.*python-version:[[:space:]]*['\"]?([0-9.]+)['\"]?.*/\1/") | ||
| echo "new_version=$NEW_VERSION" >> $GITHUB_OUTPUT | ||
| echo "Updated to Python $NEW_VERSION" | ||
|
|
||
| - name: Set up new Python version for testing | ||
| if: steps.check-update.outputs.update_available == 'true' | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ steps.get-version.outputs.new_version }} | ||
|
|
||
| - name: Create venv and install deps | ||
| if: steps.check-update.outputs.update_available == 'true' | ||
| run: | | ||
| set -euo pipefail | ||
| python3 -m venv .venv | ||
| ./.venv/bin/python -m pip install --upgrade pip | ||
| ./.venv/bin/python -m pip install -r scripts/requirements.txt -r scripts/requirements-dev.txt | ||
|
|
||
| - name: Run linting tests | ||
| id: lint-test | ||
| if: steps.check-update.outputs.update_available == 'true' | ||
| continue-on-error: true | ||
| run: | | ||
| set -euo pipefail | ||
| echo "Running flake8..." | ||
| ./.venv/bin/python -m flake8 -q | ||
| echo "lint_passed=true" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Run pytest tests | ||
| id: pytest-test | ||
| if: steps.check-update.outputs.update_available == 'true' | ||
| continue-on-error: true | ||
| run: | | ||
| set -euo pipefail | ||
| echo "Running pytest..." | ||
| ./.venv/bin/python -m pytest -q | ||
| echo "tests_passed=true" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Set up Docker Buildx | ||
| if: steps.check-update.outputs.update_available == 'true' | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Test build Docker images | ||
| id: docker-test | ||
| if: steps.check-update.outputs.update_available == 'true' | ||
| continue-on-error: true | ||
| run: | | ||
| set -euo pipefail | ||
| echo "Testing QR Dockerfile build..." | ||
| docker build -f .github/ci/Dockerfile.qr -t test-qr:latest . | ||
| echo "Testing Infra Dockerfile build..." | ||
| docker build -f .github/ci/Dockerfile.infra -t test-infra:latest . | ||
| echo "docker_build_passed=true" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Run tests in Docker container | ||
| id: docker-pytest | ||
| if: steps.check-update.outputs.update_available == 'true' | ||
| continue-on-error: true | ||
| run: | | ||
| set -euo pipefail | ||
| echo "Running pytest in Docker container..." | ||
| docker run --rm --user 0 -v "$PWD:/workspace" -w /workspace test-infra:latest bash -c "python -m venv --system-site-packages .venv && ./.venv/bin/python -m pytest -q" | ||
| echo "docker_tests_passed=true" >> $GITHUB_OUTPUT | ||
|
Comment on lines
+81
to
+125
|
||
|
|
||
| - name: Check for existing update issue | ||
| id: check-issue | ||
| if: steps.check-update.outputs.update_available == 'true' | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| echo "Checking for existing Python update issues..." | ||
| EXISTING_ISSUES=$(gh issue list --state open --label "python-update" --json number,title --limit 1) | ||
| if [ "$EXISTING_ISSUES" != "[]" ]; then | ||
| ISSUE_NUMBER=$(echo "$EXISTING_ISSUES" | jq -r '.[0].number') | ||
| echo "Found existing open issue #$ISSUE_NUMBER" | ||
| echo "has_existing=true" >> $GITHUB_OUTPUT | ||
| echo "issue_number=$ISSUE_NUMBER" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "No existing open Python update issue found" | ||
| echo "has_existing=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| - name: Check for existing update PR | ||
| id: check-pr | ||
| if: steps.check-update.outputs.update_available == 'true' | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| echo "Checking for existing Python update PRs..." | ||
| EXISTING_PRS=$(gh pr list --state open --label "python-update" --json number,title --limit 1) | ||
| if [ "$EXISTING_PRS" != "[]" ]; then | ||
| PR_NUMBER=$(echo "$EXISTING_PRS" | jq -r '.[0].number') | ||
| echo "Found existing open PR #$PR_NUMBER" | ||
| echo "has_existing=true" >> $GITHUB_OUTPUT | ||
| echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "No existing open Python update PR found" | ||
| echo "has_existing=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| - name: Create issue for update | ||
| id: create-issue | ||
| if: | | ||
| steps.check-update.outputs.update_available == 'true' && | ||
| steps.lint-test.outcome == 'success' && | ||
| steps.pytest-test.outcome == 'success' && | ||
| steps.docker-test.outcome == 'success' && | ||
| steps.docker-pytest.outcome == 'success' && | ||
| steps.check-issue.outputs.has_existing != 'true' && | ||
| steps.check-pr.outputs.has_existing != 'true' | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| NEW_VERSION="${{ steps.get-version.outputs.new_version }}" | ||
|
|
||
| # Ensure the 'python-update' label exists | ||
| if ! gh label view "python-update" >/dev/null 2>&1; then | ||
| echo "Creating missing label 'python-update'" | ||
| gh label create "python-update" --description "Automated Python version updates" --color 0366d6 || true | ||
| fi | ||
|
|
||
| # Create issue with heredoc | ||
| cat > /tmp/issue_body.txt << 'ISSUE_EOF' | ||
| ## Python and Docker Version Update Available | ||
|
|
||
| A new stable Python version is available. | ||
|
|
||
| ### Proposed Changes | ||
| - Update `.github/ci/Dockerfile.infra` to use new Python slim image | ||
| - Update `.github/ci/Dockerfile.qr` to use new Python slim image | ||
| - Update `.github/workflows/build-ci-image.yml` to use new Python version | ||
| - Update `.github/workflows/check-todo.yml` to use new Python version | ||
|
|
||
| ### Automated Test Results | ||
| All automated tests have passed with the new version: | ||
| - ✅ Linting (flake8) | ||
| - ✅ Unit tests (pytest) | ||
| - ✅ Docker image builds | ||
| - ✅ Tests in Docker containers | ||
|
|
||
| ### Next Steps | ||
| A pull request will be automatically created and linked to this issue with the necessary changes. | ||
| ISSUE_EOF | ||
|
|
||
| ISSUE_NUMBER=$(gh issue create \ | ||
| --title "Update Python to version ${NEW_VERSION}" \ | ||
| --body-file /tmp/issue_body.txt \ | ||
| --label "python-update" \ | ||
| --json number -q '.number') | ||
|
|
||
| echo "issue_number=$ISSUE_NUMBER" >> $GITHUB_OUTPUT | ||
| echo "Created issue #$ISSUE_NUMBER" | ||
|
|
||
| - name: Create Pull Request | ||
| if: | | ||
| steps.check-update.outputs.update_available == 'true' && | ||
| steps.lint-test.outcome == 'success' && | ||
| steps.pytest-test.outcome == 'success' && | ||
| steps.docker-test.outcome == 'success' && | ||
| steps.docker-pytest.outcome == 'success' && | ||
| steps.check-issue.outputs.has_existing != 'true' && | ||
| steps.check-pr.outputs.has_existing != 'true' | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| # Configure git | ||
| git config user.name "github-actions[bot]" | ||
| git config user.email "github-actions[bot]@users.noreply.github.com" | ||
|
|
||
| NEW_VERSION="${{ steps.get-version.outputs.new_version }}" | ||
| ISSUE_NUMBER="${{ steps.create-issue.outputs.issue_number }}" | ||
| BRANCH_NAME="automated/update-python-${NEW_VERSION}" | ||
|
|
||
| # Create and switch to new branch | ||
| git checkout -b "$BRANCH_NAME" | ||
|
|
||
| # Add changes | ||
| git add .github/ci/Dockerfile.infra | ||
| git add .github/ci/Dockerfile.qr | ||
| git add .github/workflows/build-ci-image.yml | ||
| git add .github/workflows/check-todo.yml | ||
|
|
||
| # Commit changes with proper formatting | ||
| git commit \ | ||
| -m "chore: update Python to version ${NEW_VERSION}" \ | ||
| -m "" \ | ||
| -m "- Update Dockerfile.infra to use python:${NEW_VERSION}-slim" \ | ||
| -m "- Update Dockerfile.qr to use python:${NEW_VERSION}-slim" \ | ||
| -m "- Update build-ci-image.yml to use Python ${NEW_VERSION}" \ | ||
| -m "- Update check-todo.yml to use Python ${NEW_VERSION}" \ | ||
| -m "" \ | ||
| -m "All tests passed:" \ | ||
| -m "- Linting: ✓" \ | ||
| -m "- Pytest: ✓" \ | ||
| -m "- Docker builds: ✓" \ | ||
| -m "- Docker tests: ✓" \ | ||
| -m "" \ | ||
| -m "Fixes #${ISSUE_NUMBER}" | ||
|
|
||
| # Push branch (use force-with-lease to allow updates if no remote changes) | ||
| if ! git push --force-with-lease origin "$BRANCH_NAME"; then | ||
| echo "⚠️ Force-with-lease failed - remote branch has been updated by another process" | ||
| echo "Attempting regular push (will fail if branch exists with different history)" | ||
| git push origin "$BRANCH_NAME" | ||
| fi | ||
|
|
||
| # Create PR body with heredoc | ||
| cat > /tmp/pr_body.txt << 'PR_EOF' | ||
| ## Automated Python and Docker Version Update | ||
|
|
||
| This PR updates the Python version used across the repository. | ||
|
|
||
| ### Changes Made | ||
| - Updated `.github/ci/Dockerfile.infra` to use new Python slim image | ||
| - Updated `.github/ci/Dockerfile.qr` to use new Python slim image | ||
| - Updated `.github/workflows/build-ci-image.yml` to use new Python version | ||
| - Updated `.github/workflows/check-todo.yml` to use new Python version | ||
|
|
||
| ### Test Results | ||
| All automated tests have passed: | ||
| - ✅ Linting (flake8) | ||
| - ✅ Unit tests (pytest) | ||
| - ✅ Docker image builds | ||
| - ✅ Tests in Docker containers | ||
|
|
||
| ### What to Review | ||
| - Verify that all CI workflows pass successfully | ||
| - Check that no new deprecation warnings are introduced | ||
| - Ensure Docker images build and push correctly | ||
|
|
||
| This PR was automatically created by the `update-python-docker` workflow. | ||
| PR_EOF | ||
|
|
||
| # Add issue reference to PR body | ||
| echo "" >> /tmp/pr_body.txt | ||
| echo "Closes #${ISSUE_NUMBER}" >> /tmp/pr_body.txt | ||
|
|
||
| # Create PR linked to the issue | ||
| gh pr create \ | ||
| --title "chore: Update Python to version ${NEW_VERSION}" \ | ||
| --body-file /tmp/pr_body.txt \ | ||
| --label "python-update" \ | ||
| --base main \ | ||
| --head "$BRANCH_NAME" | ||
|
|
||
| - name: Handle test failures | ||
| if: | | ||
| !cancelled() && | ||
| steps.check-update.outputs.update_available == 'true' && | ||
| (steps.lint-test.outcome == 'failure' || | ||
| steps.pytest-test.outcome == 'failure' || | ||
| steps.docker-test.outcome == 'failure' || | ||
| steps.docker-pytest.outcome == 'failure') | ||
| run: | | ||
| echo "❌ Tests failed with the new Python version" | ||
| echo "Lint outcome: ${{ steps.lint-test.outcome }}" | ||
| echo "Pytest outcome: ${{ steps.pytest-test.outcome }}" | ||
| echo "Docker build outcome: ${{ steps.docker-test.outcome }}" | ||
| echo "Docker tests outcome: ${{ steps.docker-pytest.outcome }}" | ||
| echo "" | ||
| echo "Manual intervention required to fix compatibility issues." | ||
| echo "The version update has been applied but not committed." | ||
| exit 1 | ||
|
|
||
| - name: No update needed | ||
| if: steps.check-update.outputs.update_available != 'true' | ||
| run: | | ||
| echo "✅ Already using the latest Python version" | ||
|
|
||
| - name: Skip - existing issue or PR found | ||
| if: steps.check-update.outputs.update_available == 'true' && (steps.check-issue.outputs.has_existing == 'true' || steps.check-pr.outputs.has_existing == 'true') | ||
| run: | | ||
| echo "ℹ️ Skipping creation - existing Python update issue/PR found" | ||
| if [ "${{ steps.check-issue.outputs.has_existing }}" == "true" ]; then | ||
| echo "Existing issue: #${{ steps.check-issue.outputs.issue_number }}" | ||
| fi | ||
| if [ "${{ steps.check-pr.outputs.has_existing }}" == "true" ]; then | ||
| echo "Existing PR: #${{ steps.check-pr.outputs.pr_number }}" | ||
| fi | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The workflow doesn't define a concurrency group, which could allow multiple simultaneous runs if triggered both by schedule and manual dispatch. This could lead to duplicate PR creation attempts or wasted CI resources running the same tests multiple times. Consider adding a concurrency configuration at the job or workflow level to cancel in-progress runs when a new one starts, similar to how other workflows might handle this.