Skip to content

Cache virtualenv between jobs #401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mgoerens merged 1 commit into from
May 14, 2025
Merged

Cache virtualenv between jobs #401

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/actions/install-ci-scripts/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Install our CI scripts
description: |
Handles installation and caching/restore of the CI scripts in a virtualenv.
inputs:
working-directory:
description: Basedir in which to save / restore cache and install venv.
default: './'
required: false
python-version:
description: Installed Python version
required: true
runs:
using: composite
steps:
# This attempts to restore the virtualenv from a GitHub cache with the corresponding cache key.
# On cache-miss, this automatically adds a post step that creates a cache from this path and using this cache key.
- name: Cache virtualenv
id: cache
uses: actions/cache@v4
with:
key: venv-${{ runner.os }}-${{ inputs.python-version }}-${{ hashFiles('scripts/requirements.txt') }}
path: ${{ inputs.working-directory }}/ve1

- name: Setup venv on cache-miss
if: steps.cache.outputs.cache-hit != 'true'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
5 changes: 5 additions & 0 deletions .github/actions/setup-python/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,14 @@ description: |

Serves as the single place to update python versions over time across the
project.
outputs:
python-version:
description: Version of Python installed
value: ${{ steps.setup-python.outputs.python-version }}
runs:
using: composite
steps:
- uses: actions/setup-python@v5
id: setup-python
with:
python-version: '3.10'
14 changes: 5 additions & 9 deletions .github/workflows/behave.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,17 +116,13 @@ jobs:
fetch-depth: ${{ inputs.checkout-fetch-depth }}

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Set up CI scripts
run: |
# set up python scripts
echo "set up python script in $PWD"
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

# Pull request numbers are included in generated chart names in E2E, so it's included
# as an environment variable which E2E consumes.
Expand Down
65 changes: 25 additions & 40 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,13 @@ jobs:
uses: actions/checkout@v4

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Set up Python 3.x Part 2
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

- name: Check for CI changes
id: check_ci_changes
Expand Down Expand Up @@ -128,16 +125,13 @@ jobs:
path: "pr-branch"

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Set up Python 3.x Part 2
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

- name: Extract PR information
id: validate-submission
Expand Down Expand Up @@ -217,16 +211,13 @@ jobs:
path: "pr-branch"

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Set up Python 3.x Part 2
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

- name: Download submission information
uses: actions/download-artifact@v4
Expand Down Expand Up @@ -410,16 +401,13 @@ jobs:
path: "pr-branch"

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Set up Python 3.x Part 2
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

# Submission information should always be present
- name: Download submission information
Expand Down Expand Up @@ -537,16 +525,13 @@ jobs:
path: "pr-branch"

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Set up Python 3.x Part 2
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

- name: install chart verifier for action
uses: redhat-actions/openshift-tools-installer@v1
Expand Down
18 changes: 7 additions & 11 deletions .github/workflows/check-contributor.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,18 +43,14 @@ jobs:
uses: actions/checkout@v4

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Install CI Scripts
run: |
# set up python scripts
echo "set up python script in $PWD"
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..


- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

- name: Check contributor
# The return code from this script is what's important in this workflow.
id: check-contributor
Expand Down
13 changes: 5 additions & 8 deletions .github/workflows/check-locks-on-owners-submission.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,16 +30,13 @@ jobs:
uses: actions/checkout@v4

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Set up Python 3.x Part 2
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

- name: get files changed
id: get_files_changed
Expand Down
13 changes: 5 additions & 8 deletions .github/workflows/lock-sanity-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,12 @@ jobs:
- name: checkout
uses: actions/checkout@v4
- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python
- name: Set up Python 3.x Part 2
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}
- name: generate chart locks
id: generate
uses: ./.github/actions/generate-chart-locks
Expand Down
13 changes: 5 additions & 8 deletions .github/workflows/mercury_bot.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,13 @@ jobs:
uses: actions/checkout@v4

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Set up Python 3.x Part 2
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

- name: get files changed
id: get_files_changed
Expand Down
12 changes: 5 additions & 7 deletions .github/workflows/metrics.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,15 +24,13 @@ jobs:
uses: actions/checkout@v4

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Set up Python 3.x Part 2
run: |
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

- name: Release Metrics
id: release_metrics
Expand Down
13 changes: 5 additions & 8 deletions .github/workflows/owners.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,13 @@ jobs:
uses: actions/checkout@v4

- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python

- name: Set up Python 3.x Part 2
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}

- uses: jitterbit/get-changed-files@v1
id: filesChangedOrModified
Expand Down
13 changes: 5 additions & 8 deletions .github/workflows/python-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,12 @@ jobs:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
id: setup-python
uses: ./.github/actions/setup-python
- name: Set up Python 3.x Part 2
run: |
# set up python
python3 -m venv ve1
cd scripts
../ve1/bin/pip3 install -r requirements.txt
../ve1/bin/pip3 install .
cd ..
- name: Install CI scripts
uses: ./.github/actions/install-ci-scripts
with:
python-version: ${{ steps.setup-python.outputs.python-version }}
- name: Run pytest
run: |
./ve1/bin/pytest scripts/src/submission/
Loading