Skip to content

Parallelize CI test execution for faster build times #57

@GeorgGrassler

Description

@GeorgGrassler

Problem

Currently, our GitHub CI tests run sequentially in a single job:

  1. make lorentz
  2. make ql
  3. make performance_ql (PR only)
  4. make par (PR only)

This results in longer CI times as tests wait for each other to complete.

Proposed Solution

Split tests into separate parallel jobs:

jobs:
  lorentz-test:
    runs-on: ubuntu-24.04
    steps: [build, run lorentz]
  
  ql-test:
    runs-on: ubuntu-24.04  
    steps: [build, run ql]
  
  performance-test:
    runs-on: ubuntu-24.04
    steps: [build, run performance_ql]
  
  par-test:
    runs-on: ubuntu-24.04
    steps: [build, run par]

Considerations

  • Need to handle build artifacts sharing or duplicate builds
  • Artifact collection will need updates for parallel jobs
  • Test data cloning may need coordination
  • Overall CI time should improve significantly

Current Workflow Files

  • .github/workflows/test-on-pr.yml (lines 163-204)
  • .github/workflows/unit-tests-coverage.yml (separate, already parallel)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions