Skip to content

ci: integration test times out when another one is busy #201

@Jakob-Naucke

Description

@Jakob-Naucke

Seen on #199. The integration tests time out after 2h, but are forced to run in sequence, so it'll time out because most of the time was spent waiting for another one.

==========================================
Step 6/8: Acquiring exclusive lock on Beaker machine
==========================================
[INFO] Lock file: /tmp/tec-operator-ci.lock
[INFO] Lock holder ID: ci-op-8s545qdp-2023670978252378112-1771315870
[INFO] Lock timeout: 21600 seconds (6 hours)
[INFO] This ensures only one CI job runs on the Beaker machine at a time
[INFO] Transferring lock holder script to Beaker machine...
[INFO] Starting lock holder process on Beaker machine...
[INFO] Acquiring lock (timeout: 21600s / 6 hours)...
[INFO] Waiting for lock holder process to acquire lock or timeout...
[INFO] This may take up to 21600 seconds if another job is running...
[INFO] Still waiting for lock... (currently held by: ci-op-vxv5z98h-2023671078945034240-1771315870)
… [same line, many times] …
[INFO] Still waiting for lock... (currently held by: ci-op-vxv5z98h-2023671078945034240-1771315870)
{"component":"entrypoint","file":"sigs.k8s.io/prow/pkg/entrypoint/run.go:169","func":"sigs.k8s.io/prow/pkg/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 2h0m0s timeout","severity":"error","time":"2026-02-17T10:11:04Z"}
{"component":"entrypoint","file":"sigs.k8s.io/prow/pkg/entrypoint/run.go:264","func":"sigs.k8s.io/prow/pkg/entrypoint.gracefullyTerminate","level":"error","msg":"Process gracefully exited before 5m0s grace period","severity":"error","time":"2026-02-17T10:11:06Z"}
{"component":"entrypoint","error":"process timed out","file":"sigs.k8s.io/prow/pkg/entrypoint/run.go:84","func":"sigs.k8s.io/prow/pkg/entrypoint.Options.internalRun","level":"error","msg":"Error executing test process","severity":"error","time":"2026-02-17T10:11:06Z"}
error: failed to execute wrapped command: exit status 127 
�[36mINFO�[0m[2026-02-17T10:11:07Z] Step operator-lifecycle-verify-trusted-execution-clusters-ref-operator-beaker-kind-provision failed after 2h0m59s. 

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions