Skip to content

Fix #596, add a script to run the CI locally #595

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
JAuriac wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

Fix #596, add a script to run the CI locally #595

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
95e1db3
Add a script to run the CI locally
JAuriac Sep 30, 2025
d15a5ed
Fix #596, clean for PR
JAuriac Sep 30, 2025
bcf9ed7
Merge branch 'main' into runLocalCI
JAuriac Apr 27, 2026
c387276
Update to disable building documentation
JAuriac Apr 27, 2026
daf2661
Add another script to run the indent checker locally, and update targ…
JAuriac Apr 27, 2026
b2fe0f1
Modify run-local-CI-indent.sh to enable automatic indent through '-i'…
JAuriac Apr 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions AUTHORS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ spirit, we list here in alphabetical order a condensed list of their contributio
Julian Auriac - CEA (julian.auriac@cea.fr)
* Add mock PDI
* Fix CI bug where tests.xml file could not be written
* Improvement of a Decl'HDF5 test to work on OSX
* Addition of API tests
* Add local CI
* Improve a Decl'HDF5 test to work on OSX
* Add API tests

Julien Bigot - CEA (julien.bigot@cea.fr)
* Initial design and implementation
Expand Down
10 changes: 5 additions & 5 deletions bin/build_and_run_all_tests
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,11 @@ set -xe

# Setup our directories

SRCDIR="${PWD}"
# Set SRCDIR only if not provided
if [[ -z "${SRCDIR:-}" ]]; then
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SRCDIR="$(cd "$SCRIPT_DIR/.." && pwd)"
fi
Comment on lines +38 to +42
Copy link
Copy Markdown
Member

@jbigot jbigot Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?


if [ -z "${TEST_DIR}" ]
then
Expand All @@ -46,10 +50,6 @@ else
TEST_DIR="${PWD}"
fi

cd "${SRCDIR}"
cd "$(dirname "$0")/.."
SRCDIR="${PWD}"

cd "${TEST_DIR}"


Expand Down
24 changes: 24 additions & 0 deletions run-local-CI-indent.sh
View file
Open in desktop
Copy link
Copy Markdown
Member

@jbigot jbigot Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add podman support in addition to docker in bin/test_indent

Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
set -euo pipefail

IMAGE="ghcr.io/pdidev/run_clang_format:v4"

echo "Collecting files from git..."

FILES=$(git ls-files \
| grep -v '^vendor/' \
| grep -E '\.(c|h|cc|hh|cpp|hpp|cxx|hxx)$')

if [ -z "$FILES" ]; then
echo "No files to check."
exit 0
fi

echo "Running clang-format container..."

podman run --rm \
--userns=keep-id \
-v "$PWD":/src \
-w /src \
"$IMAGE" \
"$@" \
$FILES
96 changes: 96 additions & 0 deletions run-local-CI.sh
View file
Open in desktop
Copy link
Copy Markdown
Member

@jbigot jbigot Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it have to be so complex?

In my case a simple:

IMAGE_NAME=ghcr.io/pdidev/spack/latest/clang/openmpi/all:v4
podman/docker run --rm -ti -v ${PWD}:/src:ro ${IMAGE_NAME} /src/bin/build_and_run_all_tests

does the trick. Do we need more?

Copy link
Copy Markdown
Contributor Author

@JAuriac JAuriac Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The key phrase is "in my case", as all pdi contributors maybe shouldn't have to know about containers' directory mounting and the behaviour of the online CI. Two scripts (placed in the bin folder or a hidden folder for example, the second one being the indent script) could enable to test a local change, independently of the local environment, with an additional option to specify the image used.
As seen together, the following should be environment-agnostic :

IMAGE_NAME=ghcr.io/pdidev/spack/latest/clang/openmpi/all:v4
podman run --rm -ti -e CMAKE_FLAGS="-DBUILD_DOCUMENTATION=OFF" -v ${PWD}:/src:ro ${IMAGE_NAME} /src/bin/build_and_run_all_tests

Copy link
Copy Markdown
Member

@jbigot jbigot Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

totally agree that a script makes sense. I just think it can be simpler

Copy link
Copy Markdown
Contributor Author

@JAuriac JAuriac Apr 29, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction, the previous commands are not sufficient on their own, a chmod is needed, as mentioned by Iole :

chmod -R o+rX .
IMAGE_NAME=ghcr.io/pdidev/spack/latest/clang/openmpi/all:v4
podman run --rm -ti -e CMAKE_FLAGS="-DBUILD_DOCUMENTATION=OFF" -v ${PWD}:/src:ro ${IMAGE_NAME} /src/bin/build_and_run_all_tests

Copy link
Copy Markdown
Member

@jbigot jbigot Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally we shouldn't modify the user filesystem
but I don't know any way to make the user data accessible without running as root in the image. do you?

Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
#!/usr/bin/env bash
#=============================================================================
# Copyright (C) 2025-2026 Commissariat a l'energie atomique et aux energies alternatives (CEA)
#
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
# * Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
# * Neither the names of CEA, nor the names of the contributors may be used to
# endorse or promote products derived from this software without specific
# prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#=============================================================================

set -euo pipefail

# ----------- Config ------------------
IMAGE="ghcr.io/pdidev/ubuntu/rolling/openmpi/all:v4"
SCRIPT_NAME="run.sh"
SCRIPT_DIR="./.ci_tmp"
REPORT_FILE="tests.xml"
TMP_DIR="/tmp_dir_test"
BINARY_PATH="./bin/build_and_run_all_tests"
# ------------------------------------

mkdir -p "$SCRIPT_DIR"
rm -f "$SCRIPT_DIR/$SCRIPT_NAME" "$REPORT_FILE"

# Make sure the binary exists and is executable
if [[ ! -f "$BINARY_PATH" ]]; then
echo "Binary not found at $BINARY_PATH"
exit 1
fi

echo "Generating $SCRIPT_NAME in $SCRIPT_DIR..."
cat <<EOF > "$SCRIPT_DIR/$SCRIPT_NAME"
#!/bin/bash
set -xe

JOBID="\$(echo "manual-run" | md5sum | cut -b 1)"
if [[ "01234567" == *"\${JOBID}"* ]]; then
export PDI_PLUGIN_PATH=/tmp/pdi_plugins
fi

export MAKEFLAGS='-j 4'
export CTEST_FLAGS="--output-junit /tmp/tests.xml"
export TEST_DIR="/tmp_dir_test"
export SRCDIR="/src"
export CMAKE_FLAGS="-DBUILD_DOCUMENTATION=OFF"

chmod +x /tmp/build_and_run_all_tests
/tmp/build_and_run_all_tests
EOF

chmod +x "$SCRIPT_DIR/$SCRIPT_NAME"

echo "Creating and preparing container..."
CONTAINER_ID=$(podman create \
--userns=keep-id \
-v "$PWD":/src \
--tmpfs "$TMP_DIR:exec" \
"$IMAGE" \
bash /tmp/run.sh)

echo "Copying script and binary into container..."
podman cp "$SCRIPT_DIR/$SCRIPT_NAME" "$CONTAINER_ID:/tmp/run.sh"
podman cp "$BINARY_PATH" "$CONTAINER_ID:/tmp/build_and_run_all_tests"

echo "Starting container and running script..."
podman start --attach "$CONTAINER_ID"

echo "Attempting to extract test report from container..."
if podman cp "$CONTAINER_ID":/tmp/tests.xml "$REPORT_FILE"; then
echo "Test report saved as $REPORT_FILE"
else
echo "Test report not found in container."
fi

echo "Cleaning up container..."
podman rm "$CONTAINER_ID" >/dev/null 2>&1 || true
Loading