Skip to content

Provide option to use EESSI modules instead of init script #313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
casparvl wants to merge 14 commits into
base: main
Choose a base branch
from
Open

Provide option to use EESSI modules instead of init script #313

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
d46efdd
Make run_reframe.sh loop over versions defined in EESSI_VERSIONS (int…
Jan 19, 2026
35a8b06
Add 2025.06 to default list of EESSI_VERSION
Jan 19, 2026
e937e4e
Improve behaviour in case EESSI module is NOT used, or in case local …
Jan 19, 2026
ed54378
Apply suggestion from @casparvl
casparvl Apr 9, 2026
7270887
Apply suggestion from @casparvl
casparvl Apr 9, 2026
47e28b3
Remove loop. Still WIP. But need to figure out if we should specify t…
Apr 30, 2026
7813dcc
Merge remote-tracking branch 'refs/remotes/origin/make_run_reframe_lo…
Apr 30, 2026
d13df26
Make sure we support EESSI and local programming environments, and pa…
Apr 30, 2026
89b1ace
Remove stray 'REFRAME_URL='
Jun 17, 2026
c1e3ded
No longer needed now that we've internalized the hpctestlib into EESS…
Jun 17, 2026
8fea8eb
Merge branch 'support_multiple_eessi_environments' into make_run_refr…
Jun 17, 2026
2bd4564
Remove more things related to cloning the reframe repo for the hpctes…
Jun 17, 2026
281d1ac
Merge branch 'main' into make_run_reframe_loop_eessi_versions
Jun 22, 2026
7b25663
Corrected path, it was missing 'versions' in the prefix
Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 54 additions & 65 deletions CI/run_reframe.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,21 +37,9 @@ fi
source "${CI_CONFIG}"

# Set default configuration, but let anything set by CI_CONFIG take priority
if [ -z "${REFRAME_ARGS}" ]; then
REFRAME_ARGS="--tag CI --tag 1_node"
fi
if [ -z "${REFRAME_VERSION}"]; then
REFRAME_VERSION=4.8.1
fi
if [ -z "${REFRAME_URL}" ]; then
REFRAME_URL='https://github.com/reframe-hpc/reframe.git'
fi
if [ -z "${REFRAME_BRANCH}" ]; then
REFRAME_BRANCH="v${REFRAME_VERSION}"
fi
if [ -z "${EESSI_TESTSUITE_URL}" ]; then
EESSI_TESTSUITE_URL='https://github.com/EESSI/test-suite.git'
fi
REFRAME_ARGS="${REFRAME_ARGS:---tag CI --tag 1_node}"
REFRAME_VERSION="${REFRAME_VERSION:-4.8.1}"
EESSI_TESTSUITE_URL="${EESSI_TESTSUITE_URL:-https://github.com/EESSI/test-suite.git}"
if [ -z "${EESSI_TESTSUITE_BRANCH}" ]; then
git clone -n --depth=1 --filter=tree:0 ${EESSI_TESTSUITE_URL} "${TEMPDIR}/test-suite-version-checkout"
cd "${TEMPDIR}/test-suite-version-checkout"
Expand All @@ -62,57 +50,31 @@ if [ -z "${EESSI_TESTSUITE_BRANCH}" ]; then
EESSI_TESTSUITE_BRANCH="${LATEST_VERSION}"
cd ${TEMPDIR}
fi
if [ -z "${EESSI_CONFIGS_TESTSUITE_URL}" ]; then
EESSI_CONFIGS_TESTSUITE_URL="${EESSI_TESTSUITE_URL}"
fi
if [ -z "${EESSI_CONFIGS_TESTSUITE_BRANCH}" ]; then
EESSI_CONFIGS_TESTSUITE_BRANCH="${EESSI_TESTSUITE_BRANCH}"
fi
if [ -z "${USE_EESSI_SOFTWARE_STACK}" ] || [ "$USE_EESSI_SOFTWARE_STACK" == "True" ]; then
export USE_EESSI_SOFTWARE_STACK=True
if [ -z "${EESSI_CVMFS_REPO}" ]; then
export EESSI_CVMFS_REPO=/cvmfs/software.eessi.io
fi
if [ -z "${EESSI_VERSION}" ]; then
export EESSI_VERSION=2023.06
fi
fi
if [ -z "${RFM_CONFIG_FILES}" ]; then
export RFM_CONFIG_FILES="${TEMPDIR}/configs/config/${EESSI_CI_SYSTEM_NAME}.py"
fi
if [ -z "${RFM_CHECK_SEARCH_PATH}" ]; then
export RFM_CHECK_SEARCH_PATH="${TEMPDIR}/test-suite/eessi/testsuite/tests/"
fi
if [ -z "${RFM_CHECK_SEARCH_RECURSIVE}" ]; then
export RFM_CHECK_SEARCH_RECURSIVE=1
fi
if [ -z "${RFM_PREFIX}" ]; then
export RFM_PREFIX="${HOME}/reframe_CI_runs"
fi
if [ -z "${REFRAME_TIMEOUT}" ]; then
# 10 minutes short of 1 day, since typically the test suite will be run daily.
# This will prevent multiple ReFrame runs from piling up and exceeding the quota on our Magic Castle clusters
export REFRAME_TIMEOUT=1430m
fi
if [ -z "${UNSET_MODULEPATH}" ]; then
export UNSET_MODULEPATH=True
fi
if [ -z "${SET_LOCAL_MODULE_ENV}" ]; then
export SET_LOCAL_MODULE_ENV=False
fi
EESSI_CONFIGS_TESTSUITE_URL="${EESSI_CONFIGS_TESTSUITE_URL:-${EESSI_TESTSUITE_URL}}"
EESSI_CONFIGS_TESTSUITE_BRANCH="${EESSI_CONFIGS_TESTSUITE_BRANCH:-${EESSI_TESTSUITE_BRANCH}}"
export USE_MODULECMD_FROM_EESSI_VERSION="${USE_MODULECMD_FROM_EESSI_VERSION:-2025.06}"
export USE_EESSI_SOFTWARE_STACK="${USE_EESSI_SOFTWARE_STACK:-True}"
if [ "$USE_EESSI_SOFTWARE_STACK" == "True" ]; then
export EESSI_CVMFS_REPO="${EESSI_CVMFS_REPO:-/cvmfs/software.eessi.io}"
export REFRAME_EESSI_PROGRAMMING_ENVS="${REFRAME_EESSI_PROGRAMMING_ENVS:-EESSI-2023.06,EESSI-2025.06}"
fi
export RFM_CONFIG_FILES="${RFM_CONFIG_FILES:-${TEMPDIR}/configs/config/${EESSI_CI_SYSTEM_NAME}.py}"
export RFM_CHECK_SEARCH_PATH="${RFM_CHECK_SEARCH_PATH:-${TEMPDIR}/test-suite/eessi/testsuite/tests/}"
export RFM_CHECK_SEARCH_RECURSIVE="${RFM_CHECK_SEARCH_RECURSIVE:-1}"
export RFM_PREFIX="${RFM_PREFIX:-${HOME}/reframe_CI_runs}"
# 10 minutes short of 1 day, since typically the test suite will be run daily.
# This will prevent multiple ReFrame runs from piling up and exceeding the quota on our Magic Castle clusters
export REFRAME_TIMEOUT="${REFRAME_TIMEOUT:-1430m}"
export UNSET_MODULEPATH="${UNSET_MODULEPATH:-True}"
export REFRAME_LOCAL_PROGRAMMING_ENVS="${REFRAME_LOCAL_PROGRAMMING_ENVS:-}" # Assumed to be a comma seperated list
export SET_LOCAL_MODULE_ENV="${SET_LOCAL_MODULE_ENV:-False}"

# Create virtualenv for ReFrame using system python
python3 -m venv "${TEMPDIR}"/reframe_venv
source "${TEMPDIR}"/reframe_venv/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install reframe-hpc=="${REFRAME_VERSION}"

# Clone reframe repo to have the hpctestlib:
REFRAME_CLONE_ARGS="${REFRAME_URL} --branch ${REFRAME_BRANCH} --depth 1 ${TEMPDIR}/reframe"
echo "Cloning ReFrame repo: git clone ${REFRAME_CLONE_ARGS}"
git clone ${REFRAME_CLONE_ARGS}
export PYTHONPATH="${PYTHONPATH}":"${TEMPDIR}"/reframe

# Clone configs from test suite repo
EESSI_CONFIGS_CLONE_ARGS="-n --filter=tree:0 ${EESSI_CONFIGS_TESTSUITE_URL} --branch ${EESSI_CONFIGS_TESTSUITE_BRANCH} --depth 1 ${TEMPDIR}/configs"
echo "Cloning configs from EESSI test suite repo:"
Expand Down Expand Up @@ -149,10 +111,28 @@ if [ "$SET_LOCAL_MODULE_ENV" == "True" ]; then
module load ${LOCAL_MODULES}
fi

# Start the EESSI environment
if [ "$USE_EESSI_SOFTWARE_STACK" == "True" ]; then
eessi_init_path="${EESSI_CVMFS_REPO}"/versions/"${EESSI_VERSION}"/init/bash
source "${eessi_init_path}"
# With https://github.com/EESSI/test-suite/pull/326 we no longer need to load the EESSI module
# before running the reframe command (reframe will load the modules for us)
# However, on systems in which there is no module command, we do need to initialize an
# EESSI environment to get a module command
if ! command -v module &>/dev/null; then
# No module command available
if [ ! -z "$USE_MODULECMD_FROM_EESSI_VERSION" ]; then
echo "Using module command from EESSI version ${USE_MODULECMD_FROM_EESSI_VERSION}"
source "${EESSI_CVMFS_REPO}/versions/${USE_MODULECMD_FROM_EESSI_VERSION}/init/lmod/bash"
module unload EESSI
else
msg="No module command available, and this CI run was not configured to use the EESSI module command."
msg="$msg Consider setting 'USE_MODULECMD_FROM_EESSI_VERSION=<eessi_version>' in your CI runs to use a module"
msg="$msg command from the EESSI compatibility layer. Exiting..."
echo $msg
exit 1
fi
else
# Module command available. Make sure the EESSI modules are on the PATH if we are testing EESSI environments
if [ "$USE_EESSI_SOFTWARE_STACK" == "True" ]; then
module use "${EESSI_CVMFS_REPO}/init/modules/"
fi
Comment thread
casparvl marked this conversation as resolved.
fi

# Needed in order to make sure the reframe from our TEMPDIR is first on the PATH,
Expand All @@ -161,6 +141,16 @@ fi
deactivate
source "${TEMPDIR}"/reframe_venv/bin/activate

# Modify the REFRAME_ARGS to take the requested programming environments into account
REFRAME_PROGRAMMING_ENVS="${REFRAME_LOCAL_PROGRAMMING_ENVS},${REFRAME_EESSI_PROGRAMMING_ENVS}"
REFRAME_PROGRAMMING_ENVS="${REFRAME_PROGRAMMING_ENVS%,}" # Remove any leading comma in case 1st list is empty
REFRAME_PROGRAMMING_ENVS="${REFRAME_PROGRAMMING_ENVS#,}" # Remove any trailing comma in case 2nd list is empty
# Replace commas by | since ReFrame expects a regex and we want tests to run if they match any of the programming envs
REFRAME_PROGRAMMING_ENVS_PIPED="${REFRAME_PROGRAMMING_ENVS//,/|}"
if [ -z "$REFRAME_PROGRAMMING_ENVS_PIPED" ]; then

@laraPPr laraPPr Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wrong it is now setting the -p flag when $REFRAME_PROGRAMMING_ENVS_PIPED is empthy

@laraPPr laraPPr Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made this change

Suggested change
if [ -z "$REFRAME_PROGRAMMING_ENVS_PIPED" ]; then
if [ -n "$REFRAME_PROGRAMMING_ENVS_PIPED" ]; then

REFRAME_ARGS="${REFRAME_ARGS} -p '$REFRAME_PROGRAMMING_ENVS_PIPED'"
fi
Comment on lines +144 to +152

@laraPPr laraPPr Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah it this stuff that seems to cause issues

@casparvl casparvl Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, weird. It's supposed to define the ReFrame programming environments (like EESSI-2023.06, EESSI-2025.06) and take into account that you may want to run the test also on a local programming environment (say local-2025 or whatever name you use in the reframe config file for your local programming env). But for you, they all seem to be empty? Not sure why that would happen. If I recall, REFRAME_LOCAL_PROGRAMMING_ENVS may be empty, but we account for that. And REFRAME_EESSSI_PROGRAMMING_ENVS should default to EESSI-2023.06,EESSI-2025.06. So unless you'd explicitely set that env var (which I'm assuming you don't), I don't see why those would not be in there...

Can you add a few echo statements to see the values of REFRAME_LOCAL_PROGRAMMING_ENVS and REFRAME_EESSI_PROGRAMMING_ENVS, and maybe REFRAME_PROGRAMMING_ENVS as it goes through that code section?


# Print ReFrame config
echo "Starting CI run with the follwing settings:"
echo ""
Expand All @@ -170,14 +160,13 @@ echo "EESSI test suite URL: ${EESSI_TESTSUITE_URL}"
echo "EESSI test suite version: ${EESSI_TESTSUITE_BRANCH}"
echo "EESSI test suite URL for configs: ${EESSI_CONFIGS_TESTSUITE_URL}"
echo "EESSI test suite version for configs: ${EESSI_CONFIGS_TESTSUITE_BRANCH}"
echo "HPCtestlib from ReFrame URL: ${REFRAME_URL}"
echo "HPCtestlib from ReFrame branch: ${REFRAME_BRANCH}"
echo "ReFrame executable: $(which reframe)"
echo "ReFrame version: $(reframe --version)"
echo "ReFrame config file: ${RFM_CONFIG_FILES}"
echo "ReFrame check search path: ${RFM_CHECK_SEARCH_PATH}"
echo "ReFrame check search recursive: ${RFM_CHECK_SEARCH_RECURSIVE}"
echo "ReFrame prefix: ${RFM_PREFIX}"
echo "Testing ReFrame programming environments: ${REFRAME_PROGRAMMING_ENVS}"
echo "ReFrame args: ${REFRAME_ARGS}"
echo "Using EESSI: ${USE_EESSI_SOFTWARE_STACK}"
echo "Using local software stack ${SET_LOCAL_MODULE_ENV}"
Expand Down
2 changes: 1 addition & 1 deletion config/aws_mc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@
] + list(SCALES.keys()),
'prepare_cmds': [
# This system doesn't have an lmod installation by default, so source one from EESSI
'source /cvmfs/software.eessi.io/2025.06/init/lmod/bash',
'source /cvmfs/software.eessi.io/versions/2025.06/init/lmod/bash',
'module unload EESSI',
# Required when using srun as launcher with --export=NONE in partition access, in order to ensure job
# steps inherit environment. It doesn't hurt to define this even if srun is not used
Expand Down
Loading