Skip to content

Add testing base for package installation cases#4960

Open
LecrisUT wants to merge 8 commits into
mainfrom
fix/4910
Open

Add testing base for package installation cases#4960
LecrisUT wants to merge 8 commits into
mainfrom
fix/4910

Conversation

@LecrisUT
Copy link
Copy Markdown
Member

@LecrisUT LecrisUT commented Jun 5, 2026

Related to #4910, but does not cover all cases. For now the cases covered are:

(previously #4924, but that was closed because it was opened from a fork)

LecrisUT added 8 commits June 5, 2026 10:07
Signed-off-by: Cristian Le <git@lecris.dev>
Signed-off-by: Cristian Le <git@lecris.dev>
Signed-off-by: Cristian Le <git@lecris.dev>
Signed-off-by: Cristian Le <git@lecris.dev>
Signed-off-by: Cristian Le <git@lecris.dev>
Signed-off-by: Cristian Le <git@lecris.dev>
Signed-off-by: Cristian Le <git@lecris.dev>
Signed-off-by: Cristian Le <git@lecris.dev>
@LecrisUT LecrisUT added this to planning Jun 5, 2026
@github-project-automation github-project-automation Bot moved this to backlog in planning Jun 5, 2026
@LecrisUT LecrisUT moved this from backlog to review in planning Jun 5, 2026
@LecrisUT LecrisUT added test coverage Improvements or additions to test coverage of tmt itself ci | full test Pull request is ready for the full test execution plugin | artifact Related to the `prepare/artifact` plugin. labels Jun 5, 2026
@LecrisUT LecrisUT requested a review from vaibhavdaren June 5, 2026 08:11
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds integration test cases (install-cases) and mock RPM spec files to test various package installation scenarios. Key feedback includes correcting the FMF context interpolation syntax by removing the leading $ from @{verified_artifact}, escaping $ in mktemp within rlRun, removing a redundant run directory creation, double-quoting the ${xfail_plans[@]} array expansion, and creating unique run directories inside the loop to prevent log overwriting.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

order: 40
how: artifact
provide:
- file:rpms/$@{verified_artifact}/*.rpm
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Remove the leading $ from the fmf context interpolation syntax. The correct syntax is @{verified_artifact}.

      - file:rpms/@{verified_artifact}/*.rpm


rlJournalStart
rlPhaseStartSetup "Prepare test environment"
rlRun "testdir=$(mktemp -d)" 0 "Create test directory"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Escape the $ in the mktemp command so that it is executed inside rlRun. This allows rlRun to properly check the exit status of mktemp and report a failure if it fails.

Suggested change
rlRun "testdir=$(mktemp -d)" 0 "Create test directory"
rlRun "testdir=\$(mktemp -d)" 0 "Create test directory"

rlRun "cp -a install-cases $testdir/data" 0 "Copy test data"
rlRun "cp -a rpms $testdir/data/" 0 "Copy rpms data"
rlRun "pushd $testdir/data" 0 "Enter test directory"
rlRun "run=$(mktemp -d)" 0 "Create run directory"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Remove this line as the run directory is now created uniquely per test case inside the loop.

for plan in $(tmt plans ls); do
xfail=""
expected_result=0
for check_pattern in ${xfail_plans[@]}; do
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Double quote the array expansion ${xfail_plans[@]} to prevent word splitting and globbing.

Suggested change
for check_pattern in ${xfail_plans[@]}; do
for check_pattern in "${xfail_plans[@]}"; do

Comment on lines +60 to +64
rlPhaseStartTest "$phase_prefix $plan $xfail"
rlRun "tmt run $extra_env -i $run --scratch -vvv --all \
plan --name $plan \
provision -h $PROVISION_HOW --image $image" \
$expected_result "Run test case $plan $xfail"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Create a unique run directory for each test case inside the loop to preserve logs for debugging. Reusing a single $run directory with --scratch sequentially overwrites the logs of previous runs, making it very difficult to debug failures.

Suggested change
rlPhaseStartTest "$phase_prefix $plan $xfail"
rlRun "tmt run $extra_env -i $run --scratch -vvv --all \
plan --name $plan \
provision -h $PROVISION_HOW --image $image" \
$expected_result "Run test case $plan $xfail"
rlRun "run_dir=\$(mktemp -d -p \"\$testdir\")" 0 "Create run directory"
rlPhaseStartTest "$phase_prefix $plan $xfail"
rlRun "tmt run $extra_env -i \$run_dir --scratch -vvv --all \
plan --name $plan \
provision -h $PROVISION_HOW --image $image" \
$expected_result "Run test case $plan $xfail"

SYSTEM_PACKAGE: foo-1.4
EXPECTED_PACKAGE: ""
PRE_INSTALLED_PACKAGE: ""
DNF_CMD: dnf
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume we will be also later testing wiht yum? Would it not be better to prepare for that here now?

Comment on lines +47 to +50
context:
want_devel: "false"
with_bar: "false"
xfail: "false"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to lookup what want_devel means, it would be good to drop here some comments so it is explained what these context variables are intended for, would definitely help me at lesat for the first one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci | full test Pull request is ready for the full test execution plugin | artifact Related to the `prepare/artifact` plugin. test coverage Improvements or additions to test coverage of tmt itself

Projects

Status: review

Development

Successfully merging this pull request may close these issues.

2 participants