feat: new mutex GPU scheduling policy

[mesutoezdil]

Add GPUSchedulerPolicyMutex = "mutex" constant. When a pod sets hami.io/gpu-scheduler-policy: mutex, the scheduler allocates only GPUs with no existing users.

Changes

pkg/util/types.go

• Add GPUSchedulerPolicyMutex constant

pkg/scheduler/policy/gpu_policy.go

• Less(): mutex case sorts idle GPUs to tail so Fit() picks them first

pkg/device/*/device.go (13 backends)

• Fit(): skip devices with Used > 0 when policy is mutex

pkg/scheduler/policy/gpu_policy_test.go

• Add 3 mutex test cases

Usage

annotations:
  hami.io/gpu-scheduler-policy: mutex

Pod is only scheduled to a GPU with no existing users.

Closes #2009
Part of #1889

Test plan

• go test ./pkg/scheduler/policy/... --race -count=1
• go test ./pkg/device/... --race -count=1

[gemini-code-assist]

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f4adb6d6-c121-462f-b004-2de6fc725188

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[hami-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mesutoezdil
Once this PR has been reviewed and has the lgtm label, please assign dsfans2014 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

❌ Patch coverage is 35.00000% with 52 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/device/amd/device.go	20.00%	3 Missing and 1 partial ⚠️
pkg/device/ascend/device.go	20.00%	3 Missing and 1 partial ⚠️
pkg/device/awsneuron/device.go	20.00%	3 Missing and 1 partial ⚠️
pkg/device/biren/device.go	20.00%	3 Missing and 1 partial ⚠️
pkg/device/cambricon/device.go	20.00%	3 Missing and 1 partial ⚠️
pkg/device/enflame/device.go	20.00%	3 Missing and 1 partial ⚠️
pkg/device/hygon/device.go	20.00%	3 Missing and 1 partial ⚠️
pkg/device/iluvatar/device.go	20.00%	3 Missing and 1 partial ⚠️
pkg/device/metax/device.go	20.00%	3 Missing and 1 partial ⚠️
pkg/device/metax/sdevice.go	20.00%	3 Missing and 1 partial ⚠️
... and 3 more

Flag	Coverage Δ
unittests	58.89% <35.00%> (-0.20%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
pkg/scheduler/policy/gpu_policy.go	100.00% <100.00%> (ø)
pkg/util/types.go	100.00% <ø> (ø)
pkg/device/amd/device.go	73.18% <20.00%> (-2.00%)	⬇️
pkg/device/ascend/device.go	82.48% <20.00%> (-0.73%)	⬇️
pkg/device/awsneuron/device.go	74.38% <20.00%> (-0.98%)	⬇️
pkg/device/biren/device.go	91.97% <20.00%> (-2.73%)	⬇️
pkg/device/cambricon/device.go	81.74% <20.00%> (-1.25%)	⬇️
pkg/device/enflame/device.go	54.42% <20.00%> (-0.38%)	⬇️
pkg/device/hygon/device.go	91.40% <20.00%> (-1.66%)	⬇️
pkg/device/iluvatar/device.go	56.16% <20.00%> (-0.85%)	⬇️
... and 5 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.