-
Notifications
You must be signed in to change notification settings - Fork 600
docs(README): restructure and improve project overview and installation guide #1854
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
fishman
wants to merge
2
commits into
Project-HAMi:master
Choose a base branch
from
fishman:readme-update
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,28 +16,37 @@ English version | [中文版](README_cn.md) | [日本語版](README_ja.md) | |
|
|
||
| # HAMi | ||
|
|
||
| **Kubernetes GPU virtualization and heterogeneous accelerator scheduling for AI infrastructure.** | ||
| **Kubernetes GPU sharing and heterogeneous accelerator scheduling for AI infrastructure.** | ||
|
|
||
|  | ||
| HAMi (Heterogeneous AI Computing Virtualization Middleware) lets you share GPUs across Kubernetes workloads with memory and compute isolation, without changing application code. Formerly k8s-vGPU-scheduler. | ||
|
|
||
| [Docs](https://docs.project-hami.io) | [Quick Start](#quick-start) | [Discord](https://discord.gg/hami) | [Issues](https://github.com/project-hami/hami/issues) | [[Contributing](CONTRIBUTING.md) | ||
|
|
||
| HAMi stands for **Heterogeneous AI Computing Virtualization Middleware**. Formerly known as `k8s-vGPU-scheduler`, HAMi helps platform teams share expensive GPUs and other AI accelerators across Kubernetes workloads, isolate device memory and compute, and schedule pods with device-aware policies without changing application code. | ||
|  | ||
|
|
||
| HAMi is a [CNCF Sandbox](https://www.cncf.io/sandbox-projects/) and [CNCF Landscape](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami) project. It is also listed in the [CNAI Landscape](https://landscape.cncf.io/?group=cnai&item=cnai--general-orchestration--hami). | ||
|
|
||
|  | ||
|
|
||
| ## Why HAMi? | ||
| ## The Problem | ||
|
|
||
| AI infrastructure teams often run into the same Kubernetes accelerator problems: whole GPUs are allocated to small jobs, teams compete for scarce devices, different accelerator vendors expose different operational models, and schedulers lack enough device context to place workloads efficiently. | ||
| Running GPU workloads on Kubernetes creates resource contention: | ||
|
|
||
| HAMi provides a Kubernetes-native layer for: | ||
| - **Single workload per GPU**: A 40GB GPU running a 2GB workload uses 5% of capacity | ||
| - **No fair allocation**: First-come-first-served; no quota or priority mechanism | ||
| - **Vendor fragmentation**: NVIDIA, Ascend, Cambricon each require different integration code | ||
| - **Scheduler blindness**: Standard Kubernetes scheduler can't optimize for GPU topology or memory fragmentation | ||
|
|
||
| ## What HAMi Does | ||
|
|
||
| HAMi solves this by enabling: | ||
|
|
||
| - **Device sharing**: allocate a fraction of a physical accelerator by memory, core, or device count. | ||
| - **Resource isolation**: enforce per-workload accelerator memory and compute limits where the device backend supports it. | ||
| - **Device-aware scheduling**: place pods with topology-aware, binpack, spread, and device-specific scheduling policies. | ||
| - **Heterogeneous AI clusters**: manage NVIDIA GPUs, NPUs, DCUs, MLUs, and other accelerator types through one scheduling and allocation workflow. | ||
| - **Zero application changes**: keep using standard Kubernetes resource requests and limits. | ||
| - **Production operations**: expose metrics, dashboards, WebUI, Helm installation, and community-supported deployment guidance. | ||
| - **No code changes**: keep using standard Kubernetes resource requests and limits. | ||
| - **Production operations**: metrics, dashboards, WebUI, Helm installation, community support. | ||
|
|
||
| ## Use Cases | ||
|
|
||
|
|
@@ -49,37 +58,39 @@ HAMi provides a Kubernetes-native layer for: | |
|
|
||
| ## How It Works | ||
|
|
||
| HAMi is composed of a mutating webhook, scheduler extender, device plugins, and device-specific in-container virtualization components. | ||
|
|
||
| HAMi intercepts pod creation and GPU allocation through these stages: | ||
| ```text | ||
| Pod submission | ||
| -> HAMi mutating webhook | ||
| -> HAMi scheduler filter / score / bind | ||
| -> device allocation written to pod annotations | ||
| -> device plugin Allocate() | ||
| -> container runtime environment | ||
| -> HAMi monitor and metrics | ||
| -> Mutating webhook (inject device request) | ||
| -> Scheduler (select GPU and node) | ||
| -> Device plugin (allocate GPU memory/cores) | ||
| -> Container runtime (inject HAMi driver) | ||
| -> HAMi core (enforce isolation) | ||
| -> Workload (sees isolated GPU) | ||
| ``` | ||
|
|
||
| ## Device Virtualization | ||
|
|
||
| HAMi lets workloads request only the accelerator resources they need. For example, the following pod asks for one physical NVIDIA GPU with 3 GiB of GPU memory: | ||
|
|
||
| Request a fraction of a GPU by memory or compute: | ||
| ```yaml | ||
| resources: | ||
| limits: | ||
| nvidia.com/gpu: 1 | ||
| nvidia.com/gpumem: 3000 | ||
| nvidia.com/gpumem: 3000 # 3GB of 40GB | ||
| ``` | ||
|
|
||
| The workload sees the allocated device resources inside the container, while HAMi coordinates scheduling, allocation, and isolation. | ||
| Inside the container, CUDA sees exactly 3GB - not 40GB. HAMi enforces this at the driver level. | ||
|
|
||
| **Memory isolation**: Hard limit. A pod requesting 3GB cannot exceed 3GB, even if memory is available. | ||
|
|
||
| **No code changes**: Existing PyTorch, TensorFlow, CUDA code works as-is. | ||
|
|
||
|  | ||
|
|
||
| > Notes: | ||
| > | ||
| > 1. After installing HAMi, the value of `nvidia.com/gpu` registered on the node defaults to the number of vGPUs. | ||
| > 2. When requesting resources in a pod, `nvidia.com/gpu` refers to the number of physical GPUs required by the current pod. | ||
| Notes: | ||
|
|
||
| - After installing HAMi, the value of `nvidia.com/gpu` registered on the node defaults to the number of vGPUs. | ||
| - When requesting resources in a pod, `nvidia.com/gpu` refers to the number of physical GPUs required by the current pod. | ||
|
|
||
| ## Supported Devices | ||
|
|
||
|
|
@@ -104,35 +115,46 @@ For the NVIDIA device plugin path, prepare: | |
| ### Install With Helm | ||
|
|
||
| Label GPU nodes so HAMi can manage them: | ||
|
|
||
| ```bash | ||
| kubectl label nodes <node-name> gpu=on | ||
| ``` | ||
|
|
||
| Add the HAMi Helm repository: | ||
|
|
||
| ```bash | ||
| helm repo add hami-charts https://project-hami.github.io/HAMi/ | ||
| helm repo update | ||
| ``` | ||
|
|
||
| Install HAMi: | ||
|
|
||
| ```bash | ||
| helm install hami hami-charts/hami -n kube-system | ||
| ``` | ||
|
|
||
| Verify that the scheduler and device plugin are running: | ||
|
|
||
| ```bash | ||
| kubectl get pods -n kube-system | ||
| kubectl get pods -n kube-system | grep hami | ||
| # Output should show: | ||
| # hami-device-plugin-xxxx Running | ||
| # hami-scheduler-xxxx Running | ||
| ``` | ||
|
|
||
| When `hami-device-plugin` and `hami-scheduler` are both `Running`, submit an example workload: | ||
| ### Run Your First Shared Workload | ||
|
|
||
| When both pods are `Running`, submit an example workload: | ||
| ```bash | ||
| kubectl apply -f examples/nvidia/default_use.yaml | ||
| ``` | ||
| Verify the workload is running and check GPU allocation: | ||
| ```bash | ||
| kubectl get pods | ||
| kubectl describe pod <pod-name> | ||
| # Output should show: | ||
| # nvidia.com/gpu: 1 | ||
| # nvidia.com/gpumem: 3000 | ||
| ``` | ||
|
|
||
| Check what the workload sees inside the container: | ||
| ```bash | ||
| kubectl exec <pod-name> -- nvidia-smi | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| # Output shows only 3GB of GPU memory (not the full 40GB) | ||
| ``` | ||
|
|
||
| For the complete installation guide and configuration options, see the [HAMi documentation](https://project-hami.io/docs/get-started/deploy-with-helm/). | ||
|
|
||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The example pod defined in
examples/nvidia/default_use.yamlis explicitly namedgpu-pod. Using this specific name instead of the<pod-name>placeholder makes the quick start guide more user-friendly and directly executable for users following the steps.