-
Notifications
You must be signed in to change notification settings - Fork 599
docs(README): restructure and improve project overview and installation guide #1854
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -16,28 +16,37 @@ English version | [中文版](README_cn.md) | [日本語版](README_ja.md) | |||||
|
|
||||||
| # HAMi | ||||||
|
|
||||||
| **Kubernetes GPU virtualization and heterogeneous accelerator scheduling for AI infrastructure.** | ||||||
| **Kubernetes GPU sharing and heterogeneous accelerator scheduling for AI infrastructure.** | ||||||
|
|
||||||
|  | ||||||
| HAMi (Heterogeneous AI Computing Virtualization Middleware) lets you share GPUs across Kubernetes workloads with memory and compute isolation, without changing application code. Formerly k8s-vGPU-scheduler. | ||||||
|
|
||||||
| [Docs](https://docs.project-hami.io) | [Quick Start](#quick-start) | [Discord](https://discord.gg/hami) | [Issues](https://github.com/project-hami/hami/issues) | [Roadmap](ROADMAP.md) | [Contributing](CONTRIBUTING.md) | ||||||
|
|
||||||
| HAMi stands for **Heterogeneous AI Computing Virtualization Middleware**. Formerly known as `k8s-vGPU-scheduler`, HAMi helps platform teams share expensive GPUs and other AI accelerators across Kubernetes workloads, isolate device memory and compute, and schedule pods with device-aware policies without changing application code. | ||||||
|  | ||||||
|
|
||||||
| HAMi is a [CNCF Sandbox](https://www.cncf.io/sandbox-projects/) and [CNCF Landscape](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami) project. It is also listed in the [CNAI Landscape](https://landscape.cncf.io/?group=cnai&item=cnai--general-orchestration--hami). | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
| ## Why HAMi? | ||||||
| ## The Problem | ||||||
|
|
||||||
| AI infrastructure teams often run into the same Kubernetes accelerator problems: whole GPUs are allocated to small jobs, teams compete for scarce devices, different accelerator vendors expose different operational models, and schedulers lack enough device context to place workloads efficiently. | ||||||
| Running GPU workloads on Kubernetes creates resource contention: | ||||||
|
|
||||||
| HAMi provides a Kubernetes-native layer for: | ||||||
| - **Single workload per GPU**: A 40GB GPU running a 2GB workload uses 5% of capacity | ||||||
| - **No fair allocation**: First-come-first-served; no quota or priority mechanism | ||||||
| - **Vendor fragmentation**: NVIDIA, Ascend, Cambricon each require different integration code | ||||||
| - **Scheduler blindness**: Standard Kubernetes scheduler can't optimize for GPU topology or memory fragmentation | ||||||
|
|
||||||
| ## What HAMi Does | ||||||
|
|
||||||
| HAMi solves this by enabling: | ||||||
|
|
||||||
| - **Device sharing**: allocate a fraction of a physical accelerator by memory, core, or device count. | ||||||
| - **Resource isolation**: enforce per-workload accelerator memory and compute limits where the device backend supports it. | ||||||
| - **Device-aware scheduling**: place pods with topology-aware, binpack, spread, and device-specific scheduling policies. | ||||||
| - **Heterogeneous AI clusters**: manage NVIDIA GPUs, NPUs, DCUs, MLUs, and other accelerator types through one scheduling and allocation workflow. | ||||||
| - **Zero application changes**: keep using standard Kubernetes resource requests and limits. | ||||||
| - **Production operations**: expose metrics, dashboards, WebUI, Helm installation, and community-supported deployment guidance. | ||||||
| - **No code changes**: keep using standard Kubernetes resource requests and limits. | ||||||
| - **Production operations**: metrics, dashboards, WebUI, Helm installation, community support. | ||||||
|
|
||||||
| ## Use Cases | ||||||
|
|
||||||
|
|
@@ -49,37 +58,39 @@ HAMi provides a Kubernetes-native layer for: | |||||
|
|
||||||
| ## How It Works | ||||||
|
|
||||||
| HAMi is composed of a mutating webhook, scheduler extender, device plugins, and device-specific in-container virtualization components. | ||||||
|
|
||||||
| HAMi intercepts pod creation and GPU allocation through these stages: | ||||||
| ```text | ||||||
| Pod submission | ||||||
| -> HAMi mutating webhook | ||||||
| -> HAMi scheduler filter / score / bind | ||||||
| -> device allocation written to pod annotations | ||||||
| -> device plugin Allocate() | ||||||
| -> container runtime environment | ||||||
| -> HAMi monitor and metrics | ||||||
| -> Mutating webhook (inject device request) | ||||||
| -> Scheduler (select GPU and node) | ||||||
| -> Device plugin (allocate GPU memory/cores) | ||||||
| -> Container runtime (inject HAMi driver) | ||||||
| -> HAMi core (enforce isolation) | ||||||
| -> Workload (sees isolated GPU) | ||||||
| ``` | ||||||
|
|
||||||
| ## Device Virtualization | ||||||
|
|
||||||
| HAMi lets workloads request only the accelerator resources they need. For example, the following pod asks for one physical NVIDIA GPU with 3 GiB of GPU memory: | ||||||
|
|
||||||
| Request a fraction of a GPU by memory or compute: | ||||||
| ```yaml | ||||||
| resources: | ||||||
| limits: | ||||||
| nvidia.com/gpu: 1 | ||||||
| nvidia.com/gpumem: 3000 | ||||||
| nvidia.com/gpumem: 3000 # 3GB of 40GB | ||||||
| ``` | ||||||
|
|
||||||
| The workload sees the allocated device resources inside the container, while HAMi coordinates scheduling, allocation, and isolation. | ||||||
| Inside the container, CUDA sees exactly 3GB - not 40GB. HAMi enforces this at the driver level. | ||||||
|
|
||||||
| **Memory isolation**: Hard limit. A pod requesting 3GB cannot exceed 3GB, even if memory is available. | ||||||
|
|
||||||
| **No code changes**: Existing PyTorch, TensorFlow, CUDA code works as-is. | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
| > Notes: | ||||||
| > | ||||||
| > 1. After installing HAMi, the value of `nvidia.com/gpu` registered on the node defaults to the number of vGPUs. | ||||||
| > 2. When requesting resources in a pod, `nvidia.com/gpu` refers to the number of physical GPUs required by the current pod. | ||||||
| Notes: | ||||||
|
|
||||||
| - After installing HAMi, the value of `nvidia.com/gpu` registered on the node defaults to the number of vGPUs. | ||||||
| - When requesting resources in a pod, `nvidia.com/gpu` refers to the number of physical GPUs required by the current pod. | ||||||
|
|
||||||
| ## Supported Devices | ||||||
|
|
||||||
|
|
@@ -104,35 +115,46 @@ For the NVIDIA device plugin path, prepare: | |||||
| ### Install With Helm | ||||||
|
|
||||||
| Label GPU nodes so HAMi can manage them: | ||||||
|
|
||||||
| ```bash | ||||||
| kubectl label nodes <node-name> gpu=on | ||||||
| ``` | ||||||
|
|
||||||
| Add the HAMi Helm repository: | ||||||
|
|
||||||
| ```bash | ||||||
| helm repo add hami-charts https://project-hami.github.io/HAMi/ | ||||||
| helm repo update | ||||||
| ``` | ||||||
|
|
||||||
| Install HAMi: | ||||||
|
|
||||||
| ```bash | ||||||
| helm install hami hami-charts/hami -n kube-system | ||||||
| ``` | ||||||
|
|
||||||
| Verify that the scheduler and device plugin are running: | ||||||
|
|
||||||
| ```bash | ||||||
| kubectl get pods -n kube-system | ||||||
| kubectl get pods -n kube-system | grep hami | ||||||
| # Output should show: | ||||||
| # hami-device-plugin-xxxx Running | ||||||
| # hami-scheduler-xxxx Running | ||||||
| ``` | ||||||
|
|
||||||
| When `hami-device-plugin` and `hami-scheduler` are both `Running`, submit an example workload: | ||||||
| ### Run Your First Shared Workload | ||||||
|
|
||||||
| When both pods are `Running`, submit an example workload: | ||||||
| ```bash | ||||||
| kubectl apply -f examples/nvidia/default_use.yaml | ||||||
| ``` | ||||||
| Verify the workload is running and check GPU allocation: | ||||||
| ```bash | ||||||
| kubectl get pods | ||||||
| kubectl describe pod <pod-name> | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The example pod defined in
Suggested change
|
||||||
| # Output should show: | ||||||
| # nvidia.com/gpu: 1 | ||||||
| # nvidia.com/gpumem: 3000 | ||||||
| ``` | ||||||
|
|
||||||
| Check what the workload sees inside the container: | ||||||
| ```bash | ||||||
| kubectl exec <pod-name> -- nvidia-smi | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||||||
| # Output shows only 3GB of GPU memory (not the full 40GB) | ||||||
| ``` | ||||||
|
|
||||||
| For the complete installation guide and configuration options, see the [HAMi documentation](https://project-hami.io/docs/get-started/deploy-with-helm/). | ||||||
|
|
||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Discord link
https://discord.gg/hamiintroduced here is inconsistent with the links used in the header (line 14) and the community section (line 201), which both point tohttps://discord.gg/Amhy7XmbNq. It is recommended to use a single consistent link throughout the document to avoid confusion.