From faab7cb1bc0011d5251a3ec69e6a910310beb554 Mon Sep 17 00:00:00 2001 From: Reza Jelveh Date: Tue, 12 May 2026 01:00:32 +0800 Subject: [PATCH 1/2] docs(README): restructure and improve project overview and installation guide - Update project tagline to emphasize GPU sharing over virtualization - Restructure sections, add "problem"/"solution" - Clarify the architecture flow diagram with labeled stages - Tighten wording throughout (e.g. "zero application changes" -> "no code changes") --- README.md | 88 ++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 55 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index f6e377504..c4d1af34e 100644 --- a/README.md +++ b/README.md @@ -16,28 +16,37 @@ English version | [中文版](README_cn.md) | [日本語版](README_ja.md) # HAMi -**Kubernetes GPU virtualization and heterogeneous accelerator scheduling for AI infrastructure.** +**Kubernetes GPU sharing and heterogeneous accelerator scheduling for AI infrastructure.** -![HAMi Architecture](imgs/hami-architecture.png) +HAMi (Heterogeneous AI Computing Virtualization Middleware) lets you share GPUs across Kubernetes workloads with memory and compute isolation, without changing application code. Formerly k8s-vGPU-scheduler. + +[Docs](https://docs.project-hami.io) | [Quick Start](#quick-start) | [Discord](https://discord.gg/hami) | [Issues](https://github.com/project-hami/hami/issues) | [Roadmap](ROADMAP.md) | [Contributing](CONTRIBUTING.md) -HAMi stands for **Heterogeneous AI Computing Virtualization Middleware**. Formerly known as `k8s-vGPU-scheduler`, HAMi helps platform teams share expensive GPUs and other AI accelerators across Kubernetes workloads, isolate device memory and compute, and schedule pods with device-aware policies without changing application code. +![HAMi Architecture](imgs/hami-architecture.png) HAMi is a [CNCF Sandbox](https://www.cncf.io/sandbox-projects/) and [CNCF Landscape](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami) project. It is also listed in the [CNAI Landscape](https://landscape.cncf.io/?group=cnai&item=cnai--general-orchestration--hami). ![CNCF logo](imgs/cncf-logo.png) -## Why HAMi? +## The Problem -AI infrastructure teams often run into the same Kubernetes accelerator problems: whole GPUs are allocated to small jobs, teams compete for scarce devices, different accelerator vendors expose different operational models, and schedulers lack enough device context to place workloads efficiently. +Running GPU workloads on Kubernetes creates resource contention: -HAMi provides a Kubernetes-native layer for: +- **Single workload per GPU**: A 40GB GPU running a 2GB workload uses 5% of capacity +- **No fair allocation**: First-come-first-served; no quota or priority mechanism +- **Vendor fragmentation**: NVIDIA, Ascend, Cambricon each require different integration code +- **Scheduler blindness**: Standard Kubernetes scheduler can't optimize for GPU topology or memory fragmentation + +## What HAMi Does + +HAMi solves this by enabling: - **Device sharing**: allocate a fraction of a physical accelerator by memory, core, or device count. - **Resource isolation**: enforce per-workload accelerator memory and compute limits where the device backend supports it. - **Device-aware scheduling**: place pods with topology-aware, binpack, spread, and device-specific scheduling policies. - **Heterogeneous AI clusters**: manage NVIDIA GPUs, NPUs, DCUs, MLUs, and other accelerator types through one scheduling and allocation workflow. -- **Zero application changes**: keep using standard Kubernetes resource requests and limits. -- **Production operations**: expose metrics, dashboards, WebUI, Helm installation, and community-supported deployment guidance. +- **No code changes**: keep using standard Kubernetes resource requests and limits. +- **Production operations**: metrics, dashboards, WebUI, Helm installation, community support. ## Use Cases @@ -49,37 +58,39 @@ HAMi provides a Kubernetes-native layer for: ## How It Works -HAMi is composed of a mutating webhook, scheduler extender, device plugins, and device-specific in-container virtualization components. - +HAMi intercepts pod creation and GPU allocation through these stages: ```text Pod submission - -> HAMi mutating webhook - -> HAMi scheduler filter / score / bind - -> device allocation written to pod annotations - -> device plugin Allocate() - -> container runtime environment - -> HAMi monitor and metrics + -> Mutating webhook (inject device request) + -> Scheduler (select GPU and node) + -> Device plugin (allocate GPU memory/cores) + -> Container runtime (inject HAMi driver) + -> HAMi core (enforce isolation) + -> Workload (sees isolated GPU) ``` ## Device Virtualization -HAMi lets workloads request only the accelerator resources they need. For example, the following pod asks for one physical NVIDIA GPU with 3 GiB of GPU memory: - +Request a fraction of a GPU by memory or compute: ```yaml resources: limits: nvidia.com/gpu: 1 - nvidia.com/gpumem: 3000 + nvidia.com/gpumem: 3000 # 3GB of 40GB ``` -The workload sees the allocated device resources inside the container, while HAMi coordinates scheduling, allocation, and isolation. +Inside the container, CUDA sees exactly 3GB - not 40GB. HAMi enforces this at the driver level. + +**Memory isolation**: Hard limit. A pod requesting 3GB cannot exceed 3GB, even if memory is available. + +**No code changes**: Existing PyTorch, TensorFlow, CUDA code works as-is. ![HAMi Example](imgs/example.png) -> Notes: -> -> 1. After installing HAMi, the value of `nvidia.com/gpu` registered on the node defaults to the number of vGPUs. -> 2. When requesting resources in a pod, `nvidia.com/gpu` refers to the number of physical GPUs required by the current pod. +Notes: + +- After installing HAMi, the value of `nvidia.com/gpu` registered on the node defaults to the number of vGPUs. +- When requesting resources in a pod, `nvidia.com/gpu` refers to the number of physical GPUs required by the current pod. ## Supported Devices @@ -104,35 +115,46 @@ For the NVIDIA device plugin path, prepare: ### Install With Helm Label GPU nodes so HAMi can manage them: - ```bash kubectl label nodes gpu=on ``` - Add the HAMi Helm repository: - ```bash helm repo add hami-charts https://project-hami.github.io/HAMi/ helm repo update ``` - Install HAMi: - ```bash helm install hami hami-charts/hami -n kube-system ``` - Verify that the scheduler and device plugin are running: - ```bash -kubectl get pods -n kube-system +kubectl get pods -n kube-system | grep hami +# Output should show: +# hami-device-plugin-xxxx Running +# hami-scheduler-xxxx Running ``` -When `hami-device-plugin` and `hami-scheduler` are both `Running`, submit an example workload: +### Run Your First Shared Workload +When both pods are `Running`, submit an example workload: ```bash kubectl apply -f examples/nvidia/default_use.yaml ``` +Verify the workload is running and check GPU allocation: +```bash +kubectl get pods +kubectl describe pod +# Output should show: +# nvidia.com/gpu: 1 +# nvidia.com/gpumem: 3000 +``` + +Check what the workload sees inside the container: +```bash +kubectl exec -- nvidia-smi +# Output shows only 3GB of GPU memory (not the full 40GB) +``` For the complete installation guide and configuration options, see the [HAMi documentation](https://project-hami.io/docs/get-started/deploy-with-helm/). From f19959a0c4f0b371ba1a56c26b5d54edb9e476eb Mon Sep 17 00:00:00 2001 From: Reza Jelveh Date: Tue, 12 May 2026 01:19:16 +0800 Subject: [PATCH 2/2] docs(README): remove ROADMAP for now --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index c4d1af34e..30ab0ae00 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ English version | [中文版](README_cn.md) | [日本語版](README_ja.md) HAMi (Heterogeneous AI Computing Virtualization Middleware) lets you share GPUs across Kubernetes workloads with memory and compute isolation, without changing application code. Formerly k8s-vGPU-scheduler. -[Docs](https://docs.project-hami.io) | [Quick Start](#quick-start) | [Discord](https://discord.gg/hami) | [Issues](https://github.com/project-hami/hami/issues) | [Roadmap](ROADMAP.md) | [Contributing](CONTRIBUTING.md) +[Docs](https://docs.project-hami.io) | [Quick Start](#quick-start) | [Discord](https://discord.gg/hami) | [Issues](https://github.com/project-hami/hami/issues) | [[Contributing](CONTRIBUTING.md) ![HAMi Architecture](imgs/hami-architecture.png)