Skip to content

docs: update benchmark documentation to reflect current HAMi usage#1750

Open
Yonsun-w wants to merge 1 commit into
Project-HAMi:masterfrom
Yonsun-w:docs/update-benchmark-1722
Open

docs: update benchmark documentation to reflect current HAMi usage#1750
Yonsun-w wants to merge 1 commit into
Project-HAMi:masterfrom
Yonsun-w:docs/update-benchmark-1722

Conversation

@Yonsun-w

Copy link
Copy Markdown
Contributor

Summary

This PR addresses issue #1722 by updating the outdated benchmark documentation.

Changes

  • Added deprecation notice: Clearly indicates that historical benchmark data is from an older version (vGPU-device-plugin) and uses outdated test environments
  • Updated terminology: Changed "vGPU-device-plugin" references to "HAMi" where appropriate
  • Added new "Running Benchmarks" section: Provides up-to-date instructions for running benchmarks with current HAMi deployment
  • Improved documentation structure:
    • Clear prerequisites section
    • Step-by-step instructions for building benchmark images (optional)
    • Instructions for running both HAMi and NVIDIA device plugin benchmarks for comparison
    • Guide for viewing and customizing benchmark results
  • Formatting improvements: Capitalized table headers for consistency

Files Changed

  • docs/benchmark.md - English version updated
  • docs/benchmark_cn.md - Chinese version updated

The historical benchmark results are preserved for reference but clearly marked as legacy data, ensuring users understand these numbers may not reflect current performance.

Test Plan

  • Reviewed both English and Chinese documentation for accuracy
  • Verified all links point to correct locations
  • Ensured markdown formatting is correct
  • Confirmed instructions reference current file paths in the repository

Related Issue

Fixes #1722


AI Assistance Disclosure: This PR was written primarily by Claude Code (Opus 4.6).

🤖 Generated with Claude Code

@hami-robot hami-robot Bot requested a review from archlitchi April 13, 2026 03:07
@hami-robot

hami-robot Bot commented Apr 13, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Yonsun-w
Once this PR has been reviewed and has the lgtm label, please assign shouren for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hami-robot hami-robot Bot requested a review from DSFans2014 April 13, 2026 03:07
@github-actions github-actions Bot added the kind/documentation Improvements or additions to documentation label Apr 13, 2026
@hami-robot hami-robot Bot added the size/L label Apr 13, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the benchmark documentation in both English and Chinese, designating older results as historical and adding a new section with instructions for running current benchmarks. The review feedback identifies several necessary corrections: standardizing directory paths to be relative to the repository root and fixing incorrect job names in the log retrieval commands to ensure they match the actual deployment configurations.

Comment thread docs/benchmark.md Outdated
Comment thread docs/benchmark.md Outdated
Comment thread docs/benchmark.md Outdated
Comment thread docs/benchmark_cn.md Outdated
Comment thread docs/benchmark_cn.md Outdated
Comment thread docs/benchmark_cn.md Outdated
Fixes Project-HAMi#1722

This commit addresses the outdated benchmark documentation by:
- Adding a clear notice that the historical benchmark data is from an older version (vGPU-device-plugin)
- Updating terminology from "vGPU-device-plugin" to "HAMi" where appropriate
- Adding a new "Running Benchmarks" section with up-to-date instructions
- Providing clear steps for running benchmarks using the current HAMi deployment
- Including instructions for both HAMi and NVIDIA device plugin benchmarks for comparison
- Improving formatting consistency (capitalizing table headers)
- Using paths relative to repository root (removed HAMi/ prefix)
- Correcting job names to match actual deployment configurations

Both English (benchmark.md) and Chinese (benchmark_cn.md) versions have been updated.

The historical benchmark results are preserved for reference but clearly marked as legacy data.

---
This PR was written primarily by Claude Code (Opus 4.6).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Wenjiahua <Wenjiahua@kanyun.com>
@Yonsun-w Yonsun-w force-pushed the docs/update-benchmark-1722 branch from ea75d7e to e631d56 Compare April 13, 2026 03:18
@Yonsun-w

Copy link
Copy Markdown
Contributor Author

Updates Applied

I've addressed all the AI-suggested improvements:

Fixed Issues ✅

  1. Path corrections: Removed HAMi/ prefix from paths to use repository-relative paths:

    • cd HAMi/benchmarks/ai-benchmarkcd benchmarks/ai-benchmark
    • cd HAMi/benchmarks/deploymentscd benchmarks/deployments
  2. Job name correction: Fixed the kubectl logs command to use the correct job name:

    • kubectl logs job/ai-benchmark-on-nvidia-device-pluginkubectl logs job/ai-benchmark-on-official

    This matches the actual job name defined in benchmarks/deployments/job-on-nvidia-device-plugin.yml

  3. DCO signoff: Added the required Signed-off-by line to the commit

Changes Applied To:

  • docs/benchmark.md (English version)
  • docs/benchmark_cn.md (Chinese version)

All AI suggestions from the code review have been implemented. The DCO check is now passing ✅

Comment thread docs/benchmark.md
```

3. View the result by using kubctl logs
For installing the official NVIDIA device plugin, refer to the [NVIDIA k8s-device-plugin Quick Start](https://github.com/NVIDIA/k8s-device-plugin?tab=readme-ov-file#quick-start).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Yonsun-w I think it is better to replace For installing with To install

@archlitchi

Copy link
Copy Markdown
Member

Hi, we've updated the benchmarks, please checkout the latest version

@mesutoezdil

Copy link
Copy Markdown
Contributor

@Yonsun-w two small issues:

  1. The second kubectl apply block is missing cd benchmarks/deployments. Users who copy-paste it from a fresh shell will get a file-not-found error.

  2. @archlitchi mentioned on Apr 27 that benchmarks were updated. Can you check if those changes should be reflected here too?

@mesutoezdil

Copy link
Copy Markdown
Contributor

Note: As of today, this issue/PR has been flagged under our new activity policy. If there is no response or update by July 18, 2026, it will be closed. It can be reopened at any time if still relevant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes kind/documentation Improvements or additions to documentation size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Community Request: Update benchmarks

5 participants