Skip to content

onnxruntime: disable NCCL per default as it fails to compile since the last update - #546053

Closed
SuperSandro2000 wants to merge 1 commit into
NixOS:masterfrom
SuperSandro2000:onnxruntime-cuda
Closed

onnxruntime: disable NCCL per default as it fails to compile since the last update#546053
SuperSandro2000 wants to merge 1 commit into
NixOS:masterfrom
SuperSandro2000:onnxruntime-cuda

Conversation

@SuperSandro2000

Copy link
Copy Markdown
Member

Things done

@nixpkgs-ci nixpkgs-ci Bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Jul 26, 2026
@ConnorBaker ConnorBaker added the 6.topic: cuda Parallel computing platform and API label Jul 26, 2026
@ConnorBaker

Copy link
Copy Markdown
Contributor

Can you post the compilation failure? And is it NCCL or Onnxruntime failing to compile?

@SuperSandro2000
SuperSandro2000 marked this pull request as ready for review July 26, 2026 20:37
@nixpkgs-ci
nixpkgs-ci Bot requested review from ck3d and puffnfresh July 26, 2026 20:38
@nixpkgs-ci nixpkgs-ci Bot added the 2.status: merge-bot eligible This PR can be merged by commenting "@NixOS/nixpkgs-merge-bot merge". label Jul 26, 2026
@SuperSandro2000

Copy link
Copy Markdown
Member Author

Its onnxruntime when NCCL is enabled.

The last log lines related to the failures:

[ 88%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/build/source/onnxruntime/contrib_ops/cuda/collective/sharding.cc.o
In file included from /build/source/onnxruntime/contrib_ops/cuda/collective/sharded_moe.cc:10:
/build/source/onnxruntime/contrib_ops/cuda/collective/sharded_moe.h:6:10: fatal error: contrib_ops/cuda/moe/ft_moe/moe_kernel.h: No such file or directory
    6 | #include "contrib_ops/cuda/moe/ft_moe/moe_kernel.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/onnxruntime_providers_cuda.dir/build.make:5365: CMakeFiles/onnxruntime_providers_cuda.dir/build/source/onnxruntime/contrib_ops/cuda/collective/sharded_moe.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:1504: CMakeFiles/onnxruntime_providers_cuda.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

@SuperSandro2000

Copy link
Copy Markdown
Member Author

I think this broke with this PR upstream microsoft/onnxruntime#28467 (comment)

@ConnorBaker ConnorBaker self-assigned this Jul 27, 2026
@ConnorBaker

Copy link
Copy Markdown
Contributor

Does naively patching out the include fix compilation?

@GaetanLepage

Copy link
Copy Markdown
Contributor

Superseded by #547077.

@github-project-automation github-project-automation Bot moved this from New to ✅ Done in CUDA Team Jul 29, 2026
@SuperSandro2000
SuperSandro2000 deleted the onnxruntime-cuda branch July 29, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge-bot eligible This PR can be merged by commenting "@NixOS/nixpkgs-merge-bot merge". 6.topic: cuda Parallel computing platform and API 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

3 participants