Upgrade environment schema to support additional scenarios

[toniher]

The schema now accepts two additional scenarios for conda environment definitions:

1. Bare pip dependency (5 lines added)

   • Allows specifying pip as a standalone string dependency
   • Enables pip-based installs in the environment

2. Flexible pip package specifications (refactored pip packages validation)

   • Previously: only accepted version-pinned format (package==version)
   • Now accepts either:
     • Version-pinned packages: package==version (maintains reproducibility)
     • Direct URLs: http:// or https:// links to package artifacts (e.g., wheel files)

Motivation: GitHub issue #11652 — this expands the schema to handle more real-world dependency scenarios while preserving reproducibility where pinning is specified.

---

Message created by Claude Haiku 4.5

[toniher]

Hey @mashehu , any feedback or further idea?

[mashehu]

Sorry for the late response, currently on vacation.

We already allow pip dependencies, see

modules/modules/nf-core/multiqcsav/environment.yml

Line 12 in c579dd3

- pip:

the only requirement is that you also pin pip itself to a specific version.

Concerning the URLs:
If there is a release, can they not be added to bioconda, condaforge instead?

[toniher]

Sorry for the late response, currently on vacation.

We already allow pip dependencies, see

modules/modules/nf-core/multiqcsav/environment.yml

Line 12 in c579dd3

- pip:

the only requirement is that you also pin pip itself to a specific version.

Concerning the URLs: If there is a release, can they not be added to bioconda, condaforge instead?

No problem for the delay

The need is explained here: https://github.com/nf-core/website/pull/4142/changes (around line 146)

[toniher]

the only requirement is that you also pin pip itself to a specific version.

Regarding the plain pip, I don't know if this is needed somehow... Maybe @pinin4fjords can help...

[pinin4fjords]

Thanks for the PR @toniher, and sorry for the slow loop back on the ping.

I went down a rabbit hole on the llama-cpp-python case specifically (it's the example behind the GPU module guidelines in nf-core/website#4142, so I had some scar tissue here). Worth being clear up front: the wheel-URL guidance in that spec was right when I wrote it - at the time conda-forge had dropped the CUDA builds of the llama-cpp-python wrapper and the backend split hadn't landed, so pinning the pip wheel really was the only route, which is also why you reasonably pointed at it.

conda-forge has shifted since then. The GPU support now lives in the separately-packaged llama.cpp backend, which you select with a build-string match:

dependencies:
  - "conda-forge::llama-cpp-python=0.3.16"
  - "conda-forge::llama.cpp=6191=*cuda*"

(6191 is the backend version the 0.3.16 wrapper requires; the =6191=*cuda* form pins it and lets the build glob select the CUDA variant, so everything stays pinned.)

The *cuda* build glob is doing real work, not decoration. Building a GPU conda env on Wave needs the v2 build template (--build-template conda/micromamba:v2); the stock build image can't resolve the __cuda-gated CUDA packages otherwise. With that template in place I built both forms via Wave and inspected the resulting images:

Wave build (--build-template conda/micromamba:v2)	backend in the image
llama.cpp=6191=*cuda*	llama.cpp 6191 cuda126 + libcublas/libcudart (GPU)
bare wrapper, no glob	llama.cpp 6191 cpu_mkl, no CUDA libs (CPU)

So without the explicit =*cuda* the GPU environment.gpu.yml silently bakes a CPU container - the glob is what selects the CUDA backend. (Runtime offload of the CUDA backend itself I confirmed separately on a Tesla T4: all layers offloaded to the GPU.)

[toniher]

So, thanks to what you explain @pinin4fjords , at least for the llama-cpp-python module, this change would not be needed. I will try it and would then close this PR and the associated issue.

I was trying 0.3.23 version, but I think it is reasonable to wait for the upcoming release to appear in conda-forge.

[toniher]

I was trying 0.3.23 version, but I think it is reasonable to wait for the upcoming release to appear in conda-forge.

Mmm... I notice now at: https://anaconda.org/channels/conda-forge/packages/llama-cpp-python/overview that there are not many frequent releases :'( That's a bit concerning, but maybe the way is trying to help the packaging in conda-forge. I will research here: https://github.com/conda-forge/llama-cpp-python-feedstock

[pinin4fjords]

I was trying 0.3.23 version, but I think it is reasonable to wait for the upcoming release to appear in conda-forge.

Mmm... I notice now at: https://anaconda.org/channels/conda-forge/packages/llama-cpp-python/overview that there are not many frequent releases :'( That's a bit concerning, but maybe the way is trying to help the packaging in conda-forge. I will research here: https://github.com/conda-forge/llama-cpp-python-feedstock

Yep, I'm feeling my way around myself, if you find a better way let us know!

[toniher]

I managed to upgrade llama-cpp-python conda package upstream and upgrade accordingly the associated module here: #11985 - So I close this pull request since it would not be needed.