Skip to content

docs(intelligent-compute): document the On-Demand fallback cost limit - #1913

Open
pditommaso wants to merge 7 commits into
masterfrom
docs/intelligent-compute-ondemand-cost-limit
Open

pditommaso wants to merge 7 commits into
masterfrom
docs/intelligent-compute-ondemand-cost-limit

Conversation

@pditommaso

@pditommaso pditommaso commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The Intelligent Compute page says the spotFirst provisioning model "falls back to On-Demand if Spot capacity is unavailable". That's no longer the whole story. The scheduler now falls back only while On-Demand costs at most a set multiple of the Spot price. When On-Demand is dearer than that, the task is retried on Spot and can end up failing, which the page doesn't mention. This limit already refuses fallbacks in production (seqeralabs/sched#1291).

What changes for readers

  • Provisioning model row: spotFirst now says it falls back unless On-Demand is too expensive, and links to the new section.
  • New section On-Demand fallback cost limit under Configuration options. It explains:
    • how the limit is measured: the highest On-Demand hourly price compared with the highest Spot hourly price, across the instance types the scheduler selected for the task;

    • the current default for each cloud provider, with a note that Seqera can change them:

      Cloud provider Default limit
      AWS 3.25 times the Spot price
      Google Cloud 2.48 times the Spot price
      Azure No limit
    • what happens when the limit is exceeded: the task is retried on new Spot capacity, then fails with an error message starting on-demand fallback refused;

    • how to change the limit: currently only in the pipeline's Nextflow configuration. The compute environment has no setting for it:

      • seqera.executor.machineRequirement.onDemandCostRatio sets it for all the tasks of a run;
      • the seqera/machineRequirement.onDemandCostRatio process hint overrides it for one process, the same way as provisioning;
      • not set means the default applies; 0 means no limit (always fall back, not "never"); > 0 is the limit used;
    • what to do if a pipeline must complete when Spot is scarce: use the ondemand provisioning model. The spot model never falls back.

The default values match the scheduler defaults shipped in sched 0.96.0 (sched.scheduling.on-demand-fallback-cost-ratio-{aws,gcp,azure}), with no deployment override.

Before merging

Checks

  • markdownlint-cli2 reports the same 48 errors before and after this change, all in existing tables. The new text adds none.

🤖 Generated with Claude Code

spotFirst falls back to On-Demand only while On-Demand costs at most a
Seqera-set multiple of the Spot price; otherwise the task is retried on
Spot and eventually fails with "on-demand fallback refused".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@netlify

netlify Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for seqera-docs ready!

Name Link
🔨 Latest commit 1ceb0a6
🔍 Latest deploy log https://app.netlify.com/projects/seqera-docs/deploys/6ac3e8f393870500083a97df
😎 Deploy Preview https://deploy-preview-1913--seqera-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

pditommaso and others added 2 commits September 29, 2026 12:55
…ack cost limits

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
…tflow config

The limit can't be changed on the compute environment; a pipeline run can
override it with seqera.executor.machineRequirement.onDemandCostRatio
(Nextflow 26.10+). Unset keeps the default, 0 removes the limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
pditommaso and others added 2 commits September 29, 2026 13:27
…per process with a hint

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
…way to change the limit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@justinegeffen justinegeffen added the 1. Dev/PM/SME Needs a review by a Dev/PM/SME label Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1. Dev/PM/SME Needs a review by a Dev/PM/SME

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants