Skip to content

feat(model): add OrcaRouter as an OpenAI-compatible model provider#2698

Open
zhenjunchen-png wants to merge 1 commit into
coze-dev:mainfrom
zhenjunchen-png:feat/orcarouter-provider
Open

feat(model): add OrcaRouter as an OpenAI-compatible model provider#2698
zhenjunchen-png wants to merge 1 commit into
coze-dev:mainfrom
zhenjunchen-png:feat/orcarouter-provider

Conversation

@zhenjunchen-png
Copy link
Copy Markdown

@zhenjunchen-png zhenjunchen-png commented May 29, 2026

What type of PR is this?

feat

Check the PR title.

  • This PR title match the format: <type>(optional scope): <description>
  • The description of this PR title is user-oriented and clear enough for others to understand.
  • Add documentation if the current PR requires user awareness at the usage level.

(Optional) More detailed description for this PR.

This PR adds OrcaRouter as a model provider. OrcaRouter is an OpenAI-compatible router/aggregator: one API key reaches many upstream models across vendors, and orcarouter/auto picks an upstream per request by policy (cost / quality / latency / adaptive). Disclosure: I'm an engineer on the OrcaRouter team.

Scope: additive only. No existing provider's code, config, or tests are modified.

What changed

  • Protocol / provider registration
    • idl/app/developer_api.thrift + generated developer_api.go / developer_api.ts: add ModelClass.OrcaRouter.
    • bizpkg/config/modelmgr/deprecate_model_get.go: map protocol string orcarouter to ModelClass_OrcaRouter.
    • bizpkg/config/modelmgr/mode_provider.go: add the OrcaRouter entry to the provider list (icon / name / description), consistent with the other providers.
  • Model builder (bizpkg/llm/modelbuilder/orcarouter.go)
    • Reuses the existing OpenAI chat-model construction (OrcaRouter speaks the OpenAI wire protocol), so request/response handling stays in lockstep with the openai builder.
    • Adds optional attribution headers (HTTP-Referer / X-Title) via a custom http.Client, without overriding caller-set headers. Upstreams that don't understand them ignore them.
    • Defaults the base URL to https://api.orcarouter.ai/v1 when none is supplied, so the protocol can't accidentally fall back to the upstream OpenAI default.
    • For orcarouter/auto and known reasoning models, sampling params (temperature / top_p / penalties) are withheld, because the adaptive router's upstream is unknown per request and reasoning models reject temperature (HTTP 400). Pinned non-reasoning models keep full parameter control. (A general fix belongs in the gateway; this is a client-side guard.)
    • Registered in model_builder.go's dispatch map.
  • Presets / metadata (following existing conventions)
    • conf/model/template/model_template_orcarouter.yaml: template defaulting to orcarouter/auto.
    • conf/model/model_meta.json (+ helm copy): an OrcaRouter entry with default plus a small set of flagship presets. Reasoning presets omit temperature so the bot config UI does not render a slider that would be ignored.
    • docker/volumes/minio/default_icon/orcarouter.png: provider icon.

Tests: unit tests in orcarouter_test.go cover attribution-header injection (added when absent, never overrides, never mutates the caller's request), the sampling-param suppression decision table, and builder registration. go build ./... passes.

How to verify

  1. docker compose -f docker/docker-compose.yml --env-file docker/.env up -d (build the server image from this branch).
  2. Open http://localhost:8888/admin/#model-management. The OrcaRouter provider card appears.
  3. Add a model: API key (sk-orca-...), model orcarouter/auto (base URL is pre-filled).
  4. Create a bot using that model and send a message. The reply is served through OrcaRouter's adaptive routing.

Full model catalog: https://www.orcarouter.ai/models.

(Optional) Which issue(s) this PR fixes:

Fixes #2697

OrcaRouter is an OpenAI-compatible router/aggregator: one API key reaches
many upstream models across vendors, and orcarouter/auto picks an upstream
per request by policy (cost/quality/latency/adaptive).

This change is additive only and does not modify any existing provider's
code, config, or tests.

- add ModelClass.OrcaRouter (thrift + generated Go/TS) and map the
  'orcarouter' protocol string to it
- register the OrcaRouter provider (icon/name/description) in the provider
  list, consistent with the existing providers
- add a dedicated model builder that reuses the OpenAI chat-model path,
  defaults the base URL to OrcaRouter, injects optional attribution headers
  (HTTP-Referer/X-Title) without overriding caller headers, and withholds
  sampling params for orcarouter/auto and known reasoning models (which
  reject temperature) while keeping full control for pinned models
- add a model template (default orcarouter/auto), model_meta entries
  (default + flagship presets), and the provider icon
- unit tests for attribution-header injection, sampling-param suppression,
  and builder registration

I'm an engineer on the OrcaRouter team.
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented May 29, 2026

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add OrcaRouter as an OpenAI-compatible model provider

2 participants