feat(model): add OrcaRouter as an OpenAI-compatible model provider#2698
Open
zhenjunchen-png wants to merge 1 commit into
Open
feat(model): add OrcaRouter as an OpenAI-compatible model provider#2698zhenjunchen-png wants to merge 1 commit into
zhenjunchen-png wants to merge 1 commit into
Conversation
OrcaRouter is an OpenAI-compatible router/aggregator: one API key reaches many upstream models across vendors, and orcarouter/auto picks an upstream per request by policy (cost/quality/latency/adaptive). This change is additive only and does not modify any existing provider's code, config, or tests. - add ModelClass.OrcaRouter (thrift + generated Go/TS) and map the 'orcarouter' protocol string to it - register the OrcaRouter provider (icon/name/description) in the provider list, consistent with the existing providers - add a dedicated model builder that reuses the OpenAI chat-model path, defaults the base URL to OrcaRouter, injects optional attribution headers (HTTP-Referer/X-Title) without overriding caller headers, and withholds sampling params for orcarouter/auto and known reasoning models (which reject temperature) while keeping full control for pinned models - add a model template (default orcarouter/auto), model_meta entries (default + flagship presets), and the provider icon - unit tests for attribution-header injection, sampling-param suppression, and builder registration I'm an engineer on the OrcaRouter team.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Check the PR title.
(Optional) More detailed description for this PR.
This PR adds OrcaRouter as a model provider. OrcaRouter is an OpenAI-compatible router/aggregator: one API key reaches many upstream models across vendors, and
orcarouter/autopicks an upstream per request by policy (cost / quality / latency / adaptive). Disclosure: I'm an engineer on the OrcaRouter team.Scope: additive only. No existing provider's code, config, or tests are modified.
What changed
idl/app/developer_api.thrift+ generateddeveloper_api.go/developer_api.ts: addModelClass.OrcaRouter.bizpkg/config/modelmgr/deprecate_model_get.go: map protocol stringorcaroutertoModelClass_OrcaRouter.bizpkg/config/modelmgr/mode_provider.go: add the OrcaRouter entry to the provider list (icon / name / description), consistent with the other providers.bizpkg/llm/modelbuilder/orcarouter.go)openaibuilder.HTTP-Referer/X-Title) via a customhttp.Client, without overriding caller-set headers. Upstreams that don't understand them ignore them.https://api.orcarouter.ai/v1when none is supplied, so the protocol can't accidentally fall back to the upstream OpenAI default.orcarouter/autoand known reasoning models, sampling params (temperature / top_p / penalties) are withheld, because the adaptive router's upstream is unknown per request and reasoning models rejecttemperature(HTTP 400). Pinned non-reasoning models keep full parameter control. (A general fix belongs in the gateway; this is a client-side guard.)model_builder.go's dispatch map.conf/model/template/model_template_orcarouter.yaml: template defaulting toorcarouter/auto.conf/model/model_meta.json(+ helm copy): anOrcaRouterentry withdefaultplus a small set of flagship presets. Reasoning presets omittemperatureso the bot config UI does not render a slider that would be ignored.docker/volumes/minio/default_icon/orcarouter.png: provider icon.Tests: unit tests in
orcarouter_test.gocover attribution-header injection (added when absent, never overrides, never mutates the caller's request), the sampling-param suppression decision table, and builder registration.go build ./...passes.How to verify
docker compose -f docker/docker-compose.yml --env-file docker/.env up -d(build the server image from this branch).http://localhost:8888/admin/#model-management. The OrcaRouter provider card appears.sk-orca-...), modelorcarouter/auto(base URL is pre-filled).Full model catalog:
https://www.orcarouter.ai/models.(Optional) Which issue(s) this PR fixes:
Fixes #2697