Skip to content

docs(ops): document OrcaRouter as a LiteLLM model provider - #2354

Open
Marc-oss-hub wants to merge 1 commit into
cocoindex-io:mainfrom
Marc-oss-hub:docs/orcarouter-litellm-provider
Open

docs(ops): document OrcaRouter as a LiteLLM model provider#2354
Marc-oss-hub wants to merge 1 commit into
cocoindex-io:mainfrom
Marc-oss-hub:docs/orcarouter-litellm-provider

Conversation

@Marc-oss-hub

Copy link
Copy Markdown

Summary

Adds OrcaRouter to the LiteLLM provider docs. OrcaRouter is an OpenAI-compatible model routing gateway that exposes models from OpenAI, Anthropic, Google, DeepSeek, Qwen and others behind a single endpoint and API key. It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

I'm an engineer on the OrcaRouter team.

What this changes

  • docs/src/content/docs/ops/litellm.mdx: adds an ### OrcaRouter subsection under "Supported providers", alongside the existing OpenAI / Azure / Gemini entries, showing the LiteLLMEmbedder wiring with api_base and api_key.
  • docs/src/content/docs/ops/entity_resolution.mdx: adds an OrcaRouter row to the LlmPairResolver model strings table, with a one-line note on the namespaced model-id form.

How to use it

from cocoindex.ops.litellm import LiteLLMEmbedder

embedder = LiteLLMEmbedder(
    "openai/openai/text-embedding-3-small",
    api_base="https://api.orcarouter.ai/v1",
    api_key="sk-orca-...",
)

The outer openai/ prefix selects the OpenAI-compatible protocol; the inner openai/… is the router's namespaced model id (passed through verbatim). The same shape works for LlmPairResolver(model="openai/openai/gpt-5.5", ...) in entity resolution.

Verification

  • npm run build in docs/ passes (98 pages; the 5 pre-existing missing .md twin notices for index.mdx pages are unchanged by this PR).
  • Live connectivity against https://api.orcarouter.ai/v1 with an sk-orca- key:
    • litellm.aembedding("openai/openai/text-embedding-3-small", api_base=..., api_key=...) → 200, 1536-dim vector.
    • litellm.acompletion("openai/openai/gpt-5.5", ...) → 200.
    • Negative control: a bare text-embedding-3-small (no openai/ prefix) → 503 No available channel, which is why the docs show the namespaced form.

Add an OrcaRouter subsection to the LiteLLM embeddings provider list and
a row to the entity-resolution model strings table. OrcaRouter is an
OpenAI-compatible routing gateway; the docs show the namespaced model-id
form (openai/openai/...) with api_base set to https://api.orcarouter.ai/v1.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant