Support 'extra_body' in OpenAI spec. (to better support OpenRouter and vendor-specific parameters)

Currently, AnyLanguageModel provides excellent support for the standard OpenAI API schema. However, the ecosystem is shifting toward API proxies like OpenRouter, which serve as a unified interface for almost all relevant models (Anthropic, Google, xAI, etc.) using the OpenAI wire format.

While these proxies adhere to the OpenAI standard, they often require (or benefit from) additional parameters in the JSON payload that are not part of the strict OpenAI spec.

For reference see the reasoning parameter in https://openrouter.ai/x-ai/grok-4.1-fast/api.

In the official openai python sdk this is solved like this:

# Reference: How the standard OpenAI Python SDK handles this
response = client.chat.completions.create(
   model="x-ai/grok-4.1-fast",
   messages=[...],
   # This parameter is currently impossible to send via AnyLanguageModel
   extra_body={"reasoning": {"enabled": True}} 
)

Supporting extra_body (or similar) would not only solve the immediate issue for Grok but would effectively future-proof the library against other model providers who add custom parameters (like top_k, custom routing flags) without requiring a library update every time a vendor changes their spec.

Whilst not being fully familiar with the project, the extra_body could for instance be an optional JSONValue, perhaps part of GenerationOptions which is then passed on to respondWithChatCompletions.

@mattt

@mattt If needed I can create a PR for this. Question remains whether I should implement this for all models or for the openai models only for now and leave the extraBody field in GenerationOption unused for the other models.

@KotlinFactory

@KotlinFactory Thanks for opening this issue and thinking on this.

I think you're right that we should open this up to allow models to use the full extent of their capabilities. I see two options here, either we:

Add a catch-all userInfo-style property to GenerationOptions and leave it up to individual adapters to deal with, or
Make GenerationOptions generic on LanguageMode

Between these, I'm leaning toward the latter option. This is similar to what we did for #4. I suspect there is a way to do this without sacrificing compatibility with Foundation Models' API, and hopefully that doesn't require anything too clever.

This would also solve some existing problems in expressiveness for other language model providers, like llama.cpp, which supports more configuration than what's currently exposed by GenerationOptions.

Lemme take a stab at this today or tomorrow.

Yeah, I would agree that making GenerationOptions generic on LanguageModel is the cleaner option.

This would allow something like that, right? Thats quite nice.

let options = OpenAIGenerationOptions(
      temperature: 0.7,
      extraBody: ["reasoning": .object(["enabled": .bool(true)])]
)

Would love to use see this in production.

I started implementing my original idea in #50, but quickly painted myself into a corner. I'm pretty happy with the approach we landed on now, though.

Nice!

Just merged that. Working to pull in a few more open PRs for a new minor release. Look for that latest today 🚀

Thats really nice!

Navigation Menu

Repository navigation

Support 'extra_body' in OpenAI spec. (to better support OpenRouter and vendor-specific parameters) #39

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Support 'extra_body' in OpenAI spec. (to better support OpenRouter and vendor-specific parameters) #39

Description

Activity

KotlinFactory commented on Nov 25, 2025

mattt commented on Nov 25, 2025

KotlinFactory commented on Nov 27, 2025

mattt commented on Dec 1, 2025

KotlinFactory commented on Dec 1, 2025

mattt commented on Dec 1, 2025

KotlinFactory commented on Dec 1, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions