Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/en/latest/plugins/ai-proxy-multi.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,12 @@ When an instance's `provider` is set to `bedrock`, the Plugin expects requests i
| keepalive_pool | integer | False | 30 | | Keepalive pool size for when connecting with the LLM service. |
| ssl_verify | boolean | False | true | | If true, verify the LLM service's certificate. |

## Request Header Forwarding

By default, `ai-proxy-multi` forwards the incoming client request headers to the selected LLM upstream. Only `Host`, `Content-Length`, and `Accept-Encoding` are dropped, and `Content-Type` is forced to `application/json`. Headers configured under an instance's `auth.header` are merged on top and take precedence over client headers of the same name.

Because the LLM upstream is often a third-party service, be aware that any header the client sends (for example `Authorization`, `Cookie`, or internal application headers) is forwarded to that provider unless it is overridden by `auth.header`. If the client should not expose certain headers to the LLM provider, strip them before the request reaches `ai-proxy-multi`, for example with the [`proxy-rewrite`](./proxy-rewrite.md) plugin.

## Examples

The examples below demonstrate how you can configure `ai-proxy-multi` for different scenarios.
Expand Down
6 changes: 6 additions & 0 deletions docs/en/latest/plugins/ai-proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,12 @@ Priority between client request and override is controlled by `override.request_

When both `llm_options` and `request_body` are configured, `llm_options` is applied first (always force), then `request_body` deep-merges on top. This means `request_body` can override fields set by `llm_options`.

## Request Header Forwarding

By default, `ai-proxy` forwards the incoming client request headers to the configured LLM upstream. Only `Host`, `Content-Length`, and `Accept-Encoding` are dropped, and `Content-Type` is forced to `application/json`. Headers configured under `auth.header` are merged on top and take precedence over client headers of the same name.

Because the LLM upstream is often a third-party service, be aware that any header the client sends (for example `Authorization`, `Cookie`, or internal application headers) is forwarded to that provider unless it is overridden by `auth.header`. If the client should not expose certain headers to the LLM provider, strip them before the request reaches `ai-proxy`, for example with the [`proxy-rewrite`](./proxy-rewrite.md) plugin.

## Examples

The examples below demonstrate how you can configure `ai-proxy` for different scenarios.
Expand Down
6 changes: 6 additions & 0 deletions docs/zh/latest/plugins/ai-proxy-multi.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,12 @@ import TabItem from '@theme/TabItem';
| keepalive_pool | integer | 否 | 30 | | 连接 LLM 服务时的保活池大小。 |
| ssl_verify | boolean | 否 | true | | 如果为 true,验证 LLM 服务的证书。 |

## 请求头转发

默认情况下,`ai-proxy-multi` 会将传入的客户端请求头转发到所选的 LLM 上游。仅 `Host`、`Content-Length` 和 `Accept-Encoding` 会被丢弃,并且 `Content-Type` 会被强制设置为 `application/json`。配置在某个实例的 `auth.header` 中的请求头会在其之上合并,并优先于同名的客户端请求头。

由于 LLM 上游通常是第三方服务,请注意客户端发送的任何请求头(例如 `Authorization`、`Cookie` 或内部应用请求头)都会被转发到该服务商,除非被 `auth.header` 覆盖。如果不希望客户端将某些请求头暴露给 LLM 服务商,请在请求到达 `ai-proxy-multi` 之前将其移除,例如使用 [`proxy-rewrite`](./proxy-rewrite.md) 插件。

## 示例

以下示例演示了如何为不同场景配置 `ai-proxy-multi`。
Expand Down
6 changes: 6 additions & 0 deletions docs/zh/latest/plugins/ai-proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,12 @@ apisix:

当同时配置了 `llm_options` 和 `request_body` 时,`llm_options` 先应用(始终强制覆盖),然后 `request_body` 在其基础上深度合并。这意味着 `request_body` 可以覆盖 `llm_options` 设置的字段。

## 请求头转发

默认情况下,`ai-proxy` 会将传入的客户端请求头转发到所配置的 LLM 上游。仅 `Host`、`Content-Length` 和 `Accept-Encoding` 会被丢弃,并且 `Content-Type` 会被强制设置为 `application/json`。配置在 `auth.header` 中的请求头会在其之上合并,并优先于同名的客户端请求头。

由于 LLM 上游通常是第三方服务,请注意客户端发送的任何请求头(例如 `Authorization`、`Cookie` 或内部应用请求头)都会被转发到该服务商,除非被 `auth.header` 覆盖。如果不希望客户端将某些请求头暴露给 LLM 服务商,请在请求到达 `ai-proxy` 之前将其移除,例如使用 [`proxy-rewrite`](./proxy-rewrite.md) 插件。

## 示例

以下示例演示了如何为不同场景配置 `ai-proxy`。
Expand Down
Loading