feat: support configurable proxy host and user-provided aiohttp client#5632
Open
Incheonkirin wants to merge 1 commit into
Open
feat: support configurable proxy host and user-provided aiohttp client#5632Incheonkirin wants to merge 1 commit into
Incheonkirin wants to merge 1 commit into
Conversation
`create_proxy_app` hardcoded `localhost` as the proxy host and could not use a user-provided `aiohttp.ClientSession` (set via `self.client`), because it relied on `base_url` being configured on the session — which a user cannot set, since BentoML's internal proxy port is unknown at client-creation time. - Add an optional `http.proxy_host` config (default `localhost`) so requests can be forwarded to a remote/external upstream server. - When the proxy host is remote, do not spawn the upstream command locally (should_start_process = False). - Build absolute URLs in the reverse proxy and the health check instead of relying on the session's base_url, so a user-provided client works as-is. - Register `proxy_host` in the v2 config schema, the SDK HTTPSchema, and the default configuration. Closes bentoml#5568 Signed-off-by: Mingi Jeong <incheonkirin@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
create_proxy_apphardcodedlocalhostas the proxy host and could not use a user-providedaiohttp.ClientSession(set viaself.client). Closes #5568.The user's session has no
base_url— they cannot know BentoML's internalproxy_portat client-creation time — so the existing code (which relied onbase_urland built relative URLs) broke for user-provided clients. The host was also hardcoded tolocalhost, so requests could not be forwarded to a remote/external upstream server.Changes
http.proxy_host(defaultlocalhost) so requests can be forwarded to a remote/external ML server.proxy_hostis remote,should_start_processisFalse— BentoML cannot spawn the upstream command on another machine.reverse_proxyand the startup health check instead of relying on the session'sbase_url, so a user-provided client works as-is (custom auth/SSL/retries supported).proxy_hostin the v2 config schema, the SDKHTTPSchema, and the default configuration.Testing
py_compilepasses for all changed Python files; default config YAML parses.localhost/127.0.0.1/::1→ local; others → remote).Notes
Backward compatible: when
proxy_hostis unset it defaults tolocalhost, preserving current behavior. No new dependencies.