Add Anthropic as a supported LLM provider - #49
Conversation
3ea3346 to
c57fa12
Compare
| that are used by this container. | ||
|
|
||
| This field depends on the | ||
| This is an alpha field and requires enabling the |
There was a problem hiding this comment.
most nit: These changes could be on a separated commit since it's unrelated
| // don't define a base_url field at all, so setting llmEndpoint for those | ||
| // providers has no effect on the generated config - Llama Stack silently | ||
| // ignores it on its end. That's upstream (llama-stack/OGX) behavior, not | ||
| // something this operator works around. |
There was a problem hiding this comment.
Oh, so for those providers OGX will totally ignore the custom URL ? But not for openai for example ?
That's such a weird behavior.
(Nothing to do on our side, since it's OGX/Llama-Stack)
There was a problem hiding this comment.
remote::gemini/remote::anthropic configs simply have no base_url field so it's silently dropped by Pydantic. Nothing we can do on our side it's an Llama-Stack schema thing. and yes I won't ignore for openai provider
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: omkarjoshi0304, umago The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
c57fa12 to
6dffbfe
Compare
Unrelated to the Anthropic changes; just re-syncs generated output with the currently pinned k8s.io/api version. Co-authored-by: Cursor <cursoragent@cursor.com>
Adds anthropic to the LLMEndpointType enum and generates a remote::anthropic inference provider config. Co-authored-by: Cursor <cursoragent@cursor.com>
6dffbfe to
1c57a6b
Compare
|
@omkarjoshi0304: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
Converting this PR in draft , since we need few dependencies from Lcore team inorder to enable us to use anthropic provider which they are planing for Q4. |
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
2 participants
Adds anthropic to the LLMEndpointType enum and generates a remote::anthropic inference provider config.