Skip to content

Fix AnthropicModel.count_tokens with native tools#5704

Open
kazmer97 wants to merge 1 commit into
pydantic:mainfrom
kazmer97:codex/fix-anthropic-count-tokens-native-tools
Open

Fix AnthropicModel.count_tokens with native tools#5704
kazmer97 wants to merge 1 commit into
pydantic:mainfrom
kazmer97:codex/fix-anthropic-count-tokens-native-tools

Conversation

@kazmer97
Copy link
Copy Markdown

@kazmer97 kazmer97 commented May 28, 2026

Summary

  • Omit Anthropic native/server tools from count_tokens requests by clearing native_tools on the count-only request parameters.
  • Preserve native tools for normal message creation.
  • Add a regression test covering CodeExecutionTool and WebFetchTool during preflight token counting.

Validation

  • uv run pytest tests/models/test_anthropic.py::test_anthropic_count_tokens_with_mock tests/models/test_anthropic.py::test_anthropic_count_tokens_with_no_messages tests/models/test_anthropic.py::test_anthropic_count_tokens_omits_native_tools tests/models/test_anthropic.py::test_anthropic_code_execution_tool_version_auto tests/models/test_anthropic.py::test_anthropic_code_execution_tool_version_setting
  • uv run ruff check pydantic_ai_slim/pydantic_ai/models/anthropic.py tests/models/test_anthropic.py
  • uv run ruff format --check pydantic_ai_slim/pydantic_ai/models/anthropic.py tests/models/test_anthropic.py
  • PYRIGHT_PYTHON_IGNORE_WARNINGS=1 uv run pyright pydantic_ai_slim/pydantic_ai/models/anthropic.py

Checklist

  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • No breaking changes in accordance with the version policy.
  • PR title is fit for the release changelog.

@github-actions github-actions Bot added the size: S Small PR (≤100 weighted lines) label May 28, 2026
@github-actions github-actions Bot added the bug Report that something isn't working, or PR implementing a fix label May 28, 2026
@kazmer97 kazmer97 force-pushed the codex/fix-anthropic-count-tokens-native-tools branch from 81b76a7 to 885ab96 Compare May 28, 2026 16:19
@kazmer97 kazmer97 marked this pull request as ready for review May 28, 2026 16:27
@github-actions github-actions Bot closed this May 28, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Thanks for your interest in this issue! However, there are already open PRs addressing issue #5702: #5703.

To avoid duplicate efforts, this PR has been closed. If you'd like to contribute, you can review the existing PRs or share your thoughts on issue #5702.

If you believe the existing PRs are inactive, please comment on the issue and a maintainer can reassess.

@dsfaccini
Copy link
Copy Markdown
Collaborator

sorry about this @kazmer97 , sometimes issues/PRs get buried under spam, feel free to ping us directly on slack when this happens so we can act more quickly

let me do a review of the PR and get back to you, I may push updates if they're needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Report that something isn't working, or PR implementing a fix size: S Small PR (≤100 weighted lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic count_tokens fails when native CodeExecutionTool is enabled

2 participants