Skip to content

Add sentencepiece@0.2.2#9664

Merged
bazel-io merged 1 commit into
bazelbuild:mainfrom
BYVoid:sentencepiece-0.2.2
Jul 14, 2026
Merged

Add sentencepiece@0.2.2#9664
bazel-io merged 1 commit into
bazelbuild:mainfrom
BYVoid:sentencepiece-0.2.2

Conversation

@BYVoid

@BYVoid BYVoid commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Adds SentencePiece v0.2.2, following up on #9552 (sentencepiece@0.2.1).

Same approach as 0.2.1, with the Bazel build definitions taken from the upstream PR google/sentencepiece#1280 (not yet merged), adapted for the 0.2.2 release tarball:

  • overlay/BUILD.bazel + overlay/src/BUILD.bazel: root aliases, //src:sentencepiece (runtime), //src:sentencepiece_train (trainer), and the spm_{train,encode,decode,normalize,export_vocab} CLIs. The protobuf code is regenerated from src/*.proto at build time (matching the SPM_PROTOBUF_PROVIDER=package CMake configuration).
  • patches/use_bcr_dependency_includes.patch: rewrites #include "third_party/absl/...""absl/...", "third_party/darts_clone/darts.h""darts.h", and "third_party/esaxx/esa.hxx""esa.hxx" so the sources resolve against the BCR modules (abseil-cpp, darts-clone@0.32h.bcr.1, esaxx@20250106.1.bcr.1) instead of the vendored copies.
  • The release tarball is a Python sdist, so strip_prefix is sentencepiece-0.2.2/sentencepiece. It does not ship the data/ directory, so the upstream spm_test target is not included. Instead, the overlay adds a bcr_tests/ test module (bcr_test_module in presubmit) with an in-memory train → encode → decode roundtrip cc_test; presubmit also verifies all library and CLI targets on Linux/macOS/Windows with C++17 flags.

Differences from 0.2.1: 0.2.2 uses init.cc (renamed from error.cc), migrated to absl::Status/log (more abseil-cpp deps, no _USE_EXTERNAL_ABSL define needed), and its MODULE.bazel pins abseil-cpp 20260526.0 to match the GIT_TAG in CMakeLists.txt.

Locally verified: all targets build (Bazel 9.1.1), the bcr_tests roundtrip test passes, and bcr_validation.py --check sentencepiece@0.2.2 passes.

@BYVoid
BYVoid marked this pull request as ready for review July 14, 2026 01:21
@bazel-io

Copy link
Copy Markdown
Member

Hello BCR maintainers, modules (sentencepiece) have been updated in this PR.
Please review the changes. You can view a diff against the previous version in the "Generate module diff" check.

@bazel-io
bazel-io requested review from a team, Wyverald and fmeum and removed request for a team July 14, 2026 01:21

@bazel-io bazel-io left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All modules in this PR have been approved by their maintainers. This PR will be merged if all presubmit checks pass.

@bazel-io
bazel-io merged commit 45475da into bazelbuild:main Jul 14, 2026
24 checks passed
@bazel-io bazel-io added the auto-merged This PR is automatically merged by the BCR reviewer bot. label Jul 14, 2026
@BYVoid
BYVoid deleted the sentencepiece-0.2.2 branch July 14, 2026 05:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merged This PR is automatically merged by the BCR reviewer bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants