Add sentencepiece@0.2.2#9664
Merged
Merged
Conversation
BYVoid
marked this pull request as ready for review
July 14, 2026 01:21
Member
|
Hello BCR maintainers, modules (sentencepiece) have been updated in this PR. |
bazel-io
requested review from
a team,
Wyverald and
fmeum
and removed request for
a team
July 14, 2026 01:21
bazel-io
approved these changes
Jul 14, 2026
bazel-io
left a comment
Member
There was a problem hiding this comment.
All modules in this PR have been approved by their maintainers. This PR will be merged if all presubmit checks pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds SentencePiece v0.2.2, following up on #9552 (sentencepiece@0.2.1).
Same approach as 0.2.1, with the Bazel build definitions taken from the upstream PR google/sentencepiece#1280 (not yet merged), adapted for the 0.2.2 release tarball:
overlay/BUILD.bazel+overlay/src/BUILD.bazel: root aliases,//src:sentencepiece(runtime),//src:sentencepiece_train(trainer), and thespm_{train,encode,decode,normalize,export_vocab}CLIs. The protobuf code is regenerated fromsrc/*.protoat build time (matching theSPM_PROTOBUF_PROVIDER=packageCMake configuration).patches/use_bcr_dependency_includes.patch: rewrites#include "third_party/absl/..."→"absl/...","third_party/darts_clone/darts.h"→"darts.h", and"third_party/esaxx/esa.hxx"→"esa.hxx"so the sources resolve against the BCR modules (abseil-cpp, darts-clone@0.32h.bcr.1, esaxx@20250106.1.bcr.1) instead of the vendored copies.strip_prefixissentencepiece-0.2.2/sentencepiece. It does not ship thedata/directory, so the upstreamspm_testtarget is not included. Instead, the overlay adds abcr_tests/test module (bcr_test_modulein presubmit) with an in-memory train → encode → decode roundtripcc_test; presubmit also verifies all library and CLI targets on Linux/macOS/Windows with C++17 flags.Differences from 0.2.1: 0.2.2 uses
init.cc(renamed fromerror.cc), migrated to absl::Status/log (more abseil-cpp deps, no_USE_EXTERNAL_ABSLdefine needed), and itsMODULE.bazelpins abseil-cpp 20260526.0 to match theGIT_TAGinCMakeLists.txt.Locally verified: all targets build (Bazel 9.1.1), the
bcr_testsroundtrip test passes, andbcr_validation.py --check sentencepiece@0.2.2passes.