Skip to content

chore(deps): update sentencepiece requirement from >=0.1.99 to >=0.2.2 - #13

Closed
dependabot[bot] wants to merge 1 commit into
mainfrom
dependabot/pip/sentencepiece-gte-0.2.2
Closed

chore(deps): update sentencepiece requirement from >=0.1.99 to >=0.2.2#13
dependabot[bot] wants to merge 1 commit into
mainfrom
dependabot/pip/sentencepiece-gte-0.2.2

Conversation

@dependabot

@dependabot dependabot Bot commented on behalf of github Aug 16, 2026

Copy link
Copy Markdown

Updates the requirements on sentencepiece to permit the latest version.

Release notes

Sourced from sentencepiece's releases.

v0.2.2

Release Notes v0.2.2

This release contains major updates, including the migration of the Python wrapper to pybind11, native NumPy support, enhanced offset mapping, performance optimizations, security hardening, and internal code modernization.

1. Python Wrapper (pybind11 Migration & Features)

  • SWIG to pybind11 Migration: Fully migrated the Python wrapper from SWIG to pybind11, enabling better performance, safer memory management, and cleaner code integration (#1266, Commit 528dc9e).
  • Native NumPy Support:
    • direct encode/decoder from/to numpy object
  • Enhanced Offset Mapping:
    • Added new offset mapping features supporting both Unicode character offsets and raw byte offsets.
    • Added bounds checking to prevent OOB reads in offset mapping (Commit c5b0ce0).
    • Fixed crash issues and OOB spans in alignment conversions (#1231, #1211).
  • Type Stubs (PEP 561): Added inline type stubs (.pyi files) and packaged PEP 561 marker to provide auto-completion and static type safety in IDEs (#1258, Commit 8b7f97b).
  • Parallel Encoding API: Introduced parallel_encode and ThreadPool APIs, allowing Python users to tokenize single large documents across multiple CPU cores in parallel (#1254, Commit 3c37c39).
  • Direct Proto Loading: Added support for loading processors and normalizers directly from in-memory Proto instances (Commit 12788ba).
  • Concurrency Enhancements: Added GIL release during C++ execution to improve multi-threaded Python performance (#1251, Commit 0a044ae).
  • BOS/EOS Behavior Hardening: Aligned BOS/EOS adding behavior with C++ constraints, raising explicit ValueError instead of silent failures or crashes (Commit 780618a).

2. Performance Optimizations

  • BPE Training Optimization: Ported O(log n) lazy priority queue update optimization to BPE trainer to speed up training (Commit 8db9878).
  • Encode/Decode Optimizations: Decoding/Encoding optimization for integer/string input case by skipping SentencePieceText conversion, resulting in ~2x speedup (Commit 4b9231b).
  • Unigram Pruning: Improved unigram pruning using token-normalized frequency calculations (#1135, Commit 1cd0c59).

3. Security & Hardening

  • Trie Validation at Load: Added verification of darts-clone trie offsets when loading models to prevent segfaults on corrupted or malicious model files (#1224, Commit 6b2f05d).
  • Bounds Checking in Maps: Added bounds checks to piece accessors and validated value offsets in DecompileCharsMap (#1265, Commit 82b7c6d).
  • Crash Prevention: Replaced internal CHECK macros with DCHECK or returned errors to prevent host application crashes on invalid inputs (Commit 66b2587).
  • Null Character Disallowment: Explicitly disallow vocabulary pieces containing null characters (Commit eb3ba49).
  • NBest Timeout: Implemented a global timeout for NBest A* search to prevent hangs on pathological inputs (Commit ba57749).

4. C++ API & Internals Modernization

  • Abseil Migration: Extensively migrated from custom utilities to Abseil primitives (#1270
  • Normalizer Maps: Supported loading normalizer from raw maps in C++ (#1273, Commit e476e5e).

5. Build & Packaging

  • Deprecate Win32 Wheels: Stopped support and distribution of 32-bit Windows wheels (win32) (Commit 24fd00c).
  • Modernized Python builds: Switched from calling setup.py directly to the standard pypa/build PEP 517 frontend (#1167, Commit f300c7d).
  • PEP 639 License: Added PEP 639 license metadata to pyproject.toml (#1173).

6. Documentation

  • API Cheat Sheet: Added comparison tables, zero-copy, and sampling comparison guides.
  • Refactored Options: Fully reorganized options.md and special_symbols.md with verified code examples.
Commits
  • e0cce7d Merge pull request #1279 from google/dependabot/github_actions/github-actions...
  • a9e2b5e Bump the github-actions group with 2 updates
  • 20ab5fb Fix Unigram float score overflow and NaN/Inf corruption
  • 1f1483c Add use_reserved_id_map parameter and control token merge tests
  • 69d8562 Add performance benchmark suite and reports
  • 24fd00c Stop support for Win32 wheels
  • 4b9231b Optimize Decode/Encode paths using absl::Span and add benchmarks
  • b91b764 Merge pull request #1277 from google/new_doc
  • e93413b Update documentation: Modernize README and merge CLI/Build guides
  • 7ce7669 Merge pull request #1276 from kkew3/fix-dockerfile-20260620
  • Additional commits viewable in compare view

@dependabot dependabot Bot added dependencies Pull requests that update a dependency file python Pull requests that update python code labels Aug 16, 2026
Updates the requirements on [sentencepiece](https://github.com/google/sentencepiece) to permit the latest version.
- [Release notes](https://github.com/google/sentencepiece/releases)
- [Commits](google/sentencepiece@v0.1.99...v0.2.2)

---
updated-dependencies:
- dependency-name: sentencepiece
  dependency-version: 0.2.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot
dependabot Bot force-pushed the dependabot/pip/sentencepiece-gte-0.2.2 branch from a84a0d0 to c039ced Compare August 17, 2026 06:28
@ReSerendipity
ReSerendipity deleted the dependabot/pip/sentencepiece-gte-0.2.2 branch August 17, 2026 06:34
@dependabot @github

dependabot Bot commented on behalf of github Aug 17, 2026

Copy link
Copy Markdown
Author

OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting @dependabot ignore this major version or @dependabot ignore this minor version. You can also ignore all major, minor, or patch releases for a dependency by adding an ignore condition with the desired update_types to your config file.

If you change your mind, just re-open this PR and I'll resolve any conflicts on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant