tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Recent spike this week

7d +0.33% +37 stars in 7d
30d Not tracked yet — first tracked 2026-09-13
90d Not tracked yet — first tracked 2026-09-13
Sep 13, 2026: 11,034 starsSep 14, 2026: 11,038 starsSep 15, 2026: 11,036 starsSep 16, 2026: 11,039 starsSep 17, 2026: 11,040 starsSep 18, 2026: 11,047 starsSep 19, 2026: 11,047 starsSep 20, 2026: 11,050 starsSep 21, 2026: 11,052 starsSep 22, 2026: 11,068 starsSep 23, 2026: 11,100 starsSep 24, 2026: 11,108 starsSep 25, 2026: 11,112 starsSep 26, 2026: 11,117 starsSep 27, 2026: 11,123 starsSep 28, 2026: 11,134 starsSep 29, 2026: 11,138 starsSep 30, 2026: 11,142 starsOct 1, 2026: 11,145 stars 11,03411,09011,145 Sep 13Oct 1

11,034 → 11,145 stars since Sep 13

11,030 Stars 1,211 Forks 210 Open issues

Timeline

💬 1 Hacker News discussion

View on GitHub ↗ Visit site ↗
bertgptlanguage-modelnatural-language-processingnatural-language-understandingnlptransformers