Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
Edit Models filters
Main
Tasks
Libraries
Languages
Licenses
Other
1
Apps
llama.cpp
LM Studio
Jan
Draw Things
DiffusionBee
Jellybox
JoyFusion
LocalAI
vLLM
Ollama
TGI
MLX LM
Docker Model Runner
Lemonade
Inference Providers
Select all
Groq
Novita
Cerebras
SambaNova
Nscale
fal
Hyperbolic
Together AI
Fireworks
Featherless AI
Zai
Replicate
Cohere
Scaleway
Public AI
OVHcloud AI Endpoints
HF Inference API
Nebius AI
WaveSpeed
Misc
Reset Misc
tokenization
Inference Endpoints
text-generation-inference
Eval Results (legacy)
Merge
4-bit precision
custom_code
8-bit precision
text-embeddings-inference
Mixture of Experts
Carbon Emissions
Eval Results
Apply filters
Models
360
Full-text search
Inference Available
Edit filters
Sort: Trending
Active filters:
tokenization
Clear all
wikilangs/ba
Text Generation
•
Updated
15 days ago
•
1
wikilangs/dtp
Text Generation
•
Updated
15 days ago
•
1
aymaralima/deeplima
Updated
May 5, 2024
•
3
gsar78/Greek_Tokenizer
Updated
Aug 5, 2024
•
1
microsoft/VidTok
Updated
Apr 5, 2025
•
42
Emova-ollm/emova_speech_tokenizer_hf
Feature Extraction
•
0.1B
•
Updated
Oct 8, 2025
•
229
•
2
metinovadilet/kyrgyz_t5_tokenizer
Updated
Feb 24, 2025
microsoft/vidtwin
Updated
Mar 20, 2025
•
2
JephteyAdolphe/ht_core_news_sm
Updated
May 1, 2025
Hanbaike/kyrgyz_bpe_16000
Updated
May 8, 2025
Hanbaike/kyrgyz_bpe_24000
Updated
May 8, 2025
Hanbaike/kyrgyz_bpe_32000
Updated
May 8, 2025
Hanbaike/kyrgyz_bpe_4000
Updated
May 8, 2025
Hanbaike/kyrgyz_bpe_51200
Updated
May 8, 2025
Hanbaike/kyrgyz_bpe_8000
Updated
May 8, 2025
Hanbaike/kyrgyz_unigram_16000
Updated
May 8, 2025
Hanbaike/kyrgyz_unigram_24000
Updated
May 8, 2025
Hanbaike/kyrgyz_unigram_32000
Updated
May 8, 2025
Hanbaike/kyrgyz_unigram_4000
Updated
May 8, 2025
Hanbaike/kyrgyz_unigram_51200
Updated
May 8, 2025
Hanbaike/kyrgyz_unigram_8000
Updated
May 8, 2025
Hanbaike/kyrgyz_spm_tokenizer
Updated
May 8, 2025
Hengzongshu/Chinse_BBPE_Vocab
Text Classification
•
Updated
Jun 1, 2025
khopilot/khmer-tokenizer-v7
Feature Extraction
•
Updated
Oct 7, 2025
•
2
toksuite/google-gemma-2-2b
Text Generation
•
2B
•
Updated
25 days ago
•
115
toksuite/common-pile-comma-v0.1
Text Generation
•
2B
•
Updated
25 days ago
•
116
toksuite/meta-llama-Llama-3.2-1B
Text Generation
•
2B
•
Updated
25 days ago
•
141
toksuite/microsoft-Phi-3-mini-4k-instruct
Text Generation
•
1B
•
Updated
25 days ago
•
95
toksuite/gpt2
Text Generation
•
1B
•
Updated
25 days ago
•
114
toksuite/bigscience-bloom
Text Generation
•
2B
•
Updated
25 days ago
•
110
Previous
1
2
3
...
12
Next