(( QTRL Catalog ))
Collection
32 items
•
Updated
mindstate
mindstate
AI & ML interests
None yet
Recent Activity
updated
a collection
1 day ago
(( QTRL Catalog ))
updated
a collection
1 day ago
(( QTRL Catalog ))
liked
a model
2 days ago
Overworld/Waypoint-1-Small
Organizations
None yet
updated
a
collection
1 day ago
liked
a
model
2 days ago
updated
a
collection
2 days ago
reacted to
codelion's
post with 🔥
2 days ago
Post
2942
Reverse Engineering a $500M Mystery: From HashHop to Memory-Augmented Language Models
I wrote a deep dive into how Magic AI's 100M token context window might work, starting from their HashHop benchmark and building up to MALM - a Memory-Augmented Language Model.
Key insight: treating each key as a single token enables perfect retrieval at unlimited context lengths.
The article covers:
- How HashHop works and why its perfect accuracy is suspicious
- Building a tokenized solver that achieves 100% accuracy
- Scaling to MALM for real code search tasks
- Why this approach could handle 100M+ tokens
Read the full article: https://huggingface.co/blog/codelion/reverse-engineering-magic-hashhop
Try the model: codelion/malm-165m
Code: https://github.com/codelion/hash-hop
I wrote a deep dive into how Magic AI's 100M token context window might work, starting from their HashHop benchmark and building up to MALM - a Memory-Augmented Language Model.
Key insight: treating each key as a single token enables perfect retrieval at unlimited context lengths.
The article covers:
- How HashHop works and why its perfect accuracy is suspicious
- Building a tokenized solver that achieves 100% accuracy
- Scaling to MALM for real code search tasks
- Why this approach could handle 100M+ tokens
Read the full article: https://huggingface.co/blog/codelion/reverse-engineering-magic-hashhop
Try the model: codelion/malm-165m
Code: https://github.com/codelion/hash-hop
- 1 reply
updated
4
collections
2 days ago
updated
2
collections
3 days ago
liked
a
model
3 days ago
updated
a
collection
5 days ago