AI & ML interests

None defined yet.

Recent Activity

csabakecskemetiΒ 
posted an update 12 days ago
view post
Post
3142
Just sharing a result of a homelab infrastructure experiment:

I've managed to setup a distributed inference infra at home using a DGX Spark (128GB unified gddr6) and a linux workstation with an RTX 6000 Pro (96GB gddr7) connected via 100Gbps RoCEv2. The model I've used (https://lnkd.in/gx6J7YuB) is about 140GB so could not fit either of the GPU. Full setup and tutorial soon on devquasar.com



Screen recording:
https://lnkd.in/gKM9H5GJ
Β·
csabakecskemetiΒ 
posted an update about 1 month ago
csabakecskemetiΒ 
posted an update about 1 month ago
view post
Post
2075
Looking for some help to test an INT8 Deepseek 3.2:
SGLang supports Channel wise INT8 quants on CPUs with AMX instructions (Xeon 5 and above AFAIK)
https://lmsys.org/blog/2025-07-14-intel-xeon-optimization/

Currently uploading an INT8 version of Deepseek 3.2 Speciale:
DevQuasar/deepseek-ai.DeepSeek-V3.2-Speciale-Channel-INT8

I cannot test this I'm on AMD
"AssertionError: W8A8Int8LinearMethod on CPU requires that CPU has AMX support"
(I assumed it can fall back to some non optimized kernel but seems not)

If anyone with the required resources (Intel Xeon 5/6 + ~768-1TB ram) can help to test this that would be awesome.

If you have hints how to make this work on AMD Threadripper 7000 Pro series please guide me.

Thanks all!
Β·
csabakecskemetiΒ 
posted an update about 2 months ago
view post
Post
304
Recently there are so much activity on token efficient formats, I've also build a package (inspired by toon).

Deep-TOON

My goal was to token efficiently handle json structures with complex embeddings.

So this is what I've built on the weekend. Feel free try:

https://pypi.org/project/deep-toon/0.1.0/

JingzeShiΒ 
posted an update about 2 months ago
csabakecskemetiΒ 
posted an update 3 months ago
view post
Post
2624
Christmas came early this year
Β·
lysandreΒ 
posted an update 4 months ago
view post
Post
7469
We're kick-starting the process of Transformers v5, with @ArthurZ and @cyrilvallez !

v5 should be significant: we're using it as a milestone for performance optimizations, saner defaults, and a much cleaner code base worthy of 2025.

Fun fact: v4.0.0-rc-1 came out on Nov 19, 2020, nearly five years ago!
  • 6 replies
Β·
jeffboudierΒ 
posted an update 4 months ago
view post
Post
3106
Quick 30s demo of the new Hub > Azure AI integration to deploy HF models in your own Azure account. Now with Py and CLI!

GG @alvarobartt @kramp @pagezyhf
XenovaΒ 
posted an update 5 months ago
view post
Post
14279
Okay this is insane... WebGPU-accelerated semantic video tracking, powered by DINOv3 and Transformers.js! 🀯
Demo (+ source code): webml-community/DINOv3-video-tracking

This will revolutionize AI-powered video editors... which can now run 100% locally in your browser, no server inference required (costs $0)! 😍

How does it work? πŸ€”
1️⃣ Generate and cache image features for each frame
2️⃣ Create a list of embeddings for selected patch(es)
3️⃣ Compute cosine similarity between each patch and the selected patch(es)
4️⃣ Highlight those whose score is above some threshold

... et voilΓ ! πŸ₯³

You can also make selections across frames to improve temporal consistency! This is super useful if the object changes its appearance slightly throughout the video.

Excited to see what the community builds with it!
Β·
XenovaΒ 
posted an update 5 months ago
view post
Post
4632
The next generation of AI-powered websites is going to be WILD! 🀯

In-browser tool calling & MCP is finally here, allowing LLMs to interact with websites programmatically.

To show what's possible, I built a demo using Liquid AI's new LFM2 model, powered by πŸ€— Transformers.js: LiquidAI/LFM2-WebGPU

As always, the demo is open source (which you can find under the "Files" tab), so I'm excited to see how the community builds upon this! πŸš€
  • 2 replies
Β·
JingzeShiΒ 
posted an update 5 months ago
XenovaΒ 
posted an update 6 months ago
view post
Post
3491
Introducing Voxtral WebGPU: State-of-the-art audio transcription directly in your browser! 🀯
πŸ—£οΈ Transcribe videos, meeting notes, songs and more
πŸ” Runs on-device, meaning no data is sent to a server
🌎 Multilingual (8 languages)
πŸ€— Completely free (forever) & open source

That's right, we're running Mistral's new Voxtral-Mini-3B model 100% locally in-browser on WebGPU, powered by Transformers.js and ONNX Runtime Web! πŸ”₯

Try it out yourself! πŸ‘‡
webml-community/Voxtral-WebGPU