SimpleTool: Parallel Decoding for Real-Time LLM Function Calling

Hugging Face | ModelScope | GitHub

This repository contains the weights for RT-Qwen (RealtimeTool), a series of models optimized for low-latency, parallel LLM function calling.

📁 Model Directory Structure

The models are organized by scale, quantization format, and inference framework.

1. SFT & AWQ Models (vLLM / Transformers)

Directly use these folders for inference via vLLM or Transformers.

• RT-Qwen2.5-0.5B / -0.5B-AWQ
• RT-Qwen2.5-1.5B / -1.5B-AWQ
• RT-Qwen2.5-3B / -3B-AWQ
• RT-Qwen2.5-7B / -7B-AWQ
• RT-Qwen2.5-14B / -14B-AWQ
• RT-Qwen3-4B / -4B-AWQ
• RT-Qwen3-30B / -30B-AWQ

2. GGUF Models (llama.cpp)

• gguf_models/: Full-precision (F16) GGUF files for all versions.
• gguf_quantized/: Quantized GGUF versions including Q4_K_M, Q5_K_M, and Q8_0.

---

📝 TODO

• Release Arxiv Paper
• Complete GitHub Documentation
• Add Performance Benchmarks
• Provide Citation Info

---

License: Apache-2.0
Status: Models Uploading / Placeholder README