🚀 TinyAgent: Function Calling at the Edge
TinyAgent is designed to empower Small Language Models (SLMs) with complex reasoning and function calling capabilities. These SLMs can be securely and privately deployed at the edge. In contrast, traditional Large Language Models (LLMs) like GPT - 4 and Gemini - 1.5, despite their power, are often too large and resource - intensive for edge deployment, bringing challenges in privacy, connectivity, and latency. TinyAgent overcomes these issues by training specialized SLMs with high - quality, curated data and focusing on function calling with LLMCompiler. As a practical application, TinyAgent can interact with various MacOS applications to assist users with daily tasks such as writing emails, managing contacts, scheduling calendar events, and organizing Zoom meetings.
Get the desktop app
|
Read the blog post

✨ Features
When dealing with difficult tasks, SLM agents need appropriate tools and in - context examples. Irrelevant examples can lead to hallucination, and descriptions of unnecessary tools can confuse the model and waste prompt space. To solve this, TinyAgent uses ToolRAG to retrieve the best tools and examples for a given query. This process has minimal latency and significantly improves the accuracy of TinyAgent. For more details, please refer to our blog post.
Model Developers: Squeeze AI Lab at University of California, Berkeley.
Variations: There are 2 sizes of TinyAgent models: TinyAgent - 1.1B and TinyAgent - 7B
License: MIT
📚 Documentation
Demo
How to Use
For details on using TinyAgent models, please refer to our Github. TinyAgent models can be used programmatically or through our user interface.
Training Details
Dataset
We curated a dataset of 40,000 real - life use cases. We used GPT - 3.5 - Turbo to generate real - world instructions and then obtained synthetic execution plans using GPT - 4 - Turbo. For more details about our dataset, please check our blog post.
Fine - tuning Procedure
TinyAgent models are fine - tuned from base models. The following table shows each TinyAgent model and its base counterpart:
Model |
Success Rate |
GPT - 3.5 - turbo |
65.04% |
GPT - 4 - turbo |
79.08% |
[TinyLLama - 1.1B - 32K - Instruct](https://huggingface.co/Doctor - Shotgun/TinyLlama - 1.1B - 32k - Instruct) |
12.71% |
[WizardLM - 2 - 7b](https://huggingface.co/MaziyarPanahi/WizardLM - 2 - 7B - GGUF) |
41.25% |
TinyAgent - 1.1B + ToolRAG / [[hf](https://huggingface.co/squeeze - ai - lab/TinyAgent - 1.1B)] [[gguf](https://huggingface.co/squeeze - ai - lab/TinyAgent - 1.1B - GGUF)] |
80.06% |
TinyAgent - 7B + ToolRAG / [[hf](https://huggingface.co/squeeze - ai - lab/TinyAgent - 7B)] [[gguf](https://huggingface.co/squeeze - ai - lab/TinyAgent - 7B - GGUF)] |
84.95% |
Using the synthetic data generation process described above, we used parameter - efficient fine - tuning with LoRA to fine - tune the base models for 3 epochs. For more details on our fine - tuning procedure, please check our [blog post](https://bair.berkeley.edu/blog/2024/05/29/tiny - agent/).
Links
Blog Post: https://bair.berkeley.edu/blog/2024/05/29/tiny - agent/
Github: https://github.com/SqueezeAILab/TinyAgent