Tinyagent ToolRAG

🚀 TinyAgent: Function Calling at the Edge

TinyAgent is designed to empower Small Language Models (SLMs) with complex reasoning and function calling capabilities. These SLMs can be securely and privately deployed at the edge. In contrast, traditional Large Language Models (LLMs) like GPT - 4 and Gemini - 1.5, despite their power, are often too large and resource - intensive for edge deployment, bringing challenges in privacy, connectivity, and latency. TinyAgent overcomes these issues by training specialized SLMs with high - quality, curated data and focusing on function calling with LLMCompiler. As a practical application, TinyAgent can interact with various MacOS applications to assist users with daily tasks such as writing emails, managing contacts, scheduling calendar events, and organizing Zoom meetings.

Get the desktop app‎ ‎ | Read the blog post

✨ Features

When dealing with difficult tasks, SLM agents need appropriate tools and in - context examples. Irrelevant examples can lead to hallucination, and descriptions of unnecessary tools can confuse the model and waste prompt space. To solve this, TinyAgent uses ToolRAG to retrieve the best tools and examples for a given query. This process has minimal latency and significantly improves the accuracy of TinyAgent. For more details, please refer to our blog post.

Model Developers: Squeeze AI Lab at University of California, Berkeley.

Variations: There are 2 sizes of TinyAgent models: TinyAgent - 1.1B and TinyAgent - 7B

License: MIT

📚 Documentation

Demo

How to Use

For details on using TinyAgent models, please refer to our Github. TinyAgent models can be used programmatically or through our user interface.

Training Details

Dataset

We curated a dataset of 40,000 real - life use cases. We used GPT - 3.5 - Turbo to generate real - world instructions and then obtained synthetic execution plans using GPT - 4 - Turbo. For more details about our dataset, please check our blog post.

Fine - tuning Procedure

TinyAgent models are fine - tuned from base models. The following table shows each TinyAgent model and its base counterpart:

Using the synthetic data generation process described above, we used parameter - efficient fine - tuning with LoRA to fine - tune the base models for 3 epochs. For more details on our fine - tuning procedure, please check our [blog post](https://bair.berkeley.edu/blog/2024/05/29/tiny - agent/).

Links

Blog Post: https://bair.berkeley.edu/blog/2024/05/29/tiny - agent/

Github: https://github.com/SqueezeAILab/TinyAgent