Tinyagent 7B

🚀 TinyAgent: Function Calling at the Edge

TinyAgent is designed to empower Small Language Models (SLMs) with complex reasoning and function - calling capabilities. These SLMs can be securely and privately deployed at the edge, addressing the limitations of traditional Large Language Models (LLMs) such as GPT - 4 and Gemini - 1.5, which are often too large and resource - intensive for edge deployment.

Get the desktop app‎ ‎ | Read the blog post

✨ Features

• Edge Deployment: Enables complex reasoning and function calling in SLMs for secure and private edge deployment.
• Function Calling Focus: Trains SLMs with curated data and focuses on function calling using LLMCompiler.
• MacOS Interaction: Can interact with various MacOS applications to assist users with daily tasks like email composition, contact management, calendar scheduling, and Zoom meeting organization.

Model Developers: Squeeze AI Lab at University of California, Berkeley.

Variations: TinyAgent models come in 2 sizes: TinyAgent - 1.1B and TinyAgent - 7B

License: MIT

🎥 Demo

📚 Documentation

💻 Usage Examples

Please see our Github for details on how to use TinyAgent models. TinyAgent models can be used programmatically or through our user interface.

🔧 Technical Details

Dataset

We curated a dataset of 40,000 real - life use cases. We use GPT - 3.5 - Turbo to generate real - world instructions. These are then used to obtain synthetic execution plans using GPT - 4 - Turbo. Please check out our blog post for more details on our dataset.

Fine - tuning Procedure

TinyAgent models are fine - tuned from base models. Below is a table of each TinyAgent model with its base counterpart:

Using the synthetic data generation process described above, we use parameter - efficient fine - tuning with LoRA to fine - tune the base models for 3 epochs. Please check out our [blog post](https://bair.berkeley.edu/blog/2024/05/29/tiny - agent/) for more details on our fine - tuning procedure.

🛠️ ToolRAG

When faced with challenging tasks, SLM agents require appropriate tools and in - context examples to guide them. If the model sees irrelevant examples, it can hallucinate. Likewise, if the model sees the descriptions of the tools that it doesn’t need, it usually gets confused, and these tools take up unnecessary prompt space. To tackle this, TinyAgent uses ToolRAG to retrieve the best tools and examples suited for a given query. This process has minimal latency and increases the accuracy of TinyAgent substantially. Please take a look at our [blog post](https://bair.berkeley.edu/blog/2024/05/29/tiny - agent/) and our [ToolRAG model](https://huggingface.co/squeeze - ai - lab/TinyAgent - ToolRAG) for more details.

🔗 Links

• Blog Post: https://bair.berkeley.edu/blog/2024/05/29/tiny - agent/
• Github: https://github.com/SqueezeAILab/TinyAgent

📄 License

This project is licensed under the MIT license.