TinyAgent-1.1B Open-Source Language Model - Suitable for Edge Devices, Fast Function Call Inference and Strong Privacy

Tinyagent 1.1B

Developed by squeeze-ai-lab

TinyAgent is a small language model designed for edge devices, focusing on function calls and complex reasoning tasks, providing privacy protection and low-latency responses.

Large Language Model

Transformers

English#Edge computing function calls #Low-resource SLM inference #On-device privacy protection

Downloads 128

Release Time : 5/27/2024

Model Overview

TinyAgent aims to provide complex reasoning and function calling capabilities for small language models (SLMs) that can be deployed securely and privately on edge devices. It addresses the challenges of privacy, connectivity, and latency faced by traditional large language models.

Model Features

Edge computing optimization

Designed specifically for edge devices, with a small footprint and low resource requirements, suitable for privacy-sensitive and low-latency scenarios

Function calling capability

Utilizes LLMCompiler to achieve powerful function calling capabilities, enabling interaction with MacOS applications to complete real-world tasks

ToolRAG enhancement

Significantly improves model accuracy and response quality by retrieving the most suitable tools and examples through ToolRAG

High success rate

Achieves an 80.06% success rate on function calling tasks, approaching the level of GPT-4-turbo

Model Capabilities

Function calling

Task planning

Edge device inference

Application interaction

Privacy-preserving computation

Use Cases

Office automation

Email composition

Assists users in composing and sending emails

Meeting scheduling

Manages calendar events and organizes Zoom meetings

Personal information management

Contact management

Helps users organize and update contact information

🚀 TinyAgent: Function Calling at the Edge

TinyAgent aims to empower Small Language Models (SLMs) with complex reasoning and function calling capabilities. These SLMs can be securely and privately deployed at the edge. Traditional Large Language Models (LLMs) like GPT - 4 and Gemini - 1.5, though powerful, are often too large and resource - intensive for edge deployment, bringing challenges in privacy, connectivity, and latency. TinyAgent overcomes these issues by training specialized SLMs with high - quality, curated data and focusing on function calling with LLMCompiler. As a practical application, TinyAgent can interact with various MacOS applications to assist users with daily tasks such as email composition, contact management, calendar event scheduling, and Zoom meeting organization.

Get the desktop app‎ ‎ | Read the blog post

Thumbnail

Model Developers: Squeeze AI Lab at University of California, Berkeley.

Variations: TinyAgent models come in 2 sizes: TinyAgent - 1.1B and TinyAgent - 7B

License: MIT

🚀 Quick Start

Please see our Github for details on how to use TinyAgent models. TinyAgent models can be used programmatically or through our user interface.

✨ Features

Edge - friendly: Enables complex reasoning and function calling in Small Language Models (SLMs) for secure and private edge deployment.
Function calling: Focuses on function calling with LLMCompiler.
MacOS integration: Can interact with various MacOS applications to assist with daily tasks.

📚 Documentation

Demo

Training Details

Dataset

We curated a dataset of 40,000 real - life use cases. We use GPT - 3.5 - Turbo to generate real - world instructions. These are then used to obtain synthetic execution plans using GPT - 4 - Turbo. Please check out our blog post for more details on our dataset.

Fine - tuning Procedure

TinyAgent models are fine - tuned from base models. Below is a table of each TinyAgent model with its base counterpart:

Model	Success Rate
GPT - 3.5 - turbo	65.04%
GPT - 4 - turbo	79.08%
[TinyLLama - 1.1B - 32K - Instruct](https://huggingface.co/Doctor - Shotgun/TinyLlama - 1.1B - 32k - Instruct)	12.71%
[WizardLM - 2 - 7b](https://huggingface.co/MaziyarPanahi/WizardLM - 2 - 7B - GGUF)	41.25%
TinyAgent - 1.1B + ToolRAG / [[hf](https://huggingface.co/squeeze - ai - lab/TinyAgent - 1.1B)] [[gguf](https://huggingface.co/squeeze - ai - lab/TinyAgent - 1.1B - GGUF)]	80.06%
TinyAgent - 7B + ToolRAG / [[hf](https://huggingface.co/squeeze - ai - lab/TinyAgent - 7B)] [[gguf](https://huggingface.co/squeeze - ai - lab/TinyAgent - 7B - GGUF)]	84.95%

Using the synthetic data generation process described above, we use parameter - efficient fine - tuning with LoRA to fine - tune the base models for 3 epochs. Please check out our [blog post](https://bair.berkeley.edu/blog/2024/05/29/tiny - agent/) for more details on our fine - tuning procedure.

🛠️ ToolRAG

When faced with challenging tasks, SLM agents require appropriate tools and in - context examples to guide them. If the model sees irrelevant examples, it can hallucinate. Likewise, if the model sees the descriptions of the tools that it doesn’t need, it usually gets confused, and these tools take up unnecessary prompt space. To tackle this, TinyAgent uses ToolRAG to retrieve the best tools and examples suited for a given query. This process has minimal latency and increases the accuracy of TinyAgent substantially. Please take a look at our [blog post](https://bair.berkeley.edu/blog/2024/05/29/tiny - agent/) and our [ToolRAG model](https://huggingface.co/squeeze - ai - lab/TinyAgent - ToolRAG) for more details.

🔗 Links

Blog Post: https://bair.berkeley.edu/blog/2024/05/29/tiny - agent/
Github: https://github.com/SqueezeAILab/TinyAgent

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご