Llama-3.2-3B-Instruct-GGUF Open-Source Multilingual Large Model - Free Deployment for Efficient Conversational Communication

Llama 3.2 3B Instruct GGUF

Developed by unsloth

Llama-3.2-3B is a multilingual large language model released by Meta, featuring a 3B-parameter pre-trained and instruction-tuned generative model optimized for multilingual dialogue use cases.

Large Language Model English#Multilingual Dialogue Optimization #Agent Retrieval Enhancement #Low-Memory Fine-Tuning

Downloads 40.11k

Release Time : 9/25/2024

Model Overview

Llama-3.2-3B is a multilingual large language model optimized for multilingual dialogue use cases, including agent retrieval and summarization tasks. It outperforms many open-source and closed-source chat models on common industry benchmarks.

Model Features

Multilingual Support

Supports multiple languages including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.

Optimized Transformer Architecture

Utilizes an optimized transformer architecture with Grouped Query Attention (GQA) to enhance inference scalability.

Instruction Tuning

Employs Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) to align with human preferences, ensuring helpfulness and safety.

High Performance

Outperforms many open-source and closed-source chat models on common industry benchmarks.

Model Capabilities

Text Generation

Multilingual Dialogue

Agent Retrieval

Summarization Tasks

Use Cases

Dialogue Systems

Multilingual Chatbot

Used to build chatbots that support multiple languages.

Outperforms many open-source and closed-source chat models on common industry benchmarks.

Information Retrieval

Agent Retrieval

Used for information retrieval and agent tasks.

Text Processing

Text Summarization

Used to generate text summaries.

🚀 Llama-3.2-3B

Explore all versions of Llama 3.2, including GGUF, 4-bit, and original 16-bit formats, in our collection. Unsloth enables you to finetune Llama 3.2, Gemma 2, and Mistral 2 up to 5x faster with 70% less memory!

🚀 Quick Start

For more details on the model, please go to Meta's original model card.

✨ Features

GGUF uploads

16bit, 8bit, 6bit, 5bit, 4bit, 3bit, and 2bit uploads are available.

Finetune Llama 3.2, Gemma 2, Mistral 2-5x faster with 70% less memory via Unsloth!

We have a free Google Colab Tesla T4 notebook for Llama 3.2 (3B) here: Google Colab Notebook

✨ Finetune for Free

All notebooks are beginner friendly! Add your dataset, click "Run All", and you'll get a 2x faster finetuned model which can be exported to GGUF, vLLM or uploaded to Hugging Face.

Unsloth supports	Free Notebooks	Performance	Memory use
Llama-3.2 (3B)	▶️ Start on Colab	2.4x faster	58% less
Llama-3.1 (11B vision)	▶️ Start on Colab	2.4x faster	58% less
Llama-3.1 (8B)	▶️ Start on Colab	2.4x faster	58% less
Phi-3.5 (mini)	▶️ Start on Colab	2x faster	50% less
Gemma 2 (9B)	▶️ Start on Colab	2.4x faster	58% less
Mistral (7B)	▶️ Start on Colab	2.2x faster	62% less
DPO - Zephyr	▶️ Start on Colab	1.9x faster	19% less

This conversational notebook is useful for ShareGPT ChatML / Vicuna templates.
This text completion notebook is for raw text. This DPO notebook replicates Zephyr.
* Kaggle has 2x T4s, but we use 1. Due to overhead, 1x T4 is 5x faster.

Special Thanks

A huge thank you to the Meta and Llama team for creating and releasing these models.

📚 Documentation

Model Information

The Meta Llama 3.2 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction-tuned generative models in 1B and 3B sizes (text in/text out). The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. They outperform many of the available open source and closed chat models on common industry benchmarks.

Property	Details
Model developer	Meta
Model Architecture	Llama 3.2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.
Supported languages	English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai are officially supported. Llama 3.2 has been trained on a broader collection of languages than these 8 supported languages. Developers may fine-tune Llama 3.2 models for languages beyond these supported languages, provided they comply with the Llama 3.2 Community License and the Acceptable Use Policy. Developers are always expected to ensure that their deployments, including those that involve additional languages, are completed safely and responsibly.
Llama 3.2 family of models	Token counts refer to pretraining data only. All model versions use Grouped-Query Attention (GQA) for improved inference scalability.
Model Release Date	Sept 25, 2024
Status	This is a static model trained on an offline dataset. Future versions may be released that improve model capabilities and safety.
License	Use of Llama 3.2 is governed by the Llama 3.2 Community License (a custom, commercial license agreement).

Where to send questions or comments about the model Instructions on how to provide feedback or comments on the model can be found in the model README. For more technical information about generation parameters and recipes for how to use Llama 3.1 in applications, please go here.

📄 License

Use of Llama 3.2 is governed by the Llama 3.2 Community License (a custom, commercial license agreement).

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご