Llama-3-Groq-8B-Tool-Use-GGUF Open-Source Model - Supports Tool Use and Function Call Features

Llama 3 Groq 8B Tool Use GGUF

Developed by tensorblock

Groq-optimized version based on Llama-3-8B, supporting tool usage and function call capabilities

Large Language Model English#Tool Call Optimization #Function Execution Support #Low-Resource Deployment

Downloads 125

Release Time : 11/29/2024

Model Overview

This is a Groq-optimized variant of the Llama-3-8B model, specifically enhanced for tool usage and function call capabilities, suitable for scenarios requiring interaction with external tools or APIs.

Model Features

Enhanced Tool Usage

Specifically optimized for interaction with external tools and APIs

Function Call Support

Capable of understanding and executing function call instructions

Multiple Quantization Versions

Provides 11 different quantization levels of GGUF format model files, from Q2_K to Q8_0

Model Capabilities

Text generation

Tool calling

Function execution

Context understanding

Use Cases

Automated Workflows

API Integration

Call external APIs through natural language instructions

Simplifies the automation of complex workflows

Developer Tools

Code Assistance

Generate code snippets and execute tests

Improves development efficiency

🚀 Groq/Llama-3-Groq-8B-Tool-Use - GGUF

This repository offers GGUF format model files for Groq/Llama-3-Groq-8B-Tool-Use. These files are quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4011.

Feedback and support: TensorBlock's Twitter/X, Telegram Group and Discord server

📚 Documentation

Our projects

Project	Description
Awesome MCP Servers	A comprehensive collection of Model Context Protocol (MCP) servers. 👉 See what we built 👈
TensorBlock Studio	A lightweight, open, and extensible multi-LLM interaction studio. 👉 See what we built 👈

Prompt template

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>

{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Model file specification

Property	Details
Model Type	Groq/Llama-3-Groq-8B-Tool-Use - GGUF
Training Data	Not provided

Filename	Quant type	File Size	Description
Llama-3-Groq-8B-Tool-Use-Q2_K.gguf	Q2_K	3.179 GB	smallest, significant quality loss - not recommended for most purposes
Llama-3-Groq-8B-Tool-Use-Q3_K_S.gguf	Q3_K_S	3.665 GB	very small, high quality loss
Llama-3-Groq-8B-Tool-Use-Q3_K_M.gguf	Q3_K_M	4.019 GB	very small, high quality loss
Llama-3-Groq-8B-Tool-Use-Q3_K_L.gguf	Q3_K_L	4.322 GB	small, substantial quality loss
Llama-3-Groq-8B-Tool-Use-Q4_0.gguf	Q4_0	4.661 GB	legacy; small, very high quality loss - prefer using Q3_K_M
Llama-3-Groq-8B-Tool-Use-Q4_K_S.gguf	Q4_K_S	4.693 GB	small, greater quality loss
Llama-3-Groq-8B-Tool-Use-Q4_K_M.gguf	Q4_K_M	4.921 GB	medium, balanced quality - recommended
Llama-3-Groq-8B-Tool-Use-Q5_0.gguf	Q5_0	5.599 GB	legacy; medium, balanced quality - prefer using Q4_K_M
Llama-3-Groq-8B-Tool-Use-Q5_K_S.gguf	Q5_K_S	5.599 GB	large, low quality loss - recommended
Llama-3-Groq-8B-Tool-Use-Q5_K_M.gguf	Q5_K_M	5.733 GB	large, very low quality loss - recommended
Llama-3-Groq-8B-Tool-Use-Q6_K.gguf	Q6_K	6.596 GB	very large, extremely low quality loss
Llama-3-Groq-8B-Tool-Use-Q8_0.gguf	Q8_0	8.541 GB	very large, extremely low quality loss - not recommended

📦 Installation

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, download the individual model file to a local directory

huggingface-cli download tensorblock/Llama-3-Groq-8B-Tool-Use-GGUF --include "Llama-3-Groq-8B-Tool-Use-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you want to download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/Llama-3-Groq-8B-Tool-Use-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

📄 License

The model is under the llama3 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご