Meltemi 7B Instruct V1.5 GGUF

🚀 ilsp/Meltemi-7B-Instruct-v1.5 - GGUF

This repository provides GGUF format model files for ilsp/Meltemi-7B-Instruct-v1.5. These files are quantized using machines from TensorBlock, and they are compatible with llama.cpp as of commit b4011.

✨ Features

• Model Format: The model files are in GGUF format, which is compatible with llama.cpp.
• Quantization: Quantized using machines provided by TensorBlock.
• Compatibility: Compatible with llama.cpp as of commit b4011.

📦 Installation

Command line

1. Install Huggingface Client:

pip install -U "huggingface_hub[cli]"

2. Download the individual model file to a local directory:

huggingface-cli download tensorblock/Meltemi-7B-Instruct-v1.5-GGUF --include "Meltemi-7B-Instruct-v1.5-Q2_K.gguf" --local-dir MY_LOCAL_DIR

3. If you want to download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/Meltemi-7B-Instruct-v1.5-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

💻 Usage Examples

Prompt template

<|system|>
{system_prompt}</s>
<|user|>
{prompt}</s>
<|assistant|>

📚 Documentation

Our projects

Project	Description	Image	Link
Awesome MCP Servers	A comprehensive collection of Model Context Protocol (MCP) servers.		👉 See what we built 👈
TensorBlock Studio	A lightweight, open, and extensible multi-LLM interaction studio.		👉 See what we built 👈

Model file specification

📄 License

The model is licensed under the Apache-2.0 license.