Mt0 Xxl Mt Q4 K M GGUF

🚀 Markobes/mt0-xxl-mt-Q4_K_M-GGUF

This model is a conversion to the GGUF format from the original model bigscience/mt0-xxl-mt. The conversion was carried out using llama.cpp via the ggml.ai's GGUF-my-repo space. For more in - depth details about the model, please refer to the original model card.

🚀 Quick Start

Use with llama.cpp

You can install llama.cpp using brew, which is applicable to both Mac and Linux systems.

brew install llama.cpp

After installation, you can either use the llama.cpp server or the CLI.

CLI

llama-cli --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -p "The meaning to life and the universe is"

Server

llama-server --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -c 2048

Note: You can also follow the usage steps in the Llama.cpp repository to use this checkpoint directly.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Navigate to the llama.cpp folder and build it with the LLAMA_CURL = 1 flag, along with other hardware - specific flags (e.g., LLAMA_CUDA = 1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference using the main binary.

./llama-cli --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -p "The meaning to life and the universe is"

Or

./llama-server --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -c 2048

✨ Features

• Multi - language support: The model supports a wide range of languages, including but not limited to Afrikaans (af), Amharic (am), Arabic (ar), and many others.
• Text - to - text generation: It is designed for text - to - text generation tasks, such as translation, question - answering, and more.

📦 Installation

Install llama.cpp

brew install llama.cpp

💻 Usage Examples

Basic Usage

CLI

llama-cli --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -p "The meaning to life and the universe is"

Server

llama-server --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -c 2048

Advanced Usage

If you want to use hardware - specific optimizations, for example, using Nvidia GPUs on Linux, you can build llama.cpp with the LLAMA_CUDA = 1 flag.

cd llama.cpp && LLAMA_CUDA=1 LLAMA_CURL=1 make

Then run inference:

./llama-cli --hf-repo Markobes/mt0-xxl-mt-Q4_K_M-GGUF --hf-file mt0-xxl-mt-q4_k_m.gguf -p "The meaning to life and the universe is"

📚 Documentation

Datasets

Languages

The model supports the following languages: af, am, ar, az, be, bg, bn, ca, ceb, co, cs, cy, da, de, el, en, eo, es, et, eu, fa, fi, fil, fr, fy, ga, gd, gl, gu, ha, haw, hi, hmn, ht, hu, hy, ig, is, it, iw, ja, jv, ka, kk, km, kn, ko, ku, ky, la, lb, lo, lt, lv, mg, mi, mk, ml, mn, mr, ms, mt, my, ne, nl, no, ny, pa, pl, ps, pt, ro, ru, sd, si, sk, sl, sm, sn, so, sq, sr, st, su, sv, sw, ta, te, tg, th, tr, uk, und, ur, uz, vi, xh, yi, yo, zh, zu

Tags

• text2text-generation
• llama-cpp
• gguf-my-repo

Widget Examples

Model Index

The model mt0-xxl-mt has the following results on different tasks and datasets:

📄 License

The model is released under the apache - 2.0 license.