Flan T5 Base Q4 K M GGUF

🚀 marroyo777/flan-t5-base-Q4_K_M-GGUF

This project provides a model converted to the GGUF format, offering text - to - text generation capabilities. It can handle various tasks like translation, question - answering, and logical reasoning, leveraging the power of the base google/flan-t5-base model.

🚀 Quick Start

Install llama.cpp

Install llama.cpp via brew (compatible with Mac and Linux):

brew install llama.cpp

Use llama.cpp

CLI Usage

llama-cli --hf-repo marroyo777/flan-t5-base-Q4_K_M-GGUF --hf-file flan-t5-base-q4_k_m-imat.gguf -p "The meaning to life and the universe is"

Server Usage

llama-server --hf-repo marroyo777/flan-t5-base-Q4_K_M-GGUF --hf-file flan-t5-base-q4_k_m-imat.gguf -c 2048

Alternative Usage Steps

1. Clone llama.cpp from GitHub:

git clone https://github.com/ggerganov/llama.cpp

2. Build llama.cpp: Move into the llama.cpp directory and build it with the LLAMA_CURL = 1 flag and other hardware - specific flags (e.g., LLAMA_CUDA = 1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

3. Run Inference:

./llama-cli --hf-repo marroyo777/flan-t5-base-Q4_K_M-GGUF --hf-file flan-t5-base-q4_k_m-imat.gguf -p "The meaning to life and the universe is"

or

./llama-server --hf-repo marroyo777/flan-t5-base-Q4_K_M-GGUF --hf-file flan-t5-base-q4_k_m-imat.gguf -c 2048

✨ Features

• Multilingual Support: Supports languages such as English, French, Romanian, German, and offers multilingual capabilities.
• Diverse Tasks: Capable of handling a wide range of tasks including text - to - text generation, translation, question - answering, logical reasoning, and more.

📦 Installation

The model was converted to the GGUF format from google/flan-t5-base using llama.cpp via the ggml.ai's GGUF - my - repo space.

📚 Documentation

Model Details

• Base Model: google/flan-t5-base
• Model Conversion: Converted to GGUF format using llama.cpp
• Refer to: Original model card for more details on the model.

Supported Datasets

Widget Examples

The model comes with several widget examples to showcase its capabilities:

📄 License

This model is released under the apache - 2.0 license.