Granite 3.2 8b Instruct GGUF

🚀 Granite-3.2-8B-Instruct-GGUF

This repository offers models converted to the GGUF format with various quantizations from an IBM Granite base model, suitable for multiple language instruction - following tasks.

🚀 Quick Start

⚠️ Important Note

This repository contains models that have been converted to the GGUF format with various quantizations from an IBM Granite base model.

Please reference the base model's full model card here: https://huggingface.co/ibm-granite/granite-3.2-8b-instruct

✨ Features

Model Summary

Granite-3.2-8B-Instruct is an 8-billion-parameter, long-context AI model fine-tuned for thinking capabilities. Built on top of Granite-3.1-8B-Instruct, it has been trained using a mix of permissively licensed open-source datasets and internally generated synthetic data designed for reasoning tasks. The model allows controllability of its thinking capability, ensuring it is applied only when required.

Supported Languages

English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. However, users may finetune this Granite model for languages beyond these 12 languages.

Intended Use

This model is designed to handle general instruction-following tasks and can be integrated into AI assistants across various domains, including business applications.

Capabilities

• Thinking
• Summarization
• Text classification
• Text extraction
• Question-answering
• Retrieval Augmented Generation (RAG)
• Code related tasks
• Function-calling tasks
• Multilingual dialog use cases
• Long-context tasks including long document/meeting summarization, long document QA, etc.

📄 License

The model is released under the Apache 2.0 license.