Granite 4.0 Tiny Base Preview

🚀 Granite-4.0-Tiny-Base-Preview

Granite-4.0-Tiny-Base-Preview is a 7B-parameter hybrid mixture-of-experts (MoE) language model with a 128k token context window, offering enhanced expressiveness and better length generalization.

🚀 Quick Start

First, install the necessary components. Then, you can run the generation example by copying the provided code snippet.

✨ Features

• Large Context Window: It features a 128k token context window, enabling it to handle long - context tasks effectively.
• Hybrid Architecture: Leverages Mamba - 2 with softmax attention for enhanced expressiveness and no positional encoding for better length generalization.
• Multilingual Support: Supports multiple languages including English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese.

📦 Installation

You need to install transformer from source to use this checkpoint.

• HuggingFace PR: https://github.com/huggingface/transformers/pull/37658
• Install transformer from source: https://huggingface.co/docs/transformers/en/installation#install-from-source

💻 Usage Examples

Basic Usage

After installation, copy the following code snippet to run the example.

from transformers import AutoModelForCausalLM, AutoTokenizer
device = "auto"
model_path = "ibm-granite/granite-4.0-tiny-base-preview"
tokenizer = AutoTokenizer.from_pretrained(model_path)
# drop device_map if running on CPU
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()
# change input text as desired
input_text = "Where is the Thomas J. Watson Research Center located?"
# tokenize the text
input_tokens = tokenizer(input_text, return_tensors="pt").to(device)
# generate output tokens
output = model.generate(**input_tokens,
                        max_length=4000)
# decode output tokens into text
output = tokenizer.batch_decode(output)
# print output
print(output)

📚 Documentation

Model Information

• Developers: Granite Team, IBM
• Website: Granite Docs
• Release Date: May 2nd, 2025
• License: Apache 2.0
• Supported Languages: English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. Users may finetune Granite 4.0 models for languages beyond these 12 languages.
• Intended Use: Prominent use cases of LLMs in text - to - text generation include summarization, text classification, extraction, question - answering, and other long - context tasks. All Granite Base models are able to handle these tasks as they were trained on a large amount of data from various domains. Moreover, they can serve as a baseline to create specialized models for specific application scenarios.

Evaluation Results

Model Architecture

Granite - 4.0 - Tiny - Base - Preview is based on a decoder - only dense transformer architecture. Core components of this architecture are: GQA and NoPE, MLP with SwiGLU, RMSNorm, and shared input/output embeddings.

📄 License

This project is licensed under the Apache 2.0 license.