Falcon H1 7B Instruct

🚀 Falcon-H1 Model

Falcon-H1 is a family of hybrid-head language models developed by TII, offering high efficiency and performance across various NLP tasks.

🚀 Quick Start

To use the Falcon-H1 model, you can rely on Hugging Face transformers, vLLM, or a custom fork of the llama.cpp library.

Installation

Make sure to install the latest version of transformers or vLLM. You can install these packages from source:

pip install git+https://github.com/huggingface/transformers.git

Refer to the official vLLM documentation for more details on building vLLM from source.

Inference

🤗 transformers

Refer to the snippet below to run H1 models using 🤗 transformers:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-H1-1B-Base"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
  device_map="auto"
)

# Perform text generation

vLLM

For vLLM, simply start a server by executing the command below:

# pip install vllm
vllm serve tiiuae/Falcon-H1-1B-Instruct --tensor-parallel-size 2 --data-parallel-size 1

llama.cpp

While the team is working on integrating the architecture directly into the llama.cpp library, you can install the custom fork of the library and use it directly: https://github.com/tiiuae/llama.cpp-Falcon-H1. Use the same installing guidelines as llama.cpp.

✨ Features

• Model Type: Causal decoder-only
• Architecture: Hybrid Transformers + Mamba architecture
• Language(s) (NLP): English, Multilingual
• License: Falcon-LLM License

📦 Installation

For installation details, refer to the "Quick Start" section above.

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-H1-1B-Base"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
  device_map="auto"
)

# Perform text generation

📚 Documentation

Model Details

• Developed by: https://www.tii.ae
• Model type: Causal decoder-only
• Architecture: Hybrid Transformers + Mamba architecture
• Language(s) (NLP): English, Multilingual
• License: Falcon-LLM License

Training Details

For more details about the training protocol of this model, please refer to the Falcon-H1 technical blogpost.

Evaluation

Falcon-H1 series perform very well on a variety of tasks, including reasoning tasks.

You can check more in detail on our our release blogpost, detailed benchmarks.

📄 License

This model is licensed under the Falcon-LLM License.

📚 Citation

If the Falcon-H1 family of models were helpful to your work, feel free to give us a cite.

@misc{tiifalconh1,
    title = {Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance},
    url = {https://falcon-lm.github.io/blog/falcon-h1},
    author = {Falcon-LLM Team},
    month = {May},
    year = {2025}
}

🔗 Useful Links

• View our release blogpost.
• Feel free to join our discord server if you have any questions or to interact with our researchers and developers.