Falcon H1 3B Instruct

🚀 Transformers Library for Falcon-H1

This library provides access to the Falcon-H1 series of models, offering high performance across various natural language processing tasks. It supports multiple inference methods and has been well-evaluated on a range of benchmarks.

🚀 Quick Start

To quickly get started with the Falcon-H1 models, you can choose from different inference methods as described in the Usage section.

✨ Features

• Hybrid Architecture: Combines Transformers and Mamba architecture for enhanced performance.
• Multilingual Support: Supports English and other languages.
• Multiple Inference Options: Can be used with Hugging Face transformers, vLLM, or a custom fork of llama.cpp.

📦 Installation

To use this model, you need to install the necessary libraries. Here are the installation commands for different inference methods:

Install transformers from source

pip install git+https://github.com/huggingface/transformers.git

Install vLLM

pip install vllm

Install custom fork of llama.cpp

You can install the custom fork of llama.cpp from here. Follow the same installation guidelines as llama.cpp.

💻 Usage Examples

Basic Usage

Using transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-H1-1B-Base"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
  device_map="auto"
)

# Perform text generation

Using vLLM

# pip install vllm
vllm serve tiiuae/Falcon-H1-1B-Instruct --tensor-parallel-size 2 --data-parallel-size 1

Advanced Usage

While we are working on integrating our architecture directly into llama.cpp library, you can use our custom fork for now. Refer to https://github.com/tiiuae/llama.cpp-Falcon-H1 for installation and usage details.

📚 Documentation

Model Details

Training Details

For more details about the training protocol of this model, please refer to the Falcon-H1 technical blogpost.

Evaluation

Falcon-H1 series perform very well on a variety of tasks, including reasoning tasks.

You can check more in detail on our release blogpost, detailed benchmarks.

📄 License

This model is licensed under the Falcon-LLM License.

📚 Citation

If the Falcon-H1 family of models were helpful to your work, feel free to cite us using the following BibTeX entry:

@misc{tiifalconh1,
    title = {Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance},
    url = {https://falcon-lm.github.io/blog/falcon-h1},
    author = {Falcon-LLM Team},
    month = {May},
    year = {2025}
}

🔗 Useful links

• View our release blogpost.
• Feel free to join our discord server if you have any questions or to interact with our researchers and developers.