Falcon-H1-7B-Base Open-Source Language Model - Free Deployment, Supports Multilingual Processing, with Excellent Performance

Falcon H1 7B Base

Developed by tiiuae

Falcon-H1 is a causal decoder-only language model with a hybrid Transformers + Mamba architecture developed by TII, supporting multilingual processing with excellent performance.

Large Language Model

Transformers

Supports Multiple LanguagesOpen Source License:Other #Hybrid Architecture Transformers-Mamba #Multilingual Reasoning #Efficient Mathematical Computation

Downloads 227

Release Time : 5/1/2025

Model Overview

Falcon-H1 is an efficient hybrid architecture language model that combines the strengths of Transformers and Mamba, suitable for various natural language processing tasks.

Model Features

Hybrid Architecture

Combines the strengths of Transformers and Mamba architectures to enhance model efficiency and performance.

Multilingual Support

Supports 18 languages, including English, Chinese, French, German, etc.

High Performance

Excels in various benchmarks, particularly in reasoning and mathematical tasks.

Model Capabilities

Text generation

Multilingual processing

Reasoning tasks

Mathematical computation

Code generation

Use Cases

General Tasks

QA Systems

Used to build efficient QA systems for answering complex questions.

Performs excellently in BBH and MMLU benchmarks.

Text Generation

Generates high-quality, coherent text content.

Performs well in HellaSwag and Winogrande benchmarks.

Mathematics & Science

Mathematical Problem Solving

Solves complex mathematical problems, including GSM8k and MATH lvl5 tasks.

Outperforms in GSM8k and MATH lvl5 benchmarks.

Scientific QA

Answers science-related questions, especially in MMLU-Pro and MMLU-stem tasks.

Excels in GPQA and MMLU-Pro benchmarks.

Code Generation

Code Completion

Generates high-quality code snippets supporting multiple programming languages.

Performs well in HumanEval and MBPP benchmarks.

🚀 Falcon-H1

The Falcon-H1 series is a family of hybrid - head language models developed by TII, offering high - performance solutions for various NLP tasks with a unique architecture and multilingual support.

🚀 Quick Start

To start using the Falcon - H1 model, you can choose from different libraries such as Hugging Face transformers, vLLM, or our custom fork of llama.cpp library.

✨ Features

Developed by: https://www.tii.ae
Model type: Causal decoder - only
Architecture: Hybrid Transformers + Mamba architecture
Language(s) (NLP): English, Multilingual
License: Falcon - LLM License

📦 Installation

Install `transformers` from source

pip install git+https://github.com/huggingface/transformers.git

Refer to the official vLLM documentation for more details on building vLLM from source.

💻 Usage Examples

Basic Usage

Using `transformers`

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-H1-1B-Base"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
  device_map="auto"
)

# Perform text generation

Using `vLLM`

# pip install vllm
vllm serve tiiuae/Falcon-H1-1B-Instruct --tensor-parallel-size 2 --data-parallel-size 1

Using `llama.cpp`

You can install our fork of the library and use it directly: https://github.com/tiiuae/llama.cpp - Falcon - H1. Use the same installing guidelines as llama.cpp.

📚 Documentation

Model Details

The Falcon - H1 model is a causal decoder - only model with a hybrid architecture of Transformers and Mamba. It supports English and multiple languages, and is released under the Falcon - LLM License.

Training Details

For more details about the training protocol of this model, please refer to the Falcon - H1 technical blogpost.

Evaluation

The Falcon - H1 series performs very well on a variety of tasks, including reasoning tasks. The following table shows the performance comparison with other models:

Tasks	Falcon - H1 - 7B	Qwen3 - 8B	Qwen2.5 - 7B	Gemma3 - 12B	Llama3.1 - 8B	Falcon3 - 7B	Falcon3 - 10B
General
BBH	60.61	58.44	53.72	54.33	46.52	50.88	59.3
MMLU	77.38	76.63	74.17	74.23	65.17	69.98	73.22
ARC - C	65.19	67.75	63.91	67.58	57.68	62.71	67.49
HellaSwag	81.26	79.6	80.2	84.22	81.97	76.69	79.64
Winogrande	79.01	76.8	76.01	79.79	77.11	73.64	79.01
Math
GSM8k	73.46	83.02	83.09	71.19	49.51	76.95	82.11
MATH lvl5	34.67	28.85	22.58	17.22	6.57	20.09	25.38
Science
GPQA	36.58	35.65	32.3	34.56	31.46	35.07	35.4
MMLU - Pro	48.38	48.25	43.55	42.72	32.71	39.23	42.45
MMLU - stem	77.2	78.53	71.04	68.51	55.72	67.71	70.85
Code
HumanEval	67.68	87.8	57.32	45.12	39.02	50.0	51.83
HumanEval+	63.41	82.32	48.78	36.59	31.71	43.29	44.51
MBPP	78.57	75.13	76.72	73.02	61.38	67.99	73.54
MBPP+	67.2	64.02	63.49	59.79	51.32	57.14	61.38

You can check more in detail on our our release blogpost, detailed benchmarks.

Useful Links

View our release blogpost.
Feel free to join our discord server if you have any questions or to interact with our researchers and developers.

📄 License

The model is released under the Falcon - LLM License.

📚 Citation

If the Falcon - H1 family of models were helpful to your work, feel free to give us a cite.

@misc{tiifalconh1,
    title = {Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance},
    url = {https://falcon-lm.github.io/blog/falcon-h1},
    author = {Falcon-LLM Team},
    month = {May},
    year = {2025}
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご