Falcon E 3B Base

🚀 Transformers

The transformers library provides a wide - range of pre - trained models, including those in the Falcon - E series, which are powerful language models with unique features and efficient performance.

🚀 Quick Start

This model can be used through the Hugging Face transformers library or the BitNet library. There are multiple ways to interact with the model depending on your target usage. For each of the Falcon - E series model, you have three variants: the BitNet model, the prequantized checkpoint for fine - tuning and the bfloat16 version of the BitNet model.

✨ Features

• Developed by: https://www.tii.ae
• Model type: Causal decoder - only / Base version
• Architecture: Pure - transformer - 1.58bit version
• Language(s) (NLP): English
• License: Falcon - LLM License

📦 Installation

To use the model, you need to install relevant libraries. For example, when using the transformers library, you can install it via pip install transformers. When using the BitNet library, you can follow these steps:

git clone https://github.com/microsoft/BitNet && cd BitNet
pip install -r requirements.txt

💻 Usage Examples

Basic Usage

🤗 transformers

In case you want to perform inference on the BitNet checkpoint run:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-E-1B-Base"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
).to("cuda")

# Perform text generation

If you want to rather use the classic bfloat16 version, you can run:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-E-1B-Base"
revision = "bfloat16"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
  revision=revision,
).to("cuda")

# Perform text generation

BitNet

git clone https://github.com/microsoft/BitNet && cd BitNet
pip install -r requirements.txt
python setup_env.py --hf-repo tiiuae/Falcon-E-1B-Base -q i2_s
python run_inference.py -m models/Falcon-E-1B-Base/ggml-model-i2_s.gguf -p "You are a helpful assistant" -cnv

Advanced Usage

Fine - tuning

For fine - tuning the model, you should load the prequantized revision of the model and use the onebitllms Python package:

import torch

from transformers import AutoModelForCausalLM, AutoTokenizer
from trl import SFTTrainer
+ from onebitllms import replace_linear_with_bitnet_linear, quantize_to_1bit

model_id = "tiiuae/Falcon-E-1B-Base"

tokenizer = AutoTokenizer.from_pretrained(model_id, revision="prequantized")
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
+    revision="prequantized"
)
+ model = replace_linear_with_bitnet_linear(model)

trainer = SFTTrainer(
    model,
    ...
)

trainer.train()

+ quantize_to_1bit(output_directory)

📚 Documentation

Training details

For more details about the training protocol of this model, please refer to the Falcon - E technical blogpost.

Evaluation

We report in the following table our internal pipeline benchmarks:

Note evaluation results are normalized score from former Hugging Face leaderboard v2 tasks

Below are the results for instruction fine - tuned models:

Useful links

• View our release blogpost.
• Learn more about onebitllms library.
• Feel free to join our discord server if you have any questions or to interact with our researchers and developers.

📄 License

This model is under the Falcon - LLM License.

📚 Citation

If the Falcon - E family of models were helpful to your work, feel free to give us a cite.

@misc{tiionebitllms,
    title = {Falcon-E, a series of powerful, universal and fine-tunable 1.58bit language models.},
    author = {Falcon-LLM Team},
    month = {April},
    url = {https://falcon-lm.github.io/blog/falcon-edge},
    year = {2025}
}