Falcon E 3B Instruct

🚀 Falcon-E Model

Falcon-E is a series of powerful language models developed by TII. It offers high performance with relatively low memory footprint, suitable for various NLP tasks.

🚀 Quick Start

Currently, you can use this model through either the Hugging Face transformers library or the BitNet library. There are multiple ways to interact with the model depending on your target usage. For each of the Falcon-E series models, you have three variants: the BitNet model, the prequantized checkpoint for fine - tuning, and the bfloat16 version of the BitNet model.

✨ Features

• Low Memory Footprint: The Falcon - E models, such as Falcon - E - 1B - Base and Falcon - E - 3B - Base, have significantly lower memory footprints compared to some of their counterparts, making them more resource - efficient.
• Multiple Variants: Available in different versions like BitNet, prequantized for fine - tuning, and bfloat16 for inference.

📦 Installation

For using the model with the Hugging Face transformers library, you need to have the transformers library installed. For BitNet, you can follow these steps:

git clone https://github.com/microsoft/BitNet && cd BitNet
pip install -r requirements.txt
python setup_env.py --hf-repo tiiuae/Falcon-E-3B-Instruct -q i2_s

💻 Usage Examples

Basic Usage

🤗 transformers

If you want to perform inference on the BitNet checkpoint:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-E-3B-Instruct"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
).to("cuda")

# Perform text generation

If you want to use the classic bfloat16 version:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-E-3B-Instruct"
revision = "bfloat16"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
  revision=revision,
).to("cuda")

# Perform text generation

BitNet

python run_inference.py -m models/Falcon-E-3B-Instruct/ggml-model-i2_s.gguf -p "You are a helpful assistant" -cnv

Advanced Usage

For fine - tuning the model, you should load the prequantized revision of the model and use the onebitllms Python package:

import torch

from transformers import AutoModelForCausalLM, AutoTokenizer
from trl import SFTTrainer
from onebitllms import replace_linear_with_bitnet_linear, quantize_to_1bit

model_id = "tiiuae/Falcon-E-3B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id, revision="prequantized")
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    revision="prequantized"
)
model = replace_linear_with_bitnet_linear(model)

trainer = SFTTrainer(
    model,
    ...
)

trainer.train()

quantize_to_1bit(output_directory)

📚 Documentation

Model Details

Training Details

For more details about the training protocol of this model, please refer to the Falcon - E technical blogpost.

Evaluation

We report in the following table our internal pipeline benchmarks. Note that evaluation results are normalized scores from former Hugging Face leaderboard v2 tasks.

Below are the results for instruction fine - tuned models:

Useful links

• View our release blogpost.
• Learn more about onebitllms library.
• Feel free to join our discord server if you have any questions or to interact with our researchers and developers.

📄 License

This model is licensed under the Falcon - LLM License.

📚 Citation

If the Falcon - E family of models were helpful to your work, feel free to give us a cite.

@misc{tiionebitllms,
    title = {Falcon-E, a series of powerful, universal and fine-tunable 1.58bit language models.},
    author = {Falcon-LLM Team},
    month = {April},
    url = {https://falcon-lm.github.io/blog/falcon-edge},
    year = {2025}
}