Falcon E 1B Base

🚀 Falcon-E Model

Falcon-E is a series of powerful, universal, and fine-tunable 1.58bit language models developed by tiiuae, offering high performance in various NLP tasks with different model scales.

🚀 Quick Start

To use the Falcon-E model, you can rely on the Hugging Face transformers library or the BitNet library. There are multiple ways to interact with the model depending on your target usage.

✨ Features

• Model Type: Causal decoder-only / Base version
• Architecture: Pure-transformer - 1.58bit version
• Language(s) (NLP): English
• License: Falcon-LLM License

📦 Installation

Currently, to use this model, you can either rely on the Hugging Face transformers library or the BitNet library. For the BitNet library, you can install it as follows:

git clone https://github.com/microsoft/BitNet && cd BitNet
pip install -r requirements.txt

💻 Usage Examples

Basic Usage

🤗 transformers

If you want to perform inference on the BitNet checkpoint, run:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-E-1B-Base"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
).to("cuda")

# Perform text generation

If you want to use the classic bfloat16 version, you can run:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-E-1B-Base"
revision = "bfloat16"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
  revision=revision,
).to("cuda")

# Perform text generation

BitNet

python setup_env.py --hf-repo tiiuae/Falcon-E-1B-Base -q i2_s
python run_inference.py -m models/Falcon-E-1B-Base/ggml-model-i2_s.gguf -p "You are a helpful assistant" -cnv

Advanced Usage

Fine-tuning

For fine-tuning the model, you should load the prequantized revision of the model and use the onebitllms Python package:

import torch

from transformers import AutoModelForCausalLM, AutoTokenizer
from trl import SFTTrainer
from onebitllms import replace_linear_with_bitnet_linear, quantize_to_1bit

model_id = "tiiuae/Falcon-E-1B-Base"

tokenizer = AutoTokenizer.from_pretrained(model_id, revision="prequantized")
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    revision="prequantized"
)
model = replace_linear_with_bitnet_linear(model)

trainer = SFTTrainer(
    model,
    ...
)

trainer.train()

quantize_to_1bit(output_directory)

📚 Documentation

Model Details

Training Details

For more details about the training protocol of this model, please refer to the Falcon-E technical blogpost.

Evaluation

We report in the following table our internal pipeline benchmarks. Note that evaluation results are normalized scores from former Hugging Face leaderboard v2 tasks.

For 1B scale models and below

For 3B scale models

Instruction fine-tuned models - For 1B scale models and below

Instruction fine-tuned models - For 3B scale models

Useful links

• View our release blogpost.
• Learn more about onebitllms library.
• Feel free to join our discord server if you have any questions or to interact with our researchers and developers.

📄 License

This model is under the Falcon-LLM License.

🔧 Technical Details

For more technical details, please refer to the Falcon-E technical blogpost.

📄 Citation

If the Falcon-E family of models were helpful to your work, feel free to give us a cite.

@misc{tiionebitllms,
    title = {Falcon-E, a series of powerful, universal and fine-tunable 1.58bit language models.},
    author = {Falcon-LLM Team},
    month = {April},
    url = {https://falcon-lm.github.io/blog/falcon-edge},
    year = {2025}
}