Geitje 7B Ultra

🚀 GEITje 7B ultra

A conversational model for Dutch, aligned through AI feedback.

This model is a fine - tuned version of BramVanroy/GEITje-7B-ultra-sft on a synthetic DPO dataset of around 56M tokens that was generated with gpt-4-turbo and Rijgersberg/GEITje-7B-chat for Dutch.

💡 Usage Tip

🚀 Looking for the fast GGUF version? You can find it, and how to use it with ollama, here. 🚀

🚀 Quick Start

The following shows how to use the model in one - off and interactive conversation scenarios:

Basic Usage

from transformers import pipeline, Conversation

# load_in_8bit: lower precision but saves a lot of GPU memory
# device_map=auto: loads the model across multiple GPUs
chatbot = pipeline("conversational", model="BramVanroy/GEITje-7B-ultra", model_kwargs={"load_in_8bit": True}, device_map="auto")

start_messages = [
    {"role": "system", "content": "Je bent een grappige chatbot die Bert heet. Je maakt vaak mopjes."},
    {"role": "user", "content": "Hallo, ik ben Bram. Ik wil vanavond graag een film kijken. Heb je enkele suggesties?"}
]
conversation = Conversation(start_messages)
conversation = chatbot(conversation)
response = conversation.messages[-1]["content"]
print(response)

Advanced Usage

from transformers import pipeline, Conversation

# load_in_8bit: lower precision but saves a lot of memory
# device_map=auto: loads the model across multiple GPUs
# attn_implementation: uses flash attention, if your device supports it - otherwise remove it
chatbot = pipeline("conversational", model="BramVanroy/GEITje-7B-ultra", model_kwargs={"load_in_8bit": True, "attn_implementation": "flash_attention_2"}, device_map="auto")

while (system_message := input("System message ('q' to quit): ")) != "q":
    start_messages = [
        {"role": "system", "content": system_message},
    ]
    conversation = Conversation(start_messages)
    while (user_input := input("User ('r' to reset): ")) != "r":
        conversation.add_user_input(user_input)
        conversation = chatbot(conversation)
        response = conversation.messages[-1]["content"]
        print("Assistant:", response)

✨ Features

• Conversational Capability: It is a conversational model for Dutch, aligned through AI feedback.
• Based on Strong Architecture: Ultimately based on Mistral and aligned with AI feedback via DPO.

📚 Documentation

Citation

If you use GEITje 7B Ultra (SFT) or any of its derivatives or quantizations, place cite the following paper:

@misc{vanroy2024geitje7bultraconversational,
      title={GEITje 7B Ultra: A Conversational Model for Dutch}, 
      author={Bram Vanroy},
      year={2024},
      eprint={2412.04092},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2412.04092}, 
}

Intended uses & limitations

⚠️ Important Note

Although the model has been aligned with gpt - 4 - turbo output, which has strong content filters, the model could still generate wrong, misleading, and potentially even offensive content. Use at your own risk.

⚠️ Important Note

Because the model was trained on synthetic data created with OpenAI/Azure services, this model cannot be used for commercial purposes.

Training and evaluation data

The training data consists of a synthetic dataset based on UltraFeedback binarized created with gpt - 4 - turbo and geitje - chat. A given prompt, translated from the original dataset, is given to the two models who then generated an answer. Then, gpt - 4 - turbo is always selected as the best answer which DPO will optimise for. While this is not completely fair, the author did not have the budget to actually have gpt - 4 rate both replies. Furthermore, while an impressive model, GEITje chat still seems behind gpt - 4 - turbo in the testing that the author has done.

In total the dataset consists of 56,137,090 tokens (combination of prompt + rejected + chosen) and a test set of 6,178,969 tokens (11.00%).

Training procedure

The great alignment handbook was used for training, with a custom slurm script for compatibility with the cluster. It was trained in full, without LoRA or other adapters.

The model was trained in bfloat16 with flash attention 2 on two nodes of four A100 80GB each for around 11 hours. The author thanks the Flemish Super Computer for their compute.

For conversational usage, the model relies on the Zephyr chat template, which is compatible with system messages. A small portion of the data of *-sft contained system messages, so it is assumed the model can handle system messages at least a little bit.

In earlier iterations, using the alignment handbook's defaults (beta = 0.01) led to poor results (hallucinations of random tokens). After investigation, it seems that such a low beta does not work well for this dataset as it gives the model too much room to deviate from its initial base model. After a hyperparameter search and manual analysis of the resulting metrics, the current model was selected as the best one, with a beta of 0.1.

Training hyperparameters

The following hyperparameters were used during training:

Training results

Open LLM Leaderboard Evaluation Results

Results for the English Open LLM Leaderboard. For results specific to Dutch, check out ScandEval.

Detailed results can be found here

📄 License

This model is licensed under cc - by - nc - 4.0.