Leo Hessianai 7b

🚀 LAION LeoLM: Linguistically Enhanced Open Language Model

Meet LeoLM, the first open and commercially available German Foundation Language Model built on Llama-2. It extends Llama-2's capabilities into German through continued pretraining on a large German - language corpus.

🚀 Quick Start

LAION LeoLM is a groundbreaking German Foundation Language Model based on Llama - 2. It extends Llama - 2's abilities to German via continuous pretraining on a large German - language corpus. We've released two foundation models with 8k context length, [LeoLM/leo - hessianai - 7b](https://huggingface.co/LeoLM/leo - hessianai - 7b) and [LeoLM/leo - hessianai - 13b](https://huggingface.co/LeoLM/leo - hessianai - 13b), under the [Llama - 2 community license](https://huggingface.co/meta - llama/Llama - 2 - 70b/raw/main/LICENSE.txt). A 70b model is also on the way! For more details, read our blog post or upcoming paper.

✨ Features

• German - language Enhancement: Extends Llama - 2's capabilities to handle German language tasks effectively.
• Open and Commercial Availability: Available for both open - source and commercial use under the Llama - 2 community license.
• 8k Context Length: Supports longer context for better understanding and generation.

📦 Installation

Install Direct Dependencies

pip install transformers torch sentencepiece

Install Dependencies for Faster Inference with Flash - Attention2

pip install packaging ninja
pip install flash - attn==v2.1.1 --no - build - isolation
pip install git+https://github.com/HazyResearch/flash - attention.git@v2.1.1#subdirectory=csrc/rotary

💻 Usage Examples

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    model="LeoLM/leo - hessianai - 7b",
    device_map="auto",
    torch_dtype=torch.float16,
    trust_remote_code=True  # True for flash - attn2 else False
)

📚 Documentation

Model Details

Training parameters

Benchmarks