German RAG WHISPER LARGE V3 TURBO HESSIAN AI

🚀 German-RAG-WHISPER-LARGE-v3-TURBO-HESSIAN-AI

This model is a fine - tuned speech - to - text model. It is trained on a carefully curated 13 - hour dataset, aiming to enhance the performance of automatic speech recognition in German.

🚀 Quick Start

How to use

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
from datasets import load_dataset
device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
model_id = "avemio/German-RAG-WHISPER-LARGE-v3-TURBO"
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
)
model.to(device)
processor = AutoProcessor.from_pretrained(model_id)
pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    max_new_tokens=128,
    chunk_length_s=30,
    batch_size=16,
    return_timestamps=True,
    torch_dtype=torch_dtype,
    device=device,
)
dataset = load_dataset("distil-whisper/librispeech_long", "clean", split="validation")
sample = dataset[0]["audio"]
result = pipe(sample)
print(result["text"])

✨ Features

This model is fine - tuned on a carefully curated 13 - hour dataset, which includes conversations of spoken German with a mix of English business phrases. It is optimized for recognition performance.

📚 Documentation

Evaluations - Word error rate

The data and code for evaluations are available here

Training data

The training data for this model includes conversations of spoken German with a mix of English business phrases. The data was carefully selected and processed to optimize recognition performance. The dataset will not be published because of the unclear situation regarding its potential use for voice - cloning. The rights to use the collected data are only for the intended use of training speech - to - text models.

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0

🔧 Technical Details

The model is based on the openai/whisper - large - v3 - turbo base model and is fine - tuned on a specific German - related dataset.

📄 License

This model is released under the MIT license.

The German - RAG AI Team

Marcel Rosiak Soumya Paul Siavash Mollaebrahim Zain ul Haq