đ Voila: Voice-Language Foundation Models
Voila is a new family of large voice - language foundation models. It breaks away from the limitations of traditional voice AI systems, enabling real - time, autonomous, and rich voice interactions. By integrating advanced voice and language modeling, it excels in various audio tasks across six languages, offering a transformative human - AI interaction experience.

Voila: Voice-Language Foundation Models
Project Page    |    GitHub    |    Hugging Face   |    Paper    |    Online Demo   |    Maitrix.org
⨠Features
- High - fidelity, low - latency, real - time streaming audio processing
- Effective integration of voice and language modeling capabilities
- Millions of pre - built and custom voices, fast voice switching during conversation
- Unified model for various audio tasks
đĻ Installation
No installation steps are provided in the original document.
đģ Usage Examples
Basic Usage
CLI demo
for model_name in "maitrix-org/Voila-audio-alpha" "maitrix-org/Voila-base" "maitrix-org/Voila-chat"; do
# Text chat
python infer.py \
--model-name ${model_name} \
--instruction "" \
--input-text "Hello" \
--task-type chat_tito
# Voice chat
python infer.py \
--model-name ${model_name} \
--instruction "" \
--input-audio "examples/test1.mp3" \
--task-type chat_aiao
done
# Autonomous mode
python infer.py \
--model-name "maitrix-org/Voila-autonomous-preview" \
--instruction "" \
--input-audio "examples/test_autonomous1.mp3" \
--task-type chat_aiao_auto
Gradio demo
python gradio_demo.py
For more information, please refer to the code repository.
đ Documentation
Foundation Models
Property |
Details |
Model Type |
Voila - base, Voila - Chat, Voila - Autonomous (preview), Voila - Audio - alpha, Voila - Tokenizer |
Description |
Voila base model, End - to - end audio chat model, Full - duplex audio chat model, Empowering LLM with raw audio input, Audio tokenizer |
Download Link |
Voila - base, Voila - Chat, Voila - Autonomous (preview), Voila - Audio - alpha, Voila - Tokenizer |
Datasets
Property |
Details |
Model Type |
Voila Benchmark, Voila Voice Library |
Description |
Evaluation of Voila Benchmark, Millons of pre - build voices |
Download Link |
Voila Benchmark, Voila Voice Library |
Benchmark
1. Voila Benchmark
We introduce a novel speech evaluation benchmark called the VoilaBenchmark, constructed by sampling from five widely used language model evaluation datasets: MMLU, MATH, OpenAI HumanEval, NQ - Open, and GSM8k. We compare our results with SpeechGPT and Moshi.
Model |
Voila Benchmark |
SpeechGPT |
13.29 |
Moshi |
11.45 |
Voila |
30.56 |
(higher is better)
For detailed scores of Voila Benchmark on each specific domain, please refer to our paper (Section 5.1 "Evaluation of Voila Benchmark").
2. Evaluation of ASR
As Voila supports multiple tasks, including Automatic Speech Recognition (ASR), Text - to - Speech(TTS), and spoken question answering, we also evaluate the performance of ASR and TTS. For ASR, we assess performance on the LibriSpeech test - clean dataset, using Word Error Rate (WER) as our metric.
Model |
LibriSpeech test - clean (WER) |
Whisper large v2 |
2.7 |
Whisper large v3 |
2.2 |
FastConformer |
3.6 |
VoxtLM |
2.7 |
Moshi |
5.7 |
Voila (w/o LibriSpeech train split) |
4.8 |
Voila (with LibriSpeech train split) |
2.7 |
(lower is better)
3. Evaluation of TTS
For TTS, we follow the evaluation metrics proposed in Vall - E, which involves transcribing the generated audio using HuBERT - Large.
Model |
LibriSpeech test - clean (WER) |
YourTTS |
7.7 |
Vall - E |
5.9 |
Moshi |
4.7 |
Voila (w/o LibriSpeech train split) |
3.2 |
Voila (with LibriSpeech train split) |
2.8 |
(lower is better)
đ License
This project is licensed under the MIT license.
đ Citation
If you find our work helpful, please cite us.
@article{voila2025,
author = {Yemin Shi, Yu Shu, Siwei Dong, Guangyi Liu, Jaward Sesay, Jingwen Li, Zhiting Hu},
title = {Voila: Voice - Language Foundation Models for Real - Time Autonomous Interaction and Voice Roleplay},
eprint={2505.02707},
archivePrefix={arXiv},
primaryClass={cs.CL},
year = {2025}
}