Voila Audio Alpha

🚀 Voila: Voice-Language Foundation Models

Voila is a new family of large voice - language foundation models. It aims to elevate human - AI interaction experiences. By breaking free from the limitations of traditional voice AI systems, such as high latency, loss of vocal nuances, and mechanical responses, Voila offers real - time, autonomous, and rich voice interactions. It combines advanced voice and language modeling, excelling in various audio tasks across six languages.

🚀 Quick Start

CLI demo

for model_name in "maitrix-org/Voila-audio-alpha" "maitrix-org/Voila-base" "maitrix-org/Voila-chat"; do
    # Text chat
    python infer.py \
        --model-name ${model_name} \
	    --instruction "" \
	    --input-text "Hello" \
	    --task-type chat_tito
    # Voice chat
    python infer.py \
        --model-name ${model_name} \
	    --instruction "" \
	    --input-audio "examples/test1.mp3" \
	    --task-type chat_aiao
done

# Autonomous mode
python infer.py \
    --model-name "maitrix-org/Voila-autonomous-preview" \
	--instruction "" \
	--input-audio "examples/test_autonomous1.mp3" \
	--task-type chat_aiao_auto

Gradio demo

python gradio_demo.py

For more information, please refer to the code repository.

✨ Features

• High - fidelity, low - latency, real - time streaming audio processing
• Effective integration of voice and language modeling capabilities
• Millions of pre - built and custom voices, fast voice switching during conversation
• Unified model for various audio tasks

📦 Installation

No installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

The basic usage can be demonstrated through the CLI and Gradio demos as shown above.

Advanced Usage

There are no advanced usage examples provided in the original document.

📚 Documentation

Foundation Models

Datasets

We publish the following two datasets: Voila Benchmark and Voila Voice Library. Voila - Benchmark is a novel speech evaluation benchmark, while Voila Voice Library provides millions of pre - built and customizable voices.

Benchmark

1. Voila Benchmark

We introduce a novel speech evaluation benchmark called the VoilaBenchmark. It is constructed by sampling from five widely used language model evaluation datasets: MMLU, MATH, OpenAI HumanEval, NQ - Open, and GSM8k. We compare our results with SpeechGPT and Moshi.

(higher is better)

For detailed scores of Voila Benchmark on each specific domain, please refer to our paper (Section 5.1 "Evaluation of Voila Benchmark").

2. Evaluation of ASR

As Voila supports multiple tasks, including Automatic Speech Recognition (ASR), Text - to - Speech (TTS), and spoken question answering, we also evaluate the performance of ASR and TTS. For ASR, we assess performance on the LibriSpeech test - clean dataset, using Word Error Rate (WER) as our metric. Voila attains a word error rate (WER) of 4.8%, outperforming the 5.7% reported by Moshi. In scenarios where both models utilize LibriSpeech training data, Voila achieves an impressive WER of 2.7%.

(lower is better)

3. Evaluation of TTS

For TTS, we follow the evaluation metrics proposed in Vall - E, which involves transcribing the generated audio using HuBERT - Large. Voila once again leads with a WER of 3.2% (and 2.8% when using LibriSpeech training data).

(lower is better)

🔧 Technical Details

There are no technical details provided in the original document.

📄 License

The project is licensed under the MIT license.

📚 Citation

If you find our work helpful, please cite us.

@article{voila2025,
  author    = {Yemin Shi, Yu Shu, Siwei Dong, Guangyi Liu, Jaward Sesay, Jingwen Li, Zhiting Hu},
  title     = {Voila: Voice-Language Foundation Models for Real-Time Autonomous Interaction and Voice Roleplay},
  eprint={2505.02707},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  year      = {2025}
}

Voila: Voice-Language Foundation Models

Project Page    |    GitHub    |    Hugging Face   |    Paper    |    Online Demo   |    Maitrix.org

Video Demo

Latest News!!

• April 28, 2025: We've released the inference code and model weights of Voila.