Voice Safety Classifier V2

🚀 Voice Safety Classification Model

This voice-safety classification model is designed for voice-toxicity detection and classification. It offers support for multiple languages and high - accuracy classification, enhancing the safety of voice chat environments.

🚀 Quick Start

Installation

The dependencies for the inference file can be installed as follows:

pip install -r requirements.txt

Inference

The provided Python file demonstrates how to use the classifier with arbitrary 16kHz audio input. To run the inference, please run the following command:

python inference.py --audio_file <your audio file path> --model_path <path to Huggingface model>

You can download the model weights from the model releases page here, or from HuggingFace under roblox/voice-safety-classifier-v2. If model_path isn’t specified, the model will be loaded directly from HuggingFace.

✨ Features

• Multilingual Support: The model supports eight languages: English, Spanish, German, French, Portuguese, Italian, Korean, and Japanese.
• Multilabel Classification: It classifies the input audio into six toxicity classes in a multilabel fashion: Discrimination, Harassment, Sexual, IllegalAndRegulated, DatingAndRomantic, and Profanity.
• Calibrated Output: The model outputs have been calibrated for the Roblox voice chat environment, so that the class scores after a sigmoid can be interpreted as probabilities.
• Improved Accuracy: Compared to the v1 voice safety classifier, the v2 model expands the support from English to 7 additional languages and significantly improves the classification accuracy.

📚 Documentation

Model Description

We present a voice-safety classification model that can be used for voice-toxicity detection and classification. The model has been distilled into the WavLM architecture from a larger teacher model. All the model training has been conducted with Roblox internal voice chat datasets, using both machine and human-labeled data, with over 100k hours of training data in total. We have also published a blog post about this work.

Please refer to Roblox Community Standards for a detailed explanation on the policy, which has been used for labeling the datasets.

The classifier expects 16kHz audio segments as input. Ideal segment length is 15 seconds, but the classifier can operate on shorter segments as well. The prediction accuracy may degrade for longer segments.

Evaluation Results

The table below displays evaluation precision and recall for each of the supported languages, as calculated over internal language-specific held-out datasets, which resemble the Roblox voice chat traffic. The operating thresholds for each of the categories were kept equal per language, and optimized to achieve a false positive rate of 1%. The classifier was then evaluated as a binary classifier, tagging the audio as positive if any of the heads exceeded the threshold.

Compared to the v1 voice safety classifier, the v2 model expands the support from English to 7 additional languages, as well as significantly improving the classification accuracy. With the 1% false positive rate as above, the binary recall for English is improved by 92%.

📄 License

This model is licensed under CC BY-SA 3.0.