Pyannote Segmentation

🚀 🎹 说话人分割

本模型用于解决音频中说话人分割的问题，能够进行语音活动检测、重叠语音检测和重分割等操作，为音频处理提供了有效的解决方案。

🚀 快速开始

本模型来自Hervé Bredin和Antoine Laurent的论文 End-to-end speaker segmentation for overlap-aware resegmentation 。

你可以在 Hugging Face Space 上查看在线演示。

✨ 主要特性

• 多任务处理：支持语音活动检测、重叠语音检测和重分割等多种任务。
• 基于研究成果：依托于相关学术研究，具有一定的科学性和有效性。

📦 安装指南

本模型依赖于目前正在开发中的 pyannote.audio 2.0，具体安装说明请参考 安装指南。

💻 使用示例

基础用法

语音活动检测

from pyannote.audio.pipelines import VoiceActivityDetection
pipeline = VoiceActivityDetection(segmentation="pyannote/segmentation")
HYPER_PARAMETERS = {
  # onset/offset activation thresholds
  "onset": 0.5, "offset": 0.5,
  # remove speech regions shorter than that many seconds.
  "min_duration_on": 0.0,
  # fill non-speech regions shorter than that many seconds.
  "min_duration_off": 0.0
}
pipeline.instantiate(HYPER_PARAMETERS)
vad = pipeline("audio.wav")
# `vad` is a pyannote.core.Annotation instance containing speech regions

重叠语音检测

from pyannote.audio.pipelines import OverlappedSpeechDetection
pipeline = OverlappedSpeechDetection(segmentation="pyannote/segmentation")
pipeline.instantiate(HYPER_PARAMETERS)
osd = pipeline("audio.wav")
# `osd` is a pyannote.core.Annotation instance containing overlapped speech regions

重分割

from pyannote.audio.pipelines import Resegmentation
pipeline = Resegmentation(segmentation="pyannote/segmentation", 
                          diarization="baseline")
pipeline.instantiate(HYPER_PARAMETERS)
resegmented_baseline = pipeline({"audio": "audio.wav", "baseline": baseline})
# where `baseline` should be provided as a pyannote.core.Annotation instance

原始得分

from pyannote.audio import Inference
inference = Inference("pyannote/segmentation")
segmentation = inference("audio.wav")
# `segmentation` is a pyannote.core.SlidingWindowFeature
# instance containing raw segmentation scores like the 
# one pictured above (output)

高级用法

为了复现论文 "End-to-end speaker segmentation for overlap-aware resegmentation" 的结果，使用 pyannote/segmentation@Interspeech2021 并搭配以下超参数：

语音活动检测

重叠语音检测

VBx重分割

预期输出（和VBx基线）也在 /reproducible_research 子目录中提供。

📚 详细文档

支持信息

• 商业咨询和科学咨询：请通过 邮箱 联系我。
• 技术问题和错误报告：请查看 pyannote.audio 的Github仓库，在 讨论区 提出技术问题，在 问题区 报告错误。

📄 许可证

本项目采用MIT许可证。

🔖 引用信息

@inproceedings{Bredin2021,
  Title = {{End-to-end speaker segmentation for overlap-aware resegmentation}},
  Author = {{Bredin}, Herv{\'e} and {Laurent}, Antoine},
  Booktitle = {Proc. Interspeech 2021},
  Address = {Brno, Czech Republic},
  Month = {August},
  Year = {2021},

@inproceedings{Bredin2020,
  Title = {{pyannote.audio: neural building blocks for speaker diarization}},
  Author = {{Bredin}, Herv{\'e} and {Yin}, Ruiqing and {Coria}, Juan Manuel and {Gelly}, Gregory and {Korshunov}, Pavel and {Lavechin}, Marvin and {Fustes}, Diego and {Titeux}, Hadrien and {Bouaziz}, Wassim and {Gill}, Marie-Philippe},
  Booktitle = {ICASSP 2020, IEEE International Conference on Acoustics, Speech, and Signal Processing},
  Address = {Barcelona, Spain},
  Month = {May},
  Year = {2020},
}