Mms 300m 1130 Forced Aligner

🚀 基於Hugging Face CTC模型的強制對齊工具

本Python包提供了一種高效的方法，利用Hugging Face的預訓練模型在文本和音頻之間執行強制對齊。此外，它還採用了改進的實現方式，相比TorchAudio的強制對齊API，能顯著減少內存使用。

這裡上傳的模型檢查點是將基於強制對齊數據集訓練的MMS - 300M檢查點從torchaudio轉換為HF Transformers格式的版本。

🚀 快速開始

本工具可幫助你高效地完成文本和音頻的強制對齊任務，減少內存消耗。

📦 安裝指南

你可以使用以下命令安裝本工具：

pip install git+https://github.com/MahmoudAshraf97/ctc-forced-aligner.git

💻 使用示例

基礎用法

import torch
from ctc_forced_aligner import (
    load_audio,
    load_alignment_model,
    generate_emissions,
    preprocess_text,
    get_alignments,
    get_spans,
    postprocess_results,
)

audio_path = "your/audio/path"
text_path = "your/text/path"
language = "iso" # ISO-639-3 Language code
device = "cuda" if torch.cuda.is_available() else "cpu"
batch_size = 16


alignment_model, alignment_tokenizer = load_alignment_model(
    device,
    dtype=torch.float16 if device == "cuda" else torch.float32,
)

audio_waveform = load_audio(audio_path, alignment_model.dtype, alignment_model.device)


with open(text_path, "r") as f:
    lines = f.readlines()
text = "".join(line for line in lines).replace("\n", " ").strip()

emissions, stride = generate_emissions(
    alignment_model, audio_waveform, batch_size=batch_size
)

tokens_starred, text_starred = preprocess_text(
    text,
    romanize=True,
    language=language,
)

segments, scores, blank_token = get_alignments(
    emissions,
    tokens_starred,
    alignment_tokenizer,
)

spans = get_spans(tokens_starred, segments, blank_token)

word_timestamps = postprocess_results(text_starred, spans, stride, scores)

📄 許可證

本項目採用CC - BY - NC - 4.0許可證。

🔍 詳細信息