Whisper Small Korean Pronunciation Scorer Sampledata

🚀 微调版Whisper发音评分器

该模型用于评估韩语语音的发音质量，基于openai/whisper-small模型，使用韩国AI-Hub（https://www.aihub.or.kr/）的外国人韩语发音评估数据集进行微调。

🚀 快速开始

本发音评分器接收音频输入及其对应的文本转录，并提供1到5分的韩语发音评分。它利用Whisper模型的编码器 - 解码器架构提取语音特征，并使用一个额外的线性层来预测发音分数。

✨ 主要特性

• 专门针对韩语语音进行发音质量评估。
• 基于openai/whisper-small模型微调，利用其编码器 - 解码器架构提取特征。
• 提供1到5分的发音评分。

📦 安装指南

使用此模型，需按以下步骤操作：

1. 安装所需库
2. 加载模型和处理器
3. 准备音频文件和文本转录
4. 预测发音分数

💻 使用示例

基础用法

import torch
import torchaudio
from transformers import WhisperProcessor, WhisperForConditionalGeneration
import torch.nn as nn

class WhisperPronunciationScorer(nn.Module):
    def __init__(self, pretrained_model):
        super().__init__()
        self.whisper = pretrained_model
        self.score_head = nn.Linear(self.whisper.config.d_model, 1)

    def forward(self, input_features, labels=None):
        outputs = self.whisper(input_features, labels=labels, output_hidden_states=True)
        last_hidden_state = outputs.decoder_hidden_states[-1]
        scores = self.score_head(last_hidden_state.mean(dim=1)).squeeze()
        return scores

def load_model(model_path, device):
    model_name = "openai/whisper-small"
    processor = WhisperProcessor.from_pretrained(model_name)
    pretrained_model = WhisperForConditionalGeneration.from_pretrained(model_name)
    model = WhisperPronunciationScorer(pretrained_model).to(device)
    model.load_state_dict(torch.load(model_path, map_location=device))
    model.eval()
    return model, processor

def predict_pronunciation_score(model, processor, audio_path, transcript, device):
    # Load and preprocess audio
    audio, sr = torchaudio.load(audio_path)
    if sr != 16000:
        audio = torchaudio.functional.resample(audio, sr, 16000)
    input_features = processor(audio.squeeze().numpy(), sampling_rate=16000, return_tensors="pt").input_features.to(device)
    
    # Prepare transcript
    labels = processor(text=transcript, return_tensors="pt").input_ids.to(device)
    
    # Predict score
    with torch.no_grad():
        score = model(input_features, labels)
    return score.item()

# Load model
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model_path = "path/to/your/model.pth"
model, processor = load_model(model_path, device)

# Run prediction
audio_path = "path/to/your/audio.wav"
transcript = "안녕하세요"
score = predict_pronunciation_score(model, processor, audio_path, transcript, device)
print(f"Predicted pronunciation score: {score:.2f}")

高级用法

上述代码已经包含了完整的使用流程，可根据实际需求对代码进行修改，例如调整模型路径、音频文件路径和文本转录内容等。

📄 许可证

本项目采用Apache-2.0许可证。

📚 详细文档