Whisper Ner V1

🚀 Whisper-NER

Whisper-NER是一款创新模型，可实现语音转录和实体识别的联合处理。它支持开放式命名实体识别，能在推理时识别多样化且不断演变的实体。该模型旨在作为带有命名实体识别（NER）的自动语音识别（ASR）下游任务的强大基础模型，还可在特定数据集上进行微调以提升性能。

🚀 快速开始

• 演示地址：https://huggingface.co/spaces/aiola/whisper-ner-v1
• 论文：WhisperNER: Unified Open Named Entity and Speech Recognition
• 代码仓库：https://github.com/aiola-lab/whisper-ner

✨ 主要特性

• 支持语音转录和实体识别的联合处理。
• 能够进行开放式命名实体识别，可识别多样化且不断演变的实体。
• 可作为带有NER的ASR下游任务的强大基础模型，并可在特定数据集上微调以提升性能。

📦 安装指南

文档未提及安装步骤，故跳过该章节。

💻 使用示例

基础用法

import torch
from transformers import WhisperProcessor, WhisperForConditionalGeneration

model_path = "aiola/whisper-ner-v1"
audio_file_path = "path/to/audio/file"
prompt = "person, company, location"  # comma separated entity tags
    
# load model and processor from pre-trained
processor = WhisperProcessor.from_pretrained(model_path)
model = WhisperForConditionalGeneration.from_pretrained(model_path)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

# load audio file: user is responsible for loading the audio files themselves
target_sample_rate = 16000
signal, sampling_rate = torchaudio.load(audio_file_path)
resampler = torchaudio.transforms.Resample(sampling_rate, target_sample_rate)
signal = resampler(signal)
# convert to mono or remove first dim if needed
if signal.ndim == 2:
    signal = torch.mean(signal, dim=0)
# pre-process to get the input features
input_features = processor(
    signal, sampling_rate=target_sample_rate, return_tensors="pt"
).input_features
input_features = input_features.to(device)

prompt_ids = processor.get_prompt_ids(prompt.lower(), return_tensors="pt")
prompt_ids = prompt_ids.to(device)

# generate token ids by running model forward sequentially
with torch.no_grad():
    predicted_ids = model.generate(
        input_features,
        prompt_ids=prompt_ids,
        generation_config=model.generation_config,
        language="en",
    )

# post-process token ids to text, remove prompt
transcription = processor.batch_decode(
    predicted_ids, skip_special_tokens=True
)[0]
print(transcription)

高级用法

文档未提及高级用法相关代码，故跳过该部分。

📚 详细文档

训练详情

aiola/whisper-ner-v1 在NuNER数据集上进行训练，以执行联合音频转录和命名实体识别（NER）标记。该模型仅在英文数据上进行训练和评估。如需了解完整详情，请查看论文。

🔧 技术细节

文档未提供足够的技术实现细节（少于50字），故跳过该章节。

📄 许可证

本项目采用MIT许可证。