Bert Restore Punctuation

🚀 ✨ bert-restore-punctuation

这是一个基于BERT的模型，经过微调后可用于恢复英文文本的标点和大小写。它在Yelp评论数据集上进行训练，能够处理各种失去标点的英文文本，例如语音识别（ASR）的输出。该模型可直接用于通用英文文本的标点恢复，也可针对特定领域文本进行进一步微调。

🚀 快速开始

以下是快速使用该模型的方法：

1. 首先，安装所需的包。

pip install rpunct

2. 示例Python代码。

from rpunct import RestorePuncts
# 默认为英文
rpunct = RestorePuncts()
rpunct.punctuate("""in 2018 cornell researchers built a high-powered detector that in combination with an algorithm-driven process called ptychography set a world record
by tripling the resolution of a state-of-the-art electron microscope as successful as it was that approach had a weakness it only worked with ultrathin samples that were
a few atoms thick anything thicker would cause the electrons to scatter in ways that could not be disentangled now a team again led by david muller the samuel b eckert
professor of engineering has bested its own record by a factor of two with an electron microscope pixel array detector empad that incorporates even more sophisticated
3d reconstruction algorithms the resolution is so fine-tuned the only blurring that remains is the thermal jiggling of the atoms themselves""")
# 输出如下：
# In 2018, Cornell researchers built a high-powered detector that, in combination with an algorithm-driven process called Ptychography, set a world record by tripling the
# resolution of a state-of-the-art electron microscope. As successful as it was, that approach had a weakness. It only worked with ultrathin samples that were a few atoms
# thick. Anything thicker would cause the electrons to scatter in ways that could not be disentangled. Now, a team again led by David Muller, the Samuel B. 
# Eckert Professor of Engineering, has bested its own record by a factor of two with an Electron microscope pixel array detector empad that incorporates even more
# sophisticated 3d reconstruction algorithms. The resolution is so fine-tuned the only blurring that remains is the thermal jiggling of the atoms themselves.

该模型可以处理任意长度的英文文本，并在可用时使用GPU加速。

✨ 主要特性

• 该模型可预测纯小写文本的标点和大小写，适用于语音识别输出或其他失去标点的文本场景。
• 可直接用于通用英文文本的标点恢复，也可针对特定领域文本进行进一步微调。
• 能够恢复以下标点符号 -- [! ? . , - : ; ' ] ，并恢复单词的首字母大写。

📦 安装指南

pip install rpunct

💻 使用示例

基础用法

from rpunct import RestorePuncts
# 默认为英文
rpunct = RestorePuncts()
rpunct.punctuate("""in 2018 cornell researchers built a high-powered detector that in combination with an algorithm-driven process called ptychography set a world record
by tripling the resolution of a state-of-the-art electron microscope as successful as it was that approach had a weakness it only worked with ultrathin samples that were
a few atoms thick anything thicker would cause the electrons to scatter in ways that could not be disentangled now a team again led by david muller the samuel b eckert
professor of engineering has bested its own record by a factor of two with an electron microscope pixel array detector empad that incorporates even more sophisticated
3d reconstruction algorithms the resolution is so fine-tuned the only blurring that remains is the thermal jiggling of the atoms themselves""")
# 输出如下：
# In 2018, Cornell researchers built a high-powered detector that, in combination with an algorithm-driven process called Ptychography, set a world record by tripling the
# resolution of a state-of-the-art electron microscope. As successful as it was, that approach had a weakness. It only worked with ultrathin samples that were a few atoms
# thick. Anything thicker would cause the electrons to scatter in ways that could not be disentangled. Now, a team again led by David Muller, the Samuel B. 
# Eckert Professor of Engineering, has bested its own record by a factor of two with an Electron microscope pixel array detector empad that incorporates even more
# sophisticated 3d reconstruction algorithms. The resolution is so fine-tuned the only blurring that remains is the thermal jiggling of the atoms themselves.

📚 详细文档

训练数据

以下是用于微调模型的产品评论数量：

我们发现模型在大约 3 个训练周期 时达到最佳收敛效果。

准确率

微调后的模型在 45,990 个保留文本样本上的准确率如下：

以下是模型按每个标签的性能细分：

☕ 联系我们

如有任何问题、反馈或需要类似模型，请联系 Daulet Nurmanbetov。

📄 许可证

本项目采用 MIT 许可证。