Bert Restore Punctuation

🚀 ✨ bert-restore-punctuation

這是一個基於BERT的模型，經過微調後可用於恢復英文文本的標點和大小寫。它在Yelp評論數據集上進行訓練，能夠處理各種失去標點的英文文本，例如語音識別（ASR）的輸出。該模型可直接用於通用英文文本的標點恢復，也可針對特定領域文本進行進一步微調。

🚀 快速開始

以下是快速使用該模型的方法：

1. 首先，安裝所需的包。

pip install rpunct

2. 示例Python代碼。

from rpunct import RestorePuncts
# 默認為英文
rpunct = RestorePuncts()
rpunct.punctuate("""in 2018 cornell researchers built a high-powered detector that in combination with an algorithm-driven process called ptychography set a world record
by tripling the resolution of a state-of-the-art electron microscope as successful as it was that approach had a weakness it only worked with ultrathin samples that were
a few atoms thick anything thicker would cause the electrons to scatter in ways that could not be disentangled now a team again led by david muller the samuel b eckert
professor of engineering has bested its own record by a factor of two with an electron microscope pixel array detector empad that incorporates even more sophisticated
3d reconstruction algorithms the resolution is so fine-tuned the only blurring that remains is the thermal jiggling of the atoms themselves""")
# 輸出如下：
# In 2018, Cornell researchers built a high-powered detector that, in combination with an algorithm-driven process called Ptychography, set a world record by tripling the
# resolution of a state-of-the-art electron microscope. As successful as it was, that approach had a weakness. It only worked with ultrathin samples that were a few atoms
# thick. Anything thicker would cause the electrons to scatter in ways that could not be disentangled. Now, a team again led by David Muller, the Samuel B. 
# Eckert Professor of Engineering, has bested its own record by a factor of two with an Electron microscope pixel array detector empad that incorporates even more
# sophisticated 3d reconstruction algorithms. The resolution is so fine-tuned the only blurring that remains is the thermal jiggling of the atoms themselves.

該模型可以處理任意長度的英文文本，並在可用時使用GPU加速。

✨ 主要特性

• 該模型可預測純小寫文本的標點和大小寫，適用於語音識別輸出或其他失去標點的文本場景。
• 可直接用於通用英文文本的標點恢復，也可針對特定領域文本進行進一步微調。
• 能夠恢復以下標點符號 -- [! ? . , - : ; ' ] ，並恢復單詞的首字母大寫。

📦 安裝指南

pip install rpunct

💻 使用示例

基礎用法

from rpunct import RestorePuncts
# 默認為英文
rpunct = RestorePuncts()
rpunct.punctuate("""in 2018 cornell researchers built a high-powered detector that in combination with an algorithm-driven process called ptychography set a world record
by tripling the resolution of a state-of-the-art electron microscope as successful as it was that approach had a weakness it only worked with ultrathin samples that were
a few atoms thick anything thicker would cause the electrons to scatter in ways that could not be disentangled now a team again led by david muller the samuel b eckert
professor of engineering has bested its own record by a factor of two with an electron microscope pixel array detector empad that incorporates even more sophisticated
3d reconstruction algorithms the resolution is so fine-tuned the only blurring that remains is the thermal jiggling of the atoms themselves""")
# 輸出如下：
# In 2018, Cornell researchers built a high-powered detector that, in combination with an algorithm-driven process called Ptychography, set a world record by tripling the
# resolution of a state-of-the-art electron microscope. As successful as it was, that approach had a weakness. It only worked with ultrathin samples that were a few atoms
# thick. Anything thicker would cause the electrons to scatter in ways that could not be disentangled. Now, a team again led by David Muller, the Samuel B. 
# Eckert Professor of Engineering, has bested its own record by a factor of two with an Electron microscope pixel array detector empad that incorporates even more
# sophisticated 3d reconstruction algorithms. The resolution is so fine-tuned the only blurring that remains is the thermal jiggling of the atoms themselves.

📚 詳細文檔

訓練數據

以下是用於微調模型的產品評論數量：

我們發現模型在大約 3 個訓練週期 時達到最佳收斂效果。

準確率

微調後的模型在 45,990 個保留文本樣本上的準確率如下：

以下是模型按每個標籤的性能細分：

☕ 聯繫我們

如有任何問題、反饋或需要類似模型，請聯繫 Daulet Nurmanbetov。

📄 許可證

本項目採用 MIT 許可證。