Banglat5 Small

🚀 孟加拉語T5模型（BanglaT5）

本倉庫包含了孟加拉語T5模型（小版本） 的預訓練檢查點。這是一個基於序列到序列的Transformer模型，通過"Span Corruption"目標進行預訓練。使用此檢查點進行微調的模型在許多孟加拉語自然語言生成（NLG）任務中取得了最先進的成果。

該模型可用於不同的下游任務，如機器翻譯、抽象文本摘要、問答系統等。若要進行微調，請參考官方GitHub 倉庫 中的腳本。

⚠️ 重要提示

該模型使用了特定的歸一化管道進行預訓練，可在 此處 獲取。官方GitHub倉庫中的所有微調腳本默認使用此歸一化方法。如果需要將預訓練模型應用於其他任務，請確保在分詞前使用此管道對文本單元進行歸一化，以獲得最佳效果。下面給出一個基本示例。

🚀 快速開始

✨ 主要特性

• 基於序列到序列的Transformer架構，適用於多種孟加拉語NLG任務。
• 預訓練模型在多個NLG任務中達到了最先進的性能。

📦 安裝指南

在使用該模型之前，你需要安裝必要的庫。可以使用以下命令安裝歸一化庫：

pip install git+https://github.com/csebuetnlp/normalizer

💻 使用示例

基礎用法

以下是在transformers庫中使用該模型的示例代碼（測試版本為4.11.0.dev0）：

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from normalizer import normalize # pip install git+https://github.com/csebuetnlp/normalizer

model = AutoModelForSeq2SeqLM.from_pretrained("csebuetnlp/banglat5_small")
tokenizer = AutoTokenizer.from_pretrained("csebuetnlp/banglat5_small", use_fast=False)

input_sentence = ""
input_ids = tokenizer(normalize(input_sentence), return_tensors="pt").input_ids
generated_tokens = model.generate(input_ids)
decoded_tokens = tokenizer.batch_decode(generated_tokens)[0]

print(decoded_tokens)

📚 詳細文檔

🔍 基準測試

以下是模型在監督微調下的基準測試結果：

基準測試數據集如下：

• 機器翻譯（MT）：數據集鏈接
• 抽象文本摘要（TS）：數據集鏈接
• 問答系統（QA）：數據集鏈接
• 多輪對話生成（MD）：數據集鏈接
• 新聞標題生成（NHG）：數據集鏈接
• 跨語言摘要（XLS）：數據集鏈接

📄 許可證

本項目採用CC - BY - NC - SA 4.0許可證。

📚 引用說明

如果你使用了該模型，請引用以下論文：

@article{bhattacharjee2022banglanlg,
  author    = {Abhik Bhattacharjee and Tahmid Hasan and Wasi Uddin Ahmad and Rifat Shahriyar},
  title     = {BanglaNLG: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Bangla},
  journal   = {CoRR},
  volume    = {abs/2205.11081},
  year      = {2022},
  url       = {https://arxiv.org/abs/2205.11081},
  eprinttype = {arXiv},
  eprint    = {2205.11081}
}

如果你使用了歸一化模塊，請引用以下論文：

@inproceedings{hasan-etal-2020-low,
    title = "Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for {B}engali-{E}nglish Machine Translation",
    author = "Hasan, Tahmid  and
      Bhattacharjee, Abhik  and
      Samin, Kazi  and
      Hasan, Masum  and
      Basak, Madhusudan  and
      Rahman, M. Sohel  and
      Shahriyar, Rifat",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.emnlp-main.207",
    doi = "10.18653/v1/2020.emnlp-main.207",
    pages = "2612--2623",
    abstract = "Despite being the seventh most widely spoken language in the world, Bengali has received much less attention in machine translation literature due to being low in resources. Most publicly available parallel corpora for Bengali are not large enough; and have rather poor quality, mostly because of incorrect sentence alignments resulting from erroneous sentence segmentation, and also because of a high volume of noise present in them. In this work, we build a customized sentence segmenter for Bengali and propose two novel methods for parallel corpus creation on low-resource setups: aligner ensembling and batch filtering. With the segmenter and the two methods combined, we compile a high-quality Bengali-English parallel corpus comprising of 2.75 million sentence pairs, more than 2 million of which were not available before. Training on neural models, we achieve an improvement of more than 9 BLEU score over previous approaches to Bengali-English machine translation. We also evaluate on a new test set of 1000 pairs made with extensive quality control. We release the segmenter, parallel corpus, and the evaluation set, thus elevating Bengali from its low-resource status. To the best of our knowledge, this is the first ever large scale study on Bengali-English machine translation. We believe our study will pave the way for future research on Bengali-English machine translation as well as other low-resource languages. Our data and code are available at https://github.com/csebuetnlp/banglanmt.",
}