Bert Base Irish Cased V1

🚀 bert-base-irish-cased-v1

gaBERT 是一個基於BERT-base架構的模型，在790萬條愛爾蘭語句子上進行了訓練。如需瞭解更多詳細信息，包括所使用的超參數和預訓練語料庫，請參考我們的論文。

✨ 主要特性

• 基於編碼器的Transformer模型，可用於獲取特徵，以針對愛爾蘭語的下游任務進行微調。

📚 詳細文檔

預期用途與限制

用於預訓練gaBERT的部分數據是從網絡上抓取的，這些數據可能包含存在倫理問題的文本（如偏見、仇恨、成人內容等）。因此，使用gaBERT的下游任務或應用程序應就倫理問題進行全面測試。

訓練超參數

訓練期間使用了以下超參數：

• 優化器：無
• 訓練精度：float32

框架版本

• Transformers 4.20.1
• TensorFlow 2.9.1
• Datasets 2.3.2
• Tokenizers 0.12.1

BibTeX引用和引用信息

如果您在研究中使用了此模型，請考慮引用我們的論文：

@inproceedings{barry-etal-2022-gabert,
    title = "ga{BERT} {---} an {I}rish Language Model",
    author = "Barry, James  and
      Wagner, Joachim  and
      Cassidy, Lauren  and
      Cowap, Alan  and
      Lynn, Teresa  and
      Walsh, Abigail  and
      {\'O} Meachair, M{\'\i}che{\'a}l J.  and
      Foster, Jennifer",
    booktitle = "Proceedings of the Thirteenth Language Resources and Evaluation Conference",
    month = jun,
    year = "2022",
    address = "Marseille, France",
    publisher = "European Language Resources Association",
    url = "https://aclanthology.org/2022.lrec-1.511",
    pages = "4774--4788",
    abstract = "The BERT family of neural language models have become highly popular due to their ability to provide sequences of text with rich context-sensitive token encodings which are able to generalise well to many NLP tasks. We introduce gaBERT, a monolingual BERT model for the Irish language. We compare our gaBERT model to multilingual BERT and the monolingual Irish WikiBERT, and we show that gaBERT provides better representations for a downstream parsing task. We also show how different filtering criteria, vocabulary size and the choice of subword tokenisation model affect downstream performance. We compare the results of fine-tuning a gaBERT model with an mBERT model for the task of identifying verbal multiword expressions, and show that the fine-tuned gaBERT model also performs better at this task. We release gaBERT and related code to the community.",
}

⚠️ 重要提示

用於預訓練gaBERT的部分數據是從網絡上抓取的，可能包含存在倫理問題的文本（如偏見、仇恨、成人內容等）。因此，使用gaBERT的下游任務或應用程序應就倫理問題進行全面測試。