🚀 ALBERT Persian
A Lite BERT for Self-supervised Learning of Language Representations for the Persian Language
ALBERT-Persian is the first attempt to apply ALBERT to the Persian language. Based on Google's ALBERT BASE Version 2.0, this model was trained on over 3.9 million documents, 73 million sentences, and 1.3 billion words from various writing styles and subjects (such as scientific, novels, and news), similar to the approach used for ParsBERT.
Please follow the ALBERT-Persian repository for the latest information on previous and current models.
🚀 Quick Start
Persian Text Classification [DigiMag, Persian News]
The goal of this task is to label texts in a supervised manner for both the DigiMag and Persian News datasets.
Persian News
This dataset consists of various news articles scraped from different online news agencies' websites. There are a total of 16,438 articles, divided into eight different classes:
- Economic
- International
- Political
- Science Technology
- Cultural Art
- Sport
- Medical
| Label |
# |
| Social |
2170 |
| Economic |
1564 |
| International |
1975 |
| Political |
2269 |
| Science Technology |
2436 |
| Cultural Art |
2558 |
| Sport |
1381 |
| Medical |
2085 |
Download
You can download the dataset from here
✨ Features
Results
The following table summarizes the F1 score compared to other models and architectures:
| Dataset |
ALBERT-fa-base-v2 |
ParsBERT-v1 |
mBERT |
| Persian News |
97.01 |
97.19 |
95.79 |
📄 License
This project is licensed under the Apache-2.0 license.
BibTeX entry and citation info
Please cite in publications as the following:
@misc{ALBERTPersian,
author = {Mehrdad Farahani},
title = {ALBERT-Persian: A Lite BERT for Self-supervised Learning of Language Representations for the Persian Language},
year = {2020},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/m3hrdadfi/albert-persian}},
}
@article{ParsBERT,
title={ParsBERT: Transformer-based Model for Persian Language Understanding},
author={Mehrdad Farahani, Mohammad Gharachorloo, Marzieh Farahani, Mohammad Manthouri},
journal={ArXiv},
year={2020},
volume={abs/2005.12515}
}
💡 Questions?
Post a Github issue on the ALBERT-Persian repo.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
