Zabanshenas Roberta Base Mix

🚀 Zabanshenas - Language Detector

Zabanshenas is a Transformer-based solution designed to identify the most likely language of a written document or text. The term "Zabanshenas" is Persian, with two meanings:

• A person who studies linguistics.
• A method to identify the type of written language.

🚀 Quick Start

Follow Zabanshenas repo for more information!

✨ Features

Zabanshenas can handle a wide range of languages, including but not limited to:

• multilingual, ace, afr, als, amh, ang, ara, arg, arz, asm, ast, ava, aym, azb, aze, bak, bar, bcl, bel, ben, bho, bjn, bod, bos, bpy, bre, bul, bxr, cat, cbk, cdo, ceb, ces, che, chr, chv, ckb, cor, cos, crh, csb, cym, dan, deu, diq, div, dsb, dty, egl, ell, eng, epo, est, eus, ext, fao, fas, fin, fra, frp, fry, fur, gag, gla, gle, glg, glk, glv, grn, guj, hak, hat, hau, hbs, heb, hif, hin, hrv, hsb, hun, hye, ibo, ido, ile, ilo, ina, ind, isl, ita, jam, jav, jbo, jpn, kaa, kab, kan, kat, kaz, kbd, khm, kin, kir, koi, kok, kom, kor, krc, ksh, kur, lad, lao, lat, lav, lez, lij, lim, lin, lit, lmo, lrc, ltg, ltz, lug, lzh, mai, mal, mar, mdf, mhr, min, mkd, mlg, mlt, nan, mon, mri, mrj, msa, mwl, mya, myv, mzn, nap, nav, nci, nds, nep, new, nld, nno, nob, nrm, nso, oci, olo, ori, orm, oss, pag, pam, pan, pap, pcd, pdc, pfl, pnb, pol, por, pus, que, roh, ron, rue, rup, rus, sah, san, scn, sco, sgs, sin, slk, slv, sme, sna, snd, som, spa, sqi, srd, srn, srp, stq, sun, swa, swe, szl, tam, tat, tcy, tel, tet, tgk, tgl, tha, ton, tsn, tuk, tur, tyv, udm, uig, ukr, urd, uzb, vec, vep, vie, vls, vol, vro, war, wln, wol, wuu, xho, xmf, yid, yor, zea, zho

It also supports languages in BCP 47 format:

• be-tarask, map-bms, nds-nl, roa-tara, zh-yue

📚 Documentation

Evaluation

The following tables summarize the scores obtained by the model overall and per each class.

By Paragraph

📄 License

This project is licensed under the apache - 2.0 license.