🚀 T5-large-spell model
This model corrects spelling errors and typos, standardizing all words in the text to proper English. It is trained based on the T5-large model. The training corpus consists of an extensive dataset with “artificial” errors, assembled from English Wikipedia and news blogs, and then typos and spelling mistakes were automatically introduced using the SAGE library.
🚀 Quick Start
from transformers import T5ForConditionalGeneration, AutoTokenizer
path_to_model = "ai-forever/T5-large-spell"
model = T5ForConditionalGeneration.from_pretrained(path_to_model)
tokenizer = AutoTokenizer.from_pretrained(path_to_model)
prefix = "grammar: "
sentence = "If you bought something goregous, you well be very happy."
sentence = prefix + sentence
encodings = tokenizer(sentence, return_tensors="pt")
generated_tokens = model.generate(**encodings)
answer = tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
print(answer)
✨ Features
- Corrects spelling errors and typos in English text.
- Trained on a large dataset with artificial errors.
📚 Documentation
Summary
The model corrects spelling errors and typos by bringing all words in the text to the standard English language. The proofreader was trained based on the T5-large model. An extensive dataset with “artificial” errors was taken as a training corpus: the corpus was assembled on the basis of the English-language Wikipedia and News blogs, then typos and spelling errors were automatically introduced into it using the functionality of the SAGE library.
Public references
Examples
| Input |
Output |
| Th festeivаl was excelzecnt in many ways, and in particular it beinganinternational festjival sss a chаllenging, bet brilli an t ea. |
The festival was excellent in many ways, and in particular it beinganinternational festival is a challenging, but brilliant one to see. |
| That 's why I believe in the solution which is the closest to human nature and can help us to avoid boredome. I am sure that eventually we will take off our clothes and in the future we will be undressed and free. There wo n't be any problem with being up - do - date . |
That's why I believe in the solution which is the closest to human nature and can help us to avoid boredom. I am sure that eventually we will take off our clothes and in the future we will be undressed and free. There won't be any problem with being up - do - date. |
| If you bought something goregous, you well be very happy. |
If you bought something gorgeous, you will be very happy. |
🔧 Technical Details
Metrics
Quality
Below are automatic metrics for determining the correctness of the spell checkers. We present a comparison of our solution both with open automatic spell checkers and with the ChatGPT family of models on two available datasets:
- BEA60K: English spelling errors collected from several domains;
- JFLEG: 1601 sentences in English, which contain about 2 thousand spelling errors;
BEA60K
| Model |
Precision |
Recall |
F1 |
| T5-large-spell |
66.5 |
83.1 |
73.9 |
| ChatGPT gpt-3.5-turbo-0301 |
66.9 |
84.1 |
74.5 |
| ChatGPT gpt-4-0314 |
68.6 |
85.2 |
76.0 |
| ChatGPT text-davinci-003 |
67.8 |
83.9 |
75.0 |
| Bert (https://github.com/neuspell/neuspell) |
65.8 |
79.6 |
72.0 |
| SC-LSTM (https://github.com/neuspell/neuspell) |
62.2 |
80.3 |
72.0 |
JFLEG
| Model |
Precision |
Recall |
F1 |
| T5-large-spell |
83.4 |
84.3 |
83.8 |
| ChatGPT gpt-3.5-turbo-0301 |
77.8 |
88.6 |
82.9 |
| ChatGPT gpt-4-0314 |
77.9 |
88.3 |
82.8 |
| ChatGPT text-davinci-003 |
76.8 |
88.5 |
82.2 |
| Bert (https://github.com/neuspell/neuspell) |
78.5 |
85.4 |
81.8 |
| SC-LSTM (https://github.com/neuspell/neuspell) |
80.6 |
86.1 |
83.2 |
📦 Resources
📄 License
The T5-large model, on which our solution is based, and its source code are supplied under the APACHE-2.0 license. Our solution is supplied under MIT license.
📋 Specifications
| Property |
Details |
| File size |
3 Gb |
| Framework |
pytorch |
| Format |
AI Service |
| Version |
v1.0 |
| Developer |
SberDevices, AGI NLP |
📞 Contacts
nikita.martynov.98@list.ru
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
