đ distilbert-base-turkish-cased_allnli_tr
This model is a fine - tuned version of dbmdz/distilbert-base-turkish-cased on the None dataset. It's designed for zero - shot classification tasks and achieves an accuracy of 0.7381 on the evaluation set.
đ Quick Start
This model can be used for zero - shot classification tasks. You can load it using the Hugging Face Transformers library and start making predictions.
⨠Features
- Zero - Shot Classification: Capable of classifying text into predefined labels without explicit training on those labels.
- Fine - Tuned on Turkish Data: Trained on Turkish datasets, making it suitable for Turkish language tasks.
đ Documentation
Model Performance
It achieves the following results on the evaluation set:
- Loss: 0.6481
- Accuracy: 0.7381
Training and Evaluation Data
The model is fine - tuned on the nli_tr
dataset.
Training Procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e - 05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
- lr_scheduler_type: linear
- num_epochs: 3
Training results
Training Loss |
Epoch |
Step |
Validation Loss |
Accuracy |
0.94 |
0.03 |
1000 |
0.9074 |
0.5813 |
0.8102 |
0.07 |
2000 |
0.8802 |
0.5949 |
0.7737 |
0.1 |
3000 |
0.8491 |
0.6155 |
0.7576 |
0.14 |
4000 |
0.8283 |
0.6261 |
0.7286 |
0.17 |
5000 |
0.8150 |
0.6362 |
0.7162 |
0.2 |
6000 |
0.7998 |
0.6400 |
0.7092 |
0.24 |
7000 |
0.7830 |
0.6565 |
0.6962 |
0.27 |
8000 |
0.7653 |
0.6629 |
0.6876 |
0.31 |
9000 |
0.7630 |
0.6687 |
0.6778 |
0.34 |
10000 |
0.7475 |
0.6739 |
0.6737 |
0.37 |
11000 |
0.7495 |
0.6781 |
0.6712 |
0.41 |
12000 |
0.7350 |
0.6826 |
0.6559 |
0.44 |
13000 |
0.7274 |
0.6897 |
0.6493 |
0.48 |
14000 |
0.7248 |
0.6902 |
0.6483 |
0.51 |
15000 |
0.7263 |
0.6858 |
0.6445 |
0.54 |
16000 |
0.7070 |
0.6978 |
0.6467 |
0.58 |
17000 |
0.7083 |
0.6981 |
0.6332 |
0.61 |
18000 |
0.6996 |
0.7004 |
0.6288 |
0.65 |
19000 |
0.6979 |
0.6978 |
0.6308 |
0.68 |
20000 |
0.6912 |
0.7040 |
0.622 |
0.71 |
21000 |
0.6904 |
0.7092 |
0.615 |
0.75 |
22000 |
0.6872 |
0.7094 |
0.6186 |
0.78 |
23000 |
0.6877 |
0.7075 |
0.6183 |
0.82 |
24000 |
0.6818 |
0.7111 |
0.6115 |
0.85 |
25000 |
0.6856 |
0.7122 |
0.608 |
0.88 |
26000 |
0.6697 |
0.7179 |
0.6071 |
0.92 |
27000 |
0.6727 |
0.7181 |
0.601 |
0.95 |
28000 |
0.6798 |
0.7118 |
0.6018 |
0.99 |
29000 |
0.6854 |
0.7071 |
0.5762 |
1.02 |
30000 |
0.6697 |
0.7214 |
0.5507 |
1.05 |
31000 |
0.6710 |
0.7185 |
0.5575 |
1.09 |
32000 |
0.6709 |
0.7226 |
0.5493 |
1.12 |
33000 |
0.6659 |
0.7191 |
0.5464 |
1.15 |
34000 |
0.6709 |
0.7232 |
0.5595 |
1.19 |
35000 |
0.6642 |
0.7220 |
0.5446 |
1.22 |
36000 |
0.6709 |
0.7202 |
0.5524 |
1.26 |
37000 |
0.6751 |
0.7148 |
0.5473 |
1.29 |
38000 |
0.6642 |
0.7209 |
0.5477 |
1.32 |
39000 |
0.6662 |
0.7223 |
0.5522 |
1.36 |
40000 |
0.6586 |
0.7227 |
0.5406 |
1.39 |
41000 |
0.6602 |
0.7258 |
0.54 |
1.43 |
42000 |
0.6564 |
0.7273 |
0.5458 |
1.46 |
43000 |
0.6780 |
0.7213 |
0.5448 |
1.49 |
44000 |
0.6561 |
0.7235 |
0.5418 |
1.53 |
45000 |
0.6600 |
0.7253 |
0.5408 |
1.56 |
46000 |
0.6616 |
0.7274 |
0.5451 |
1.6 |
47000 |
0.6557 |
0.7283 |
0.5385 |
1.63 |
48000 |
0.6583 |
0.7295 |
0.5261 |
1.66 |
49000 |
0.6468 |
0.7325 |
0.5364 |
1.7 |
50000 |
0.6447 |
0.7329 |
0.5294 |
1.73 |
51000 |
0.6429 |
0.7320 |
0.5332 |
1.77 |
52000 |
0.6508 |
0.7272 |
0.5274 |
1.8 |
53000 |
0.6492 |
0.7326 |
0.5286 |
1.83 |
54000 |
0.6470 |
0.7318 |
0.5359 |
1.87 |
55000 |
0.6393 |
0.7354 |
0.5366 |
1.9 |
56000 |
0.6445 |
0.7367 |
0.5296 |
1.94 |
57000 |
0.6413 |
0.7313 |
0.5346 |
1.97 |
58000 |
0.6393 |
0.7315 |
0.5264 |
2.0 |
59000 |
0.6448 |
0.7357 |
0.4857 |
2.04 |
60000 |
0.6640 |
0.7335 |
0.4888 |
2.07 |
61000 |
0.6612 |
0.7318 |
0.4964 |
2.11 |
62000 |
0.6516 |
0.7337 |
0.493 |
2.14 |
63000 |
0.6503 |
0.7356 |
0.4961 |
2.17 |
64000 |
0.6519 |
0.7348 |
0.4847 |
2.21 |
65000 |
0.6517 |
0.7327 |
0.483 |
2.24 |
66000 |
0.6555 |
0.7310 |
0.4857 |
2.28 |
67000 |
0.6525 |
0.7312 |
0.484 |
2.31 |
68000 |
0.6444 |
0.7342 |
0.4792 |
2.34 |
69000 |
0.6508 |
0.7330 |
0.488 |
2.38 |
70000 |
0.6513 |
0.7344 |
0.472 |
2.41 |
71000 |
0.6547 |
0.7346 |
0.4872 |
2.45 |
72000 |
0.6500 |
0.7342 |
0.4782 |
2.48 |
73000 |
0.6585 |
0.7358 |
0.481 |
2.51 |
74000 |
0.6477 |
0.7356 |
0.4822 |
2.55 |
75000 |
0.6587 |
0.7346 |
0.4728 |
2.58 |
76000 |
0.6572 |
0.7340 |
0.4841 |
2.62 |
77000 |
0.6443 |
0.7374 |
0.4885 |
2.65 |
78000 |
0.6494 |
0.7362 |
0.4752 |
2.68 |
79000 |
0.6509 |
0.7382 |
0.4883 |
2.72 |
80000 |
0.6457 |
0.7371 |
0.4888 |
2.75 |
81000 |
0.6497 |
0.7364 |
0.4844 |
2.79 |
82000 |
0.6481 |
0.7376 |
0.4833 |
2.82 |
83000 |
0.6451 |
0.7389 |
0.48 |
2.85 |
84000 |
0.6423 |
0.7373 |
0.4832 |
2.89 |
85000 |
0.6477 |
0.7357 |
0.4805 |
2.92 |
86000 |
0.6464 |
0.7379 |
0.4775 |
2.96 |
87000 |
0.6477 |
0.7380 |
0.4843 |
2.99 |
88000 |
0.6481 |
0.7381 |
Framework versions
- Transformers 4.12.3
- Pytorch 1.10.0+cu102
- Datasets 1.15.1
- Tokenizers 0.10.3
đ License
This model is released under the Apache 2.0 license.