đ bert-base-uncased-qqp
This model is a fine - tuned version of [bert - base - uncased](https://huggingface.co/bert - base - uncased) on the GLUE QQP dataset. It can effectively perform tasks such as text classification and natural language inference, providing high - accuracy results on the evaluation set.
đ Quick Start
This section provides a brief introduction to the model. The bert - base - uncased - qqp model is fine - tuned from the bert - base - uncased model on the GLUE QQP dataset. It achieves the following results on the evaluation set:
- Loss: 0.2829
- Accuracy: 0.9100
- F1: 0.8788
- Combined Score: 0.8944
⨠Features
- Fine - tuned from BERT: Based on the powerful
bert - base - uncased model, it is further optimized for the GLUE QQP dataset.
- High performance: Achieves excellent results in accuracy, F1 score, etc. on the evaluation set.
- Multiple tasks: Suitable for tasks such as text classification and natural language inference.
đĻ Installation
No specific installation steps are provided in the original document, so this section is skipped.
đģ Usage Examples
No code examples are provided in the original document, so this section is skipped.
đ Documentation
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e - 05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
- lr_scheduler_type: linear
- num_epochs: 3.0
Training results
| Training Loss |
Epoch |
Step |
Validation Loss |
Accuracy |
F1 |
Combined Score |
| 0.2511 |
1.0 |
11371 |
0.2469 |
0.8969 |
0.8641 |
0.8805 |
| 0.1763 |
2.0 |
22742 |
0.2379 |
0.9071 |
0.8769 |
0.8920 |
| 0.1221 |
3.0 |
34113 |
0.2829 |
0.9100 |
0.8788 |
0.8944 |
Framework versions
- Transformers 4.20.0.dev0
- Pytorch 1.11.0+cu113
- Datasets 2.1.0
- Tokenizers 0.12.1
đ§ Technical Details
The model is fine - tuned on the GLUE QQP dataset based on the bert - base - uncased model. During training, specific hyperparameters are used, and the training results show that the model's performance gradually improves with the increase of epochs.
đ License
This model is released under the Apache 2.0 license.
đ Model Index
| Property |
Details |
| Model Name |
bert - base - uncased - qqp |
| Base Model |
bert - base - uncased |
| Task Type |
Text Classification, Natural Language Inference |
| Dataset |
GLUE QQP |
| Metrics |
Accuracy, F1, Precision, Recall, AUC, Loss |
| Accuracy |
0.9099925797674994 (in some cases), 0.9100 (evaluation set) |
| F1 |
0.8788252139455897 (in some cases), 0.8788 (evaluation set) |
| Loss |
0.28284332156181335 (in some cases), 0.2829 (evaluation set) |
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
