Fnet Base Finetuned Qqp

🚀 fnet-base-finetuned-qqp

This model is a fine - tuned version of google/fnet-base on the GLUE QQP dataset, aiming to compare with bert-base-cased.

🚀 Quick Start

This model is a fine - tuned version of google/fnet-base on the GLUE QQP dataset. It achieves the following results on the evaluation set:

• Loss: 0.3686
• Accuracy: 0.8847
• F1: 0.8466
• Combined Score: 0.8657

The model was fine - tuned to compare google/fnet-base as introduced in this paper against bert-base-cased.

📦 Installation

This model is trained using the run_glue script. The following command was used:

#!/usr/bin/bash

python ../run_glue.py \
  --model_name_or_path google/fnet-base \
  --task_name qqp \
  --do_train \
  --do_eval \
  --max_seq_length 512 \
  --per_device_train_batch_size 16 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --output_dir fnet-base-finetuned-qqp \
  --push_to_hub \
  --hub_strategy all_checkpoints \
  --logging_strategy epoch \
  --save_strategy epoch \
  --evaluation_strategy epoch

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3.0

Training results

Framework versions

• Transformers 4.11.0.dev0
• Pytorch 1.9.0
• Datasets 1.12.1
• Tokenizers 0.10.3

📚 Documentation

Model Information

Model - index

• name: fnet-base-finetuned-qqp results:
  • task: name: Text Classification type: text - classification dataset: name: GLUE QQP type: glue args: qqp metrics:
    • name: Accuracy type: accuracy value: 0.8847390551570616
    • name: F1 type: f1 value: 0.8466197090382463

📄 License

This model is licensed under the Apache - 2.0 license.