Wangchanberta Base Att Spm Uncased Finetune Qa

🚀 airesearch/wangchanberta-base-att-spm-uncased

This project fine-tunes the airesearch/wangchanberta-base-att-spm-uncased model, aiming to enhance its performance in question - answering tasks by leveraging multiple Thai QA datasets.

🚀 Quick Start

Installation

To run the fine - tuning process, you need to set up the environment and execute the following commands.

export MODEL_NAME=airesearch/wangchanberta-base-att-spm-uncased
python train_question_answering_lm_finetuning.py \
  --model_name $MODEL_NAME \
  --dataset_name chimera_qa \
  --output_dir $MODEL_NAME-finetune-chimera_qa-model \
  --log_dir $MODEL_NAME-finetune-chimera_qa-log \
  --lowercase \
  --pad_on_right \
  --fp16

💻 Usage Examples

Basic Usage

The following code snippet shows how to run the fine - tuning script.

export MODEL_NAME=airesearch/wangchanberta-base-att-spm-uncased
python train_question_answering_lm_finetuning.py \
  --model_name $MODEL_NAME \
  --dataset_name chimera_qa \
  --output_dir $MODEL_NAME-finetune-chimera_qa-model \
  --log_dir $MODEL_NAME-finetune-chimera_qa-log \
  --lowercase \
  --pad_on_right \
  --fp16

📚 Documentation

Fine - tuning Details

We fine - tuned the airesearch/wangchanberta-base-att-spm-uncased model using the training sets of iapp_wiki_qa_squad, thaiqa_squad, and nsc_qa. For the latter two datasets, we removed examples that had a cosine similarity of over 0.8 with the validation and test examples, and trimmed the contexts to be around 300 newmm words.

Benchmarks

We shared the benchmarks on wandb using the validation and test sets of iapp_wiki_qa_squad.

Training Script

The model was trained with thai2transformers.

Example Widget

Here is an example widget with a question and its context:

{
  "widget": [
    {
      "text": "สวนกุหลาบเป็นโรงเรียนอะไร",
      "context": "โรงเรียนสวนกุหลาบวิทยาลัย (Suankularb Wittayalai School) (Abbreviation: S.K.) is an all - boys high school. It belongs to the Office of the Secondary Educational Service Area 1, Office of the Basic Education Commission (formerly: Department of General Education), Ministry of Education. It was established by King Chulalongkorn on March 8, 1881 (at that time, April 1 was considered the new year; in the international reckoning, it was 1882). It is the first state - owned school in Thailand."
    }
  ]
}