BETO Open-source Spanish Q&A Model - Fine-tuned on Large Corpora, Free Spanish Q&A Implementation

Bert Base Spanish Wwm Cased Finetuned Spa Squad2 Es

Developed by mrm8488

BETO is a BERT model trained on a large Spanish corpus, this version is fine-tuned for Spanish question answering tasks

Question Answering System Spanish#Spanish Q&A #Whole Word Masking BERT #High-precision Reading Comprehension

Downloads 1,648

Release Time : 3/2/2022

Model Overview

This is a question-answering model based on BETO (Spanish BERT) and fine-tuned on the SQuAD-es-v2.0 dataset, specifically designed for Spanish question answering tasks

Model Features

Whole Word Masking Technology

Trained using Whole Word Masking technology to enhance the model's understanding of Spanish

Spanish Language Optimization

Specifically trained on large-scale Spanish corpus, outperforming multilingual BERT models

Question Answering Task Optimization

Fine-tuned on the SQuAD-es-v2.0 dataset, specialized for Spanish question answering scenarios

Model Capabilities

Spanish Text Understanding

Question Answering System

Context Understanding

Answer Extraction

Use Cases

Education

Spanish Learning Assistance

Helps students understand Spanish texts and answer questions

Customer Service

Spanish FAQ System

Automatically answers common questions from Spanish-speaking customers

🚀 BETO (Spanish BERT) + Spanish SQuAD2.0

This model, provided by the BETO team, is fine - tuned on SQuAD-es-v2.0 for the Q&A downstream task, offering effective solutions for Spanish question - answering scenarios.

📚 Documentation

Details of the language model('dccuchile/bert-base-spanish-wwm-cased')

The language model 'dccuchile/bert-base-spanish-wwm-cased':

BETO is a BERT model trained on a big Spanish corpus. Similar in size to BERT - Base, it was trained using the Whole Word Masking technique. Tensorflow and Pytorch checkpoints for the uncased and cased versions are available, along with some results for Spanish benchmarks comparing BETO with Multilingual BERT and other (non - BERT - based) models.

Details of the downstream task (Q&A) - Dataset

The dataset used is SQuAD-es-v2.0.

Property	Details
SQuAD2.0 Train	130 K Q&A
SQuAD2.0 - es - v2.0	111 K Q&A
SQuAD2.0 Dev	12 K Q&A
SQuAD - es - v2.0 - small Dev	69 K Q&A

📦 Installation

The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command:

export SQUAD_DIR=path/to/nl_squad
python transformers/examples/question-answering/run_squad.py \
  --model_type bert \
  --model_name_or_path dccuchile/bert-base-spanish-wwm-cased \
  --do_train \
  --do_eval \
  --do_lower_case \
  --train_file $SQUAD_DIR/train_nl-v2.0.json \
  --predict_file $SQUAD_DIR/dev_nl-v2.0.json \
  --per_gpu_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 2.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir /content/model_output \
  --save_steps 5000 \
  --threads 4 \
  --version_2_with_negative

📈 Results

Metric	Value
Exact	76.5050
F1	86.0781

{
  "exact": 76.50501430594491,
  "f1": 86.07818773108252,
  "total": 69202,
  "HasAns_exact": 67.93020719738277,
  "HasAns_f1": 82.37912207996466,
  "HasAns_total": 45850,
  "NoAns_exact": 93.34104145255225,
  "NoAns_f1": 93.34104145255225,
  "NoAns_total": 23352,
  "best_exact": 76.51223953064941,
  "best_exact_thresh": 0.0,
  "best_f1": 86.08541295578848,
  "best_f1_thresh": 0.0
}

💻 Usage Examples

Model in action (in a Colab Notebook)

Set the context and ask some questions:

Set context and questions

Run predictions:

Run the model

Created by Manuel Romero/@mrm8488

Made with ♥ in Spain

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご