Bertin Base Xnli Es

🚀 Bertin Gaussian 512 XNLI Checkpoint

This checkpoint is trained for the XNLI dataset, derived from a Spanish RoBERTa - base model.

🚀 Quick Start

This checkpoint has been specifically trained for the XNLI dataset. It is created from Bertin Gaussian 512, a RoBERTa - base model trained from scratch in Spanish.

✨ Features

• Trained for the XNLI dataset.
• Based on a Spanish RoBERTa - base model.

📚 Documentation

Model Information

This checkpoint was created from Bertin Gaussian 512, which is a RoBERTa - base model trained from scratch in Spanish. You can find information on this base model at its own card and more in - depth details on the main project card.

Training Dataset

The training dataset for the base model is mc4. It subsamples documents to a total of about 50 million examples. The sampling is biased towards average perplexity values (using a Gaussian function), discarding more often documents with very large values (poor quality) or very small values (short, repetitive texts).

Community Week

This is part of the Flax/Jax Community Week, organized by HuggingFace and with TPU usage sponsored by Google.

🔧 Technical Details

The model is based on the RoBERTa - base architecture and is trained from scratch in Spanish. The specific training for the XNLI dataset is built on top of the base model's training on the mc4 dataset.

📄 License

This project is licensed under the CC - BY - 4.0 license.

👥 Team Members

• Eduardo González (edugp)
• Javier de la Rosa (versae)
• Manu Romero (mrm8488)
• María Grandury (mariagrandury)
• Pablo González de Prado (Pablogps)
• Paulo Villegas (paulo)