Gpt 2 Spanish

🚀 Spanish GPT-2

A GPT-2 model trained from scratch on the Spanish portion of the OSCAR dataset, offering text generation capabilities in Spanish.

🚀 Quick Start

This is a GPT-2 model trained from scratch on the Spanish portion of OSCAR. The model is trained with Flax and using TPUs sponsored by Google as part of the Flax/Jax Community Week organized by HuggingFace.

✨ Features

• Trained on the Spanish portion of the OSCAR dataset for Spanish text generation.
• Utilizes the Flax framework and Google-sponsored TPUs for training.

📚 Documentation

Model description

The model used for training is OpenAI's GPT-2, introduced in the paper "Language Models are Unsupervised Multitask Learners" by Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever. This model is available in the Model Hub.

Training data

The training data is the Spanish portion of OSCAR or Open Super-large Crawled ALMAnaCH coRpus. It is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture. This corpus is available in the Datasets library.

Team members

• Manuel Romero (mrm8488)
• Marìa Grandury (mariagrandury)
• Pablo González de Prado (Pablogps)
• Daniel Vera (daveni)
• Sri Lakshmi (srisweet)
• José Posada (jdposa)
• Santiago Hincapie (shpotes)
• Jorge (jorgealro)

📄 License

This project is licensed under the apache-2.0 license.