Gpt2 French Small

🚀 French GPT-2 Small Model

A small French language model for text generation and potential NLP tasks.

🚀 Quick Start

This French GPT-2 model is designed for French text generation and may be applicable to other NLP tasks. It offers a cost - effective way to generate French text with relatively low resources.

✨ Features

• Based on OpenAI GPT - 2: Built upon the OpenAI GPT - 2 small model.
• Low - resource Training: Trained on a very small (190Mb) dataset from French Wikipedia in just over a day using Transfer Learning and Fine - tuning techniques on one Colab Pro with 1 GPU 16GB.
• Transferable Approach: Demonstrates that it's possible to get a language model in any language with low resources.
• Hugging Face Integration: Fine - tuned using Hugging Face libraries (Transformers and Tokenizers) within the fastai v2 Deep Learning framework.

📚 Documentation

Model Creation

This model was created following the method of Pierre Guillou. You can refer to the detailed guide at Faster than training from scratch — Fine - tuning the English GPT - 2 in any language with Hugging Face and fastai v2 (practical case with Portuguese).

Model Availability

The model is now available on Hugging Face. For more information or requests, please visit the above - mentioned link.

Model Improvement

The model might be improved by using a larger dataset under more powerful training infrastructure. Currently, it can be used for small fine - tuning experiments (e.g., with aitextgen).

Performance

Note that the metrics were lost. The model can generate French text, but there are some minor grammar issues, and the text coherence is somewhat limited.

📄 License

No license information is provided in the original README.

💡 Usage Tip

💡 Usage Tip

This model can be a good starting point for small - scale French text generation experiments. If you plan to improve it, consider using a larger dataset and more powerful training resources.