Gpt2 Turkish Cased

🚀 Turkish GPT-2 Model

This repository presents a GPT-2 model trained on diverse Turkish texts. It serves as a starting point for fine - tuning on other text datasets.

🚀 Quick Start

The model can be used as follows:

Basic Usage

from transformers import AutoTokenizer, AutoModelWithLMHead
tokenizer = AutoTokenizer.from_pretrained("redrussianarmy/gpt2-turkish-cased")
model = AutoModelWithLMHead.from_pretrained("redrussianarmy/gpt2-turkish-cased")

Advanced Usage

Here's an example of using the Transformers Pipelines for text generation:

from transformers import pipeline
pipe = pipeline('text-generation', model="redrussianarmy/gpt2-turkish-cased",
                 tokenizer="redrussianarmy/gpt2-turkish-cased", config={'max_length':800})   
text = pipe("Akşamüstü yolda ilerlerken, ")[0]["generated_text"]
print(text)

✨ Features

• The model is trained on various Turkish texts, making it suitable as an entry - point for fine - tuning on other Turkish text data.
• It provides both PyTorch and TensorFlow compatible weights.

📦 Installation

How to clone the model repo?

git lfs install
git clone https://huggingface.co/redrussianarmy/gpt2-turkish-cased

📚 Documentation

Training corpora

A Turkish corpora from oscar - corpus was used. With Huggingface's Tokenizers library, a byte - level BPE was created. A 52K byte - level BPE vocab was built based on the training corpora. After that, the GPT - 2 for Turkish was trained on two 2080TI over the complete training corpus for five epochs. Logs during training: https://tensorboard.dev/experiment/3AWKv8bBTaqcqZP5frtGkw/#scalars

Model weights

Both PyTorch and TensorFlow compatible weights are available.

📄 License

No license information is provided in the original README, so this section is skipped.

🔧 Technical Details

The model training process involves using a Turkish corpora from oscar - corpus. Huggingface's Tokenizers library is used to create a byte - level BPE and a 52K byte - level BPE vocab. The training is carried out on two 2080TI over the complete training corpus for five epochs. The training logs can be accessed at https://tensorboard.dev/experiment/3AWKv8bBTaqcqZP5frtGkw/#scalars.

📞 Contact (Bugs, Feedback, Contribution and more)

For questions about the GPT2 - Turkish model, just open an issue here 🤗