gpt2-finetuned-greek Open Source Greek Text Generation Model - Free to Generate High-Quality Greek Content

Gpt2 Finetuned Greek

Developed by lighteternal

A Greek text generation model fine-tuned from the English GPT-2 model, jointly developed by the Hellenic Military Academy and the Technical University of Crete

Large Language Model OtherOpen Source License:Apache-2.0 #Greek text generation #Fine-tuned on GPT2 #Multi-source corpus training

Downloads 178

Release Time : 3/2/2022

Model Overview

This is a text generation model optimized for Greek, based on the OpenAI GPT-2 architecture, fine-tuned through gradual layer unfreezing, suitable for Greek text generation tasks

Model Features

Efficient Fine-tuning Method

Utilizes gradual layer unfreezing for fine-tuning, more efficient than training from scratch, especially suitable for low-resource languages

Large-scale Training Data

Trained on approximately 23.4GB of Greek corpus, containing text data from multiple sources

Pre-trained Model Transfer

Fine-tuned from the English GPT-2 model, leveraging the knowledge of the pre-trained model

Model Capabilities

Greek text generation

Language model continuation

Creative writing assistance

Use Cases

Text Generation

Story Continuation

Generates coherent story content based on a given beginning

Produces coherent text that conforms to Greek grammar and context

Content Creation Assistance

Assists writers or content creators in generating creative text

Provides diverse options for text continuation

🚀 Greek (el) GPT2 model

This is a text generation (autoregressive) model based on the English GPT - 2, fine - tuned for the Greek language, offering an efficient solution for Greek text generation.

🚀 Quick Start

The Greek (el) GPT2 model is a fine - tuned version of the English GPT - 2 for the Greek language. It can be easily used with the following code example:

from transformers import pipeline

model = "lighteternal/gpt2-finetuned-greek"

generator = pipeline(
    'text-generation',
    device=0,
    model=f'{model}',
    tokenizer=f'{model}')
    
text = "Μια φορά κι έναν καιρό"

print("\n".join([x.get("generated_text") for x in generator(
    text,
    max_length=len(text.split(" "))+15,
    do_sample=True,
    top_k=50,
    repetition_penalty = 1.2,
    add_special_tokens=False,
    num_return_sequences=5,
    temperature=0.95,
    top_p=0.95)]))

✨ Features

Fine - tuned for Greek: Based on the English GPT - 2, it has been fine - tuned for the Greek language, which is more suitable for Greek text generation tasks.
Efficient Training: Fine - tuned with gradual layer unfreezing, providing a more efficient and sustainable alternative compared to training from scratch, especially for low - resource languages.

📦 Installation

The code example uses the transformers library. You can install it via the following command:

pip install transformers

💻 Usage Examples

Basic Usage

from transformers import pipeline

model = "lighteternal/gpt2-finetuned-greek"

generator = pipeline(
    'text-generation',
    device=0,
    model=f'{model}',
    tokenizer=f'{model}')
    
text = "Μια φορά κι έναν καιρό"

print("\n".join([x.get("generated_text") for x in generator(
    text,
    max_length=len(text.split(" "))+15,
    do_sample=True,
    top_k=50,
    repetition_penalty = 1.2,
    add_special_tokens=False,
    num_return_sequences=5,
    temperature=0.95,
    top_p=0.95)]))

📚 Documentation

Model Information

Property	Details
Model Type	GPT2 (12 - layer, 768 - hidden, 12 - heads, 117M parameters. OpenAI GPT - 2 English model, finetuned for the Greek language)
Training Data	~23.4 GB of Greek corpora from CC100, Wikimatrix, Tatoeba, Books, SETIMES and GlobalVoices
Pre - processing	Tokenization + BPE segmentation
Metrics	Perplexity

Training data

We used a 23.4GB sample from a consolidated Greek corpus from CC100, Wikimatrix, Tatoeba, Books, SETIMES and GlobalVoices containing long sequences. This is a better version of our GPT - 2 small model (https://huggingface.co/lighteternal/gpt2-finetuned-greek-small)

Metrics

Metric	Value
Train Loss	3.67
Validation Loss	3.83
Perplexity	39.12

Acknowledgement

The research work was supported by the Hellenic Foundation for Research and Innovation (HFRI) under the HFRI PhD Fellowship grant (Fellowship Number:50, 2nd call). Based on the work of Thomas Dehaene (ML6): https://blog.ml6.eu/dutch-gpt2-autoregressive-language-modelling-on-a-budget-cff3942dd020

🔧 Technical Details

A text generation (autoregressive) model, using Huggingface transformers and fastai based on the English GPT - 2. Finetuned with gradual layer unfreezing. This is a more efficient and sustainable alternative compared to training from scratch, especially for low - resource languages. Based on the work of Thomas Dehaene (ML6) for the creation of a Dutch GPT2: https://colab.research.google.com/drive/1Y31tjMkB8TqKKFlZ5OJ9fcMp3p8suvs4?usp=sharing

📄 License

This model is licensed under the apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご