PULI-GPT-3SX Open-source Model - Free Deployment, Effortlessly Achieve Hungarian Text Generation

PULI GPT 3SX

Developed by NYTK

PULI 3SX is a 6.85 billion parameter Hungarian GPT-NeoX model, trained based on EleutherAI's GPT-NeoX architecture, suitable for text generation tasks.

Large Language Model

Transformers

Other#Hungarian text generation #Large language model #Long text generation

Downloads 1,459

Release Time : 12/19/2022

Model Overview

This is a text generation model optimized for Hungarian, based on the GPT-NeoX architecture with 6.85 billion parameters, trained on 36.3 billion words.

Model Features

Large-scale parameters

With 6.85 billion parameters, capable of handling complex language patterns

Hungarian language optimization

Specifically trained and optimized for Hungarian

Long text processing

Supports sequences up to 2048 tokens in length

Model Capabilities

Text generation

Language modeling

Hungarian text processing

Use Cases

Content creation

Story generation

Generate coherent Hungarian stories based on prompts

Produces logically consistent text that fits the context

Article continuation

Continue generating article content based on the opening paragraph

Maintains consistent style and theme in continued content

Language technology research

Hungarian language model research

Used for studying the characteristics of Hungarian language models

Provides foundational models for Hungarian NLP research

🚀 PULI 3SX base (6.85 billion parameter)

A Hungarian text - generation model with 6.85 billion parameters, offering high - quality text generation capabilities.

For further details, see our demo site.

✨ Features

Hungarian GPT - NeoX model (6.7 billion parameter)
Trained with EleutherAI's GPT - NeoX [github](https://github.com/EleutherAI/gpt - neox)
Dataset: 36.3 billion words
Checkpoint: 150 000 steps

Property	Details
Model Type	Hungarian GPT - NeoX model
Training Data	36.3 billion words
Checkpoint	150 000 steps

🔧 Technical Details

Limitations

max_seq_length = 2048

📚 Documentation

Citation

If you use this model, please cite the following paper:

@inproceedings {yang - puli,
    title = {Jönnek a nagyok! BERT - Large, GPT - 2 és GPT - 3 nyelvmodellek magyar nyelvre},
    booktitle = {XIX. Magyar Számítógépes Nyelvészeti Konferencia (MSZNY 2023)},
    year = {2023},
    publisher = {Szegedi Tudományegyetem, Informatikai Intézet},
    address = {Szeged, Hungary},
    author = {Yang, Zijian Győző and Dodé, Réka and Ferenczi, Gergő and Héja, Enikő and Jelencsik - Mátyus, Kinga and Kőrös, Ádám and Laki, László János and Ligeti - Nagy, Noémi and Vadász, Noémi and Váradi, Tamás},
    pages = {247--262}
}

💻 Usage Examples

Basic Usage

from transformers import GPTNeoXForCausalLM, AutoTokenizer

model = GPTNeoXForCausalLM.from_pretrained("NYTK/PULI-GPT-3SX")
tokenizer = AutoTokenizer.from_pretrained("NYTK/PULI-GPT-3SX")
prompt = "Elmesélek egy történetet a nyelvtechnológiáról."
input_ids = tokenizer(prompt, return_tensors="pt").input_ids

gen_tokens = model.generate(
    input_ids,
    do_sample=True,
    temperature=0.9,
    max_length=100,
)

gen_text = tokenizer.batch_decode(gen_tokens)[0]
print(gen_text)

Advanced Usage

from transformers import pipeline, GPTNeoXForCausalLM, AutoTokenizer

model = GPTNeoXForCausalLM.from_pretrained("NYTK/PULI-GPT-3SX")
tokenizer = AutoTokenizer.from_pretrained("NYTK/PULI-GPT-3SX")
prompt = "Elmesélek egy történetet a nyelvtechnológiáról."
generator = pipeline(task="text-generation", model=model, tokenizer=tokenizer)

print(generator(prompt)[0]["generated_text"])

📄 License

This project is licensed under the cc - by - nc - 4.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご