# RoPE positional encoding

A fine-tuned version based on the XLM-Roberta base model, using RoPE (Rotary Position Embedding) to replace the original positional embeddings, supporting 2048 sequence length

GPT-NeoX-20B is an open-source autoregressive language model with 20 billion parameters, designed based on the GPT-3 architecture and trained on The Pile dataset.

A large Slovak language generation model with 1.4 billion parameters, based on the GPT - J architecture

The first publicly available Transformer model trained on Slovak language corpus with 162 million parameters