Qwen2.5 1.5B Instruct

🚀 Qwen2.5-1.5B-Instruct

This model is designed for use in the Gensyn RL Swarm, enabling local fine - tuning via peer - to - peer reinforcement learning after training. Once fine - tuned, it can be used in various workflows.

🚀 Quick Start

To deploy this model into a swarm and/or participate in the Gensyn Testnet, follow the instructions in the RL Swarm repository, read about the testnet, read the RL Swarm overview, and/or read the RL Swarm technical report.

✨ Features

This repo contains an unmodified version of the instruction - tuned 1.5B Qwen2.5 model, which has the following features:

• Model Type: Causal Language Models
• Training Stage: Pretraining & Post - training
• Architecture: transformers with RoPE, SwiGLU, RMSNorm, Attention QKV bias and tied word embeddings
• Number of Parameters: 1.54B
• Number of Paramaters (Non - Embedding): 1.31B
• Number of Layers: 28
• Number of Attention Heads (GQA): 12 for Q and 2 for KV
• Context Length: Full 32,768 tokens and generation 8192 tokens

📚 Documentation

This model is intended for use in the Gensyn RL Swarm, to finetune locally using peer - to - peer reinforcement learning post - training.

Once finetuned, the model can be used as normal in any workflow, for details on how to do this please refer to the original model documentation.

For more details on the original model, please refer to the original repository here.

This model is intended for use in the Gensyn RL Swarm system, for details on model requirements when using outside of a swarm, refer to the original Qwen repo here.

📄 License

This model is released under the Apache 2.0 license.