Arsh Llm

🚀 Arsh LLM

Arsh LLM is a newly developed project designed to assist in research. It aims to demonstrate that large models don't necessarily require high - end hardware.

🚀 Quick Start

There is no specific quick - start content provided in the original document.

✨ Features

Arsh LLM is a research project with the following features:

• It is trained on the olmo mix dataset using a T4 GPU.
• The architecture is created with reference to Gpt neox and llama documents and AI for optimization.
• Initial weights are calculated using phi - 4.
• It is first trained on a part of the PILE dataset for stability and then on the olmo - mix - 1124 dataset.
• The merged model is fine - tuned using small conversational open - source datasets.

📦 Installation

There is no installation steps provided in the original document.

💻 Usage Examples

There is no code examples provided in the original document.

📚 Documentation

Model Details

Model Description

Arsh LLM is my latest research project. It requires more training, and this is just one of its early versions. My objective is to show that large models don't need powerful hardware, at least with the available tools to expedite the process.

First, I designed an architecture using Gpt neox and llama documents and AI to achieve the best optimization (I'm not certain if it's fully optimized). Then, I created a model and calculated the initial weights using phi - 4. After that, I trained it on a portion of the PILE dataset to stabilize the model for use. Next, I trained a model named arshGpt on the olmo - mix - 1124 dataset. The goal was to more easily convert data into my new large - scale model. This is a merged model fine - tuned with some small conversational open - source datasets. I believe its performance has improved.

📄 License

I've chosen the MIT license for this model, which is the best license to experience the freedom 😅!

Information Table