Zeta 2

🚀 Zeta 2: Further into the Future

Zeta 2 is the new version of Zeta, aiming to make a significant leap forward from Zeta 1. Zeta 1 demonstrated that "ordinary consumers can create LLMs", and Zeta 2 continues this journey.

🚀 Quick Start

• Ollama: DiamondGotCat/Zeta-2

✨ Features

• Multilingual Support: Available in languages like Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Arabic.
• Text Generation: It belongs to the text-generation pipeline tag.

📦 Installation

Quantized Model (GGUF)

• Static Quant
• Weighted/imatrix Quant

💻 Usage Examples

Prompt Template

{{ if .System }}{{ .System }}{{ end }}
{{ if .Prompt }}<USER>{{ .Prompt }}</USER>{{ end }}
<ASSISTANT>

📚 Documentation

About Zeta 2

Zeta 2 is an LLM with approximately 460 million parameters. It could be more appropriately called an SLM, painstakingly created on a consumer computer.

Computer Spec

Machine: Mac Mini (M2 Pro, 32GB RAM, 2023)

This is a technical matter. On a Mac with Apple Silicon, you can only choose between CPU or MPS (Apple's proprietary API). CPU has some compatibility but is too slow. MPS is somewhat faster but doesn't allow the use of optimization systems such as fp16. Zeta 2 was built using MPS.

Arguments of Trainer

• train epochs: 3
• warmup steps: 100

Datasets

Details of the dataset used can be found here

Links

• GitHub: Zeta

📄 License

This project is licensed under the MIT license.

Zeta is just a small SLM, but don't forget that it has big dreams inside.