Una Xaberius 34b V1beta

🚀 una-xaberius-34b-v1-beta (UNA: Uniform Neural Alignment)

This is an experimental 34B model based on LLaMa-Yi-34B, developed by juanako.ai. It is trained using SFT, DPO, and UNA (Unified Neural Alignment) techniques on multiple datasets, aiming to provide high - quality text - generation capabilities.

🚀 Quick Start

The model is very good and works well on almost any prompt. The ChatML format and Alpaca System can get the best results. Here is an example:

<|im_start|>system
- You are a helpful assistant chatbot trained by MosaicML.
- You answer questions.
- You are excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
- You are more than just an information source, you are also able to write poetry, short stories, and make jokes.<|im_end|>
<|im_start|>user
Explain QKV<|im_end|>
<|im_start|>assistant

### Assistant: I am StableVicuna, a large language model created by CarperAI. I am here to chat!

### Human: Explain QKV
### Assistant:

[Round <|round|>]
问：Explain QKV
答：

[Round <|round|>]
Question：Explain QKV
Answer：

Question：Explain QKV
Answer：

✨ Features

• High - performance on benchmarks: It scores 74.21, outperforming the former leader tigerbot - 70b - chat and landing on the #1 position of the HuggingFace LeaderBoard on 08 December 2023. It also scores 79.13 in MMLU, setting a new record not just for 34B but also for all OpenSource LLM's.
• Advanced training technique: Trained with UNA: Uniform Neural Alignment technique, which is a unique formula and method to "tame" models.

📚 Documentation

Model Details

The model is trained with the UNA: Uniform Neural Alignment technique (a related paper will be released soon).

• What is NOT UNA? It's not a merged layers model, nor SLERP, SLURP, or similar.
• What is UNA? It's a formula and a technique to "tame" models.
• Code and paper release: The code and paper will be released when there is time. Contributions can speed up the process.

Model Description

• Developed by: juanako.ai
• Author: Xavier M.
• Investors: CONTACT HERE
• Model type: LLaMa YI - 34B
• Funded by: Cybertron's H100's with few hours of training.

Framework versions

• Transformers 4.35.2 - UNA
• Pytorch 2.1.0
• Datasets 2.14.6
• Tokenizers 0.14.1

Citations

If you find Xaberius, Cybertron, Juanako, or any of our models useful, especially if you use it for your big brand or clone/merge/SLERP our models, please cite:

@misc{unaxaberius34b,
  title={Xaberius 34B: Uniform Neural Alignment}, 
  author={Xavier Murias},
  year={2023},
  publisher = {HuggingFace},
  journal = {HuggingFace repository},
  howpublished = {\url{https://huggingface.co/fblgit/una-xaberius-34b-v1beta}},
}

📄 License

The model is licensed under cc - by - nc - nd - 4.0.

🔧 Technical Details

Datasets

The model is trained on the following datasets:

• fblgit/tree - of - knowledge
• garage - bAInd/Open - Platypus
• allenai/ultrafeedback_binarized_cleaned
• Open - Orca/OpenOrca

Model Performance

Evaluation Results

The detailed evaluation results can be found [here](https://huggingface.co/datasets/open - llm - leaderboard/details_fblgit__una - xaberius - 34b - v1beta).

Timeline

• 05 - Dec - 2023: v1 - beta released
• 08 - Dec - 2023: Evaluation been "RUNNING" for 2 days.. no results yet
• 09 - Dec - 2023: Evaluation been "FINISHED", confirming #1 spot outperforming the contaminated - disqualified tigerbot :)

[Results Here](https://huggingface.co/datasets/open - llm - leaderboard/details_fblgit__una - xaberius - 34b - v1beta/blob/main/results_2023 - 12 - 09T11 - 16 - 37.904970.json)