Dolphin 2.9 Llama3 70b 5.0bpw H6 Exl2

🚀 Dolphin 2.9 Llama 3 70b 🐬

Dolphin 2.9 Llama 3 70b is a powerful AI model curated and trained by a group of experts, offering various instruction, conversational, and coding skills, along with initial agentic abilities and function - calling support.

Curated and trained by Eric Hartford, Lucas Atkins, Fernando Fernandes, and with help from the community of Cognitive Computations.

Discord: Join our Discord community

✨ Features

• Based on Llama - 3 - 70b: This model is built upon the powerful Llama - 3 - 70b architecture and is governed by the META LLAMA 3 COMMUNITY LICENSE AGREEMENT.
• 8k Context: The base model has an 8k context, and the qLoRA fine - tuning was done with an 8k sequence length.
• Training Details: It took 2.5 days to train on an 8xH100 node provided by Crusoe Cloud. The model was trained FFT on all parameters using the ChatML prompt template format.
• Skill Set: Dolphin - 2.9 has a wide range of instruction, conversational, and coding skills. It also has initial agentic abilities and supports function calling.
• Uncensored: The model is uncensored. The dataset has been filtered to remove alignment and bias, making it more compliant. However, users are advised to implement their own alignment layer before exposing the model as a service.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

<|im_start|>system
You are Dolphin, a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

📚 Documentation

• Sponsors: We appreciate Crusoe Cloud for providing an excellent on - demand 8xH100 node.
• Dataset: The model was trained on the following datasets:
  • cognitivecomputations/Dolphin - 2.9
  • teknium/OpenHermes - 2.5
  • m - a - p/CodeFeedback - Filtered - Instruction
  • cognitivecomputations/dolphin - coder
  • cognitivecomputations/samantha - data
  • HuggingFaceH4/ultrachat_200k
  • microsoft/orca - math - word - problems - 200k
  • abacusai/SystemChat - 1.1
  • Locutusque/function - calling - chatml
  • internlm/Agent - FLAN
• License: Dolphin is licensed according to Meta's Llama license. Permission is granted for any use, including commercial, that falls within the accordance of Meta's Llama - 3 license. The model was trained on data generated from GPT4, among other models.

🔧 Technical Details

The model was trained FFT on all parameters, using the ChatML prompt template format. The base model has 8k context, and the qLoRA fine - tuning was with 8k sequence length. It took 2.5 days on an 8xH100 node provided by Crusoe Cloud.

📄 License

This model is governed by META LLAMA 3 COMMUNITY LICENSE AGREEMENT.

Evals

Quants

• GGUF Quant
• 2.25bpw - exl2 Quant
• 2.5bpw - exl2 Quant
• 4.5bpw - exl2 Quant

⚠️ Important Note

Dolphin is uncensored. It will be highly compliant with any requests, even unethical ones. You are responsible for any content you create using this model. Please read the blog post about uncensored models: [Uncensored Models](https://erichartford.com/uncensored - models). You are advised to implement your own alignment layer before exposing the model as a service.

💡 Usage Tip

Enjoy using the model responsibly.

[](https://github.com/OpenAccess - AI - Collective/axolotl)