Chocolatine Fusion 14B

🚀 Chocolatine-Fusion-14B

FINGU-AI/Chocolatine-Fusion-14B is a merged model that combines jpacifico/Chocolatine-2-14B-Instruct-v2.0b3 and jpacifico/Chocolatine-2-14B-Instruct-v2.0b2. It retains the strengths of Chocolatine and benefits from an optimized fusion, which enhances reasoning and multi-turn conversation capabilities.

🚀 Quick Start

You can run this model using the following code:

Basic Usage

import transformers
from transformers import AutoTokenizer

# Format prompt
message = [
    {"role": "system", "content": "You are a helpful assistant chatbot."},
    {"role": "user", "content": "What is a Large Language Model?"}
]
tokenizer = AutoTokenizer.from_pretrained("FINGU-AI/Chocolatine-Fusion-14B")
prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)

# Create pipeline
pipeline = transformers.pipeline(
    "text-generation",
    model="FINGU-AI/Chocolatine-Fusion-14B",
    tokenizer=tokenizer
)

# Generate text
sequences = pipeline(
    prompt,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    num_return_sequences=1,
    max_length=200,
)
print(sequences[0]['generated_text'])

✨ Features

• Bilingual Capabilities: The model has been trained using French and English RLHF datasets (including jpacifico/french-orca-dpo-pairs-revised) for enhanced bilingual performance.
• Long-context Support: It supports up to 128K tokens and can generate up to 8K tokens.

🔧 Technical Details

Chocolatine-Fusion-14B is based on DPO fine-tuning from the Chocolatine-2 series, which originated as a fine-tuned version of sometimesanotion/Lamarck-14B-v0.7.

📄 License

This project is licensed under the Apache-2.0 license.

⚠️ Limitations

Chocolatine-Fusion-14B is a demonstration of model merging techniques rather than a standalone fine-tuned model.

⚠️ Important Note

• It does not have any built-in moderation mechanisms.
• Responses may vary based on the interaction and prompt style.
• Performance on highly technical or domain-specific queries may require further fine-tuning.

📚 Documentation