Westlake 7B V2 Laser Truthy Dpo

🚀 WestLake-7B-v2-laser-truthy-dpo

WestLake-7B-v2-laser-truthy-dpo is a text - generation model trained on specific datasets, offering high - quality text - generation capabilities and achieving good results in multiple evaluations.

🚀 Quick Start

This section provides an overview of the model, its training, and evaluation processes.

✨ Features

• Trained on Specific Dataset: Trained on jondurbin/truthy-dpo-v0.1 to enhance performance.
• Multiple Evaluation Metrics: Evaluated on various datasets such as AI2 Reasoning Challenge, HellaSwag, etc., with high accuracy scores.
• Available in Different Formats: GGUF and ExLlamav2 quantizations are available for different usage scenarios.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer
import transformers
import torch

model = "macadeliccc/WestLake-7B-v2-laser-truthy-dpo"
chat = [

  {"role": "user", "content": "Hello, how are you?"},

  {"role": "assistant", "content": "I'm doing great. How can I help you today?"},

  {"role": "user", "content": "I'd like to show off how chat templating works!"},

]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

Advanced Usage

The process of realigning the prompt template to ChatML during fine - tuning is as follows:

def chatml_format(example):
    # Format system
    if len(example['system']) > 0:
        message = {"role": "system", "content": example['system']}
        system = tokenizer.apply_chat_template([message], tokenize=False)
    else:
        system = ""

    # Format instruction
    message = {"role": "user", "content": example['prompt']}
    prompt = tokenizer.apply_chat_template([message], tokenize=False, add_generation_prompt=True)

    # Format chosen answer
    chosen = example['chosen'] + "<|im_end|>\n"

    # Format rejected answer
    rejected = example['rejected'] + "<|im_end|>\n"

    return {
        "prompt": system + prompt,
        "chosen": chosen,
        "rejected": rejected,
    }

📚 Documentation

Process

• Trained cognitivecomputations/WestLake-7B-v2-laser on jondurbin/truthy-dpo-v0.1.
• Completed 2 epochs.
• Used a 2e - 5 learning rate.

Evaluations

Evaluated the GGUF for usability reasons. EQ - Bench uses Ooba for inference.

----Benchmark Complete----
2024-01-31 14:38:14
Time taken: 18.9 mins
Prompt Format: ChatML
Model: macadeliccc/WestLake-7B-v2-laser-truthy-dpo-GGUF
Score (v2): 75.15
Parseable: 171.0
---------------
Batch completed
Time taken: 19.0 mins
---------------

GGUF

GGUF versions are available here

ExLlamav2

Thanks to user bartowski we now have exllamav2 quantizations in 3.5 through 8 bpw. They are available here:

• bartowski/WestLake-7B-v2-laser-truthy-dpo-exl2

Chat Template

This was the process during fine - tune to realign the prompt template to ChatML. There seems to be an error where you can use either Mistral (original) prompt template or you can use ChatML in the GGUF version.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

🔧 Technical Details

No specific technical details are provided in the original document, so this section is skipped.

📄 License

The model is licensed under the Apache 2.0 license.