Zephyr 7b Gemma V0.1

🚀 Zephyr 7B Gemma

Zephyr 7B Gemma is a fine - tuned language model, trained to serve as a helpful assistant, offering high - quality text generation capabilities.

🚀 Quick Start

The model can be used for chat. Here's how you can run the model using the pipeline() function from 🤗 Transformers:

# pip install transformers>=4.38.2
# pip install accelerate

import torch
from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="HuggingFaceH4/zephyr-7b-gemma-v0.1",
    device_map="auto",
    torch_dtype=torch.bfloat16,
)
messages = [
    {
        "role": "system",
        "content": "",  # Model not yet trained for follow this
    },
    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
]
outputs = pipe(
    messages,
    max_new_tokens=128,
    do_sample=True,
    temperature=0.7,
    top_k=50,
    top_p=0.95,
    stop_sequence="<|im_end|>",
)
print(outputs[0]["generated_text"][-1]["content"])
# It is not possible for a human to eat a helicopter in one sitting, as a
# helicopter is a large and inedible machine. Helicopters are made of metal,
# plastic, and other materials that are not meant to be consumed by humans.
# Eating a helicopter would be extremely dangerous and would likely cause
# serious health problems, including choking, suffocation, and poisoning. It is
# important to only eat food that is safe and intended for human consumption.

✨ Features

Zephyr is a series of language models trained to act as helpful assistants. Zephyr 7B Gemma is fine - tuned on a mix of publicly available, synthetic datasets using Direct Preference Optimization (DPO), which can provide better text generation performance.

📚 Documentation

Model description

Model Sources

• Repository: https://github.com/huggingface/alignment - handbook
• Demo: https://huggingface.co/spaces/HuggingFaceH4/zephyr - 7b - gemma - chat

Performance

Intended uses & limitations

The model was initially fine - tuned on the [DEITA 10K](https://huggingface.co/datasets/HuggingFaceH4/deita - 10k - v0 - sft) dataset, which contains a diverse range of synthetic dialogues generated by ChatGPT. Then it was further aligned with 🤗 TRL's DPOTrainer on the [argilla/dpo - mix - 7k](https://huggingface.co/datasets/argilla/dpo - mix - 7k) dataset, which contains 7k prompts and model completions ranked by GPT - 4. As a result, the model can be used for chat and you can check out our [demo](https://huggingface.co/spaces/HuggingFaceH4/zephyr - chat) to test its capabilities.

Bias, Risks, and Limitations

Zephyr 7B Gemma has not been aligned to human preferences for safety within the RLHF phase or deployed with in - the - loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition of the corpus was used to train the base model (google/gemma - 7b), however it is likely to have included a mix of Web data and technical sources like books and code. See the [StarCoder2 model card](https://huggingface.co/bigcode/starcoder2 - 15b) for an example of this.

Training and evaluation data

This model is a fine - tuned version of [HuggingFaceH4/zephyr - 7b - gemma - sft - v0.1](https://huggingface.co/HuggingFaceH4/zephyr - 7b - gemma - sft - v0.1) on the argilla/dpo - mix - 7k dataset.

It achieves the following results on the evaluation set:

• Loss: 0.4695
• Rewards/chosen: - 3.3746
• Rewards/rejected: - 4.9715

📄 License

The model is under the Gemma Terms of Use.