🚀 dolly-v2-7b Model Card

dolly-v2-7b is an instruction-following large language model developed by Databricks. Trained on the Databricks machine learning platform, it's based on pythia-6.9b and fine - tuned on a dataset of about 15k instruction/response records. This model is licensed for commercial use and shows high - quality instruction - following behavior.

🚀 Quick Start

Prerequisites

To use the model with the transformers library on a machine with GPUs, first make sure you have the transformers and accelerate libraries installed. In a Databricks notebook, you can run the following command:

%pip install "accelerate>=0.16.0,<1" "transformers[torch]>=4.28.1,<5" "torch>=1.13.1,<2"

Loading the Model

The instruction following pipeline can be loaded using the pipeline function as shown below. This loads a custom InstructionTextGenerationPipeline found in the model repo here, which is why trust_remote_code=True is required. Including torch_dtype=torch.bfloat16 is generally recommended if this type is supported to reduce memory usage. It does not appear to impact output quality. You can remove it if there is sufficient memory.

import torch
from transformers import pipeline

generate_text = pipeline(model="databricks/dolly-v2-7b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")

Using the Pipeline

You can then use the pipeline to answer instructions:

res = generate_text("Explain to me the difference between nuclear fission and fusion.")
print(res[0]["generated_text"])

Alternative Loading Method

If you prefer not to use trust_remote_code=True, you can download instruct_pipeline.py, store it alongside your notebook, and construct the pipeline yourself from the loaded model and tokenizer:

import torch
from instruct_pipeline import InstructionTextGenerationPipeline
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("databricks/dolly-v2-7b", padding_side="left")
model = AutoModelForCausalLM.from_pretrained("databricks/dolly-v2-7b", device_map="auto", torch_dtype=torch.bfloat16)

generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)

LangChain Usage

To use the pipeline with LangChain, you must set return_full_text=True, as LangChain expects the full text to be returned and the default for the pipeline is to only return the new text.

import torch
from transformers import pipeline

generate_text = pipeline(model="databricks/dolly-v2-7b", torch_dtype=torch.bfloat16,
                         trust_remote_code=True, device_map="auto", return_full_text=True)

You can create a prompt that either has only an instruction or has an instruction with context:

from langchain import PromptTemplate, LLMChain
from langchain.llms import HuggingFacePipeline

# template for an instrution with no input
prompt = PromptTemplate(
    input_variables=["instruction"],
    template="{instruction}")

# template for an instruction with input
prompt_with_context = PromptTemplate(
    input_variables=["instruction", "context"],
    template="{instruction}\n\nInput:\n{context}")

hf_pipeline = HuggingFacePipeline(pipeline=generate_text)

llm_chain = LLMChain(llm=hf_pipeline, prompt=prompt)
llm_context_chain = LLMChain(llm=hf_pipeline, prompt=prompt_with_context)

Example predicting using a simple instruction:

print(llm_chain.predict(instruction="Explain to me the difference between nuclear fission and fusion.").lstrip())

Example predicting using an instruction with context:

context = """George Washington (February 22, 1732[b] - December 14, 1799) was an American military officer, statesman,
and Founding Father who served as the first president of the United States from 1789 to 1797."""

print(llm_context_chain.predict(instruction="When was George Washington president?", context=context).lstrip())

✨ Features

• Instruction - Following: dolly-v2-7b is designed to follow instructions, making it suitable for a variety of natural language processing tasks.
• Commercial Use: Licensed for commercial use, it can be applied in business scenarios.
• Multiple Sizes Available: Besides the 7B parameter model, there are also 3B and 12B parameter models available.

📦 Installation

As mentioned in the Quick Start section, to use the model with the transformers library on a machine with GPUs, you need to install the transformers and accelerate libraries. In a Databricks notebook, run:

%pip install "accelerate>=0.16.0,<1" "transformers[torch]>=4.28.1,<5" "torch>=1.13.1,<2"

📚 Documentation

Model Overview

dolly-v2-7b is a 6.9 billion parameter causal language model created by Databricks. It is derived from EleutherAI's Pythia - 6.9b and fine - tuned on a ~15K record instruction corpus generated by Databricks employees and released under a permissive license (CC - BY - SA).

Known Limitations

Performance Limitations

dolly-v2-7b is not a state - of - the - art generative language model. It struggles with syntactically complex prompts, programming problems, mathematical operations, factual errors, dates and times, open - ended question answering, hallucination, enumerating lists of specific length, stylistic mimicry, having a sense of humor, etc. Moreover, it lacks some capabilities present in the original model, such as well - formatted letter writing.

Dataset Limitations

• The Pile: The pre - training corpus of GPT - J contains content mostly from the public internet, which may include objectionable content. The model is likely to reflect these shortcomings.
• databricks-dolly-15k: The training data is generated by Databricks employees from March to April 2023 and includes Wikipedia passages. It may contain typos, factual errors, and biases from Wikipedia. It also reflects the interests and semantic choices of Databricks employees, which may not represent the global population.

Benchmark Metrics

The following table shows the benchmark performance of various models on the [EleutherAI LLM Evaluation Harness](https://github.com/EleutherAI/lm - evaluation - harness). Model results are sorted by geometric mean.

📄 License

This model is licensed under the MIT license.

Citation

@online{DatabricksBlog2023DollyV2,
    author    = {Mike Conover and Matt Hayes and Ankit Mathur and Jianwei Xie and Jun Wan and Sam Shah and Ali Ghodsi and Patrick Wendell and Matei Zaharia and Reynold Xin},
    title     = {Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM},
    year      = {2023},
    url       = {https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm},
    urldate   = {2023-06-30}
}

Happy Hacking!