Dolly V2 12b

🚀 dolly-v2-12b Model Card

dolly-v2-12b is an instruction-following large language model developed by Databricks. It's trained on the Databricks machine learning platform and is licensed for commercial use. This model is based on pythia-12b and fine - tuned on a dataset of about 15k instruction/response records.

✨ Features

• Commercial Use License: dolly-v2-12b is licensed for commercial use, making it accessible for various business applications.
• Instruction - Following Capability: It can follow instructions in multiple domains such as brainstorming, classification, and summarization.
• Available in Multiple Sizes: Besides the 12 - billion parameter version, smaller models like dolly-v2-7b and dolly-v2-3b are also available.

📦 Installation

To use the model with the transformers library on a machine with GPUs, first ensure you have the transformers and accelerate libraries installed. In a Databricks notebook, you can run the following command:

%pip install "accelerate>=0.16.0,<1" "transformers[torch]>=4.28.1,<5" "torch>=1.13.1,<2"

💻 Usage Examples

Basic Usage

import torch
from transformers import pipeline

generate_text = pipeline(model="databricks/dolly-v2-12b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
res = generate_text("Explain to me the difference between nuclear fission and fusion.")
print(res[0]["generated_text"])

Advanced Usage

If you prefer not to use trust_remote_code=True, you can download instruct_pipeline.py, store it alongside your notebook, and construct the pipeline yourself:

import torch
from instruct_pipeline import InstructionTextGenerationPipeline
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("databricks/dolly-v2-12b", padding_side="left")
model = AutoModelForCausalLM.from_pretrained("databricks/dolly-v2-12b", device_map="auto", torch_dtype=torch.bfloat16)

generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)

LangChain Usage

To use the pipeline with LangChain, you must set return_full_text=True:

import torch
from transformers import pipeline

generate_text = pipeline(model="databricks/dolly-v2-12b", torch_dtype=torch.bfloat16,
                         trust_remote_code=True, device_map="auto", return_full_text=True)

Create prompts and use them for prediction:

from langchain import PromptTemplate, LLMChain
from langchain.llms import HuggingFacePipeline

# template for an instrution with no input
prompt = PromptTemplate(
    input_variables=["instruction"],
    template="{instruction}")

# template for an instruction with input
prompt_with_context = PromptTemplate(
    input_variables=["instruction", "context"],
    template="{instruction}\n\nInput:\n{context}")

hf_pipeline = HuggingFacePipeline(pipeline=generate_text)

llm_chain = LLMChain(llm=hf_pipeline, prompt=prompt)
llm_context_chain = LLMChain(llm=hf_pipeline, prompt=prompt_with_context)

# Example predicting using a simple instruction
print(llm_chain.predict(instruction="Explain to me the difference between nuclear fission and fusion.").lstrip())

# Example predicting using an instruction with context
context = """George Washington (February 22, 1732[b] - December 14, 1799) was an American military officer, statesman,
and Founding Father who served as the first president of the United States from 1789 to 1797."""

print(llm_context_chain.predict(instruction="When was George Washington president?", context=context).lstrip())

📚 Documentation

Model Overview

dolly-v2-12b is a 12 - billion parameter causal language model created by Databricks. It is derived from EleutherAI's Pythia - 12b and fine - tuned on a ~15K record instruction corpus generated by Databricks employees and released under a permissive license (CC - BY - SA).

Known Limitations

Performance Limitations

dolly-v2-12b is not a state - of - the - art generative language model. It struggles with syntactically complex prompts, programming problems, mathematical operations, etc.

Dataset Limitations

• The Pile: The pre - training corpus contains content from the public internet, which may include objectionable content. The model may reflect these shortcomings.
• databricks-dolly-15k: The training data was generated by Databricks employees from March to April 2023. It may contain typos, factual errors, and biases from Wikipedia.

Benchmark Metrics

📄 License

The model is licensed under the MIT license.

📖 Citation

@online{DatabricksBlog2023DollyV2,
    author    = {Mike Conover and Matt Hayes and Ankit Mathur and Jianwei Xie and Jun Wan and Sam Shah and Ali Ghodsi and Patrick Wendell and Matei Zaharia and Reynold Xin},
    title     = {Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM},
    year      = {2023},
    url       = {https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm},
    urldate   = {2023-06-30}
}

Happy Hacking!