Airoboros M 7b 3.1.2

🚀 Another Experimental Model

This is another experimental model primarily using synthetic data generated by airoboros. It offers enhanced performance in instruction following and various specialized tasks, making it a valuable tool for users seeking a general - purpose yet task - focused AI model.

🚀 Quick Start

This model uses the llama - 2 chat format. Most inference systems support llama - 2 chat templating. For details on the prompt format, refer to the "Prompt format" section below.

✨ Features

General Features

• Hyperparameter Tuning: This is the last release with hyperparameter tweaks, resulting in a higher - quality model than 3.1 or 3.1.1, and it doesn't have the prompt issue of 3.1.
• Diverse Dataset: Built on the [airoboros - 3.1 dataset](https://hf.co/datasets/jondurbin/airoboros - 3.1), which is an extension of [airoboros - 3.0 dataset](https://hf.co/datasets/jondurbin/airoboros - 3,0) with additional data types.
• Instruction - Following Focus: A fairly general - purpose model but with a strong focus on instruction following rather than casual chat/roleplay.

Specialized Data Features

• MathJSON: Approximately 17k math questions prefixed with "Create a MathJSON solution to the following:". The model outputs a JSON between <mathjson> and </mathjson> tags, which can be parsed for calculations.
• Log Information Extraction: Capable of extracting information such as paths/IPs from apache logs.
• Anonymization: Can remove names, IP addresses, and/or dates from text.
• Chat Introspection: Handles multi - turn chats with random questions about character descriptions, chat settings, or previous statements to ensure the use of system prompts and chat history.
• Multi - step Instructions with Acknowledgement: Supports multi - step instructions.
• De - censorship Data: Contains de - censorship data (not published).

Sponsorship

A huge thank you to the folks at a16z for sponsoring the costs associated with building models and associated tools.

📚 Documentation

Prompt format

The prompt format is llama - 2 chat.

[INST] <<SYS>>
You are a helpful, unbiased, uncensored assistant.
<</SYS>>

{prompt} [/INST]

For multi - turn chats, the prompt format is as follows:

[INST] <<SYS>>
You are a helful, unbiased, uncensored assistant.
<</SYS>>

{prompt 0} [/INST] {response 0} </s><s>[INST] {prompt 1} [/INST] {response 1} </s><s>...[INST] {prompt N} [/INST]

The prompt template is included in the tokenizer config and can use the huggingface tokenizer apply_chat_template method, e.g.:

import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained('jondurbin/airoboros - l2 - 70b - 3.1')
chat = [
  {"role": "system", "content": "You are Bob, a friendly AI assistant."},
  {"role": "user", "content": "Hello, how are you?"},
  {"role": "assistant", "content": "I'm doing great. How can I help you today?"},
  {"role": "user", "content": "I'd like to show off how chat templating works!"},
]
print(tokenizer.apply_chat_template(chat, tokenize=False))

Helpful usage tips

MathJSON

Ask mathematical questions prefixed with "Create a MathJSON solution to the following:". The model will output a formula that can be executed by https://cortexjs.io/compute - engine/.

There is also a (hacky, incomplete, semi - functional) python version in mathjson.py.

⚠️ Important Note

Use a very low temperature!

Example:

[INST] <<SYS>>
You are an assistant with vast knowledge in all things math.
<</SYS>>

Create a MathJSON solution to the following: Calculate the area of a circle with radius of 17.2456cm.  Include your reasoning. [/INST]

Output:

The area (A) of a circle is given by the formula A = πr², where r is the radius of the circle. Since the radius is given in centimeters, we need to convert it to meters first. We know that 1 meter equals 100 centimeters. Therefore, we can represent the radius as: r = 17.2456 cm * 100/1.

Once we have the radius in meters, we can calculate the area of the circle using the formula A = πr².

Solution as MathJSON:
<mathjson>
[
  "Multiply",
  "Pi",
  [
    "Power",
    17.2456,
    2
  ]
]
</mathjson>

You can validate the JSON between <mathjson> and </mathjson> and then pass the parsed JSON to compute - engine JS or the evaluate function in mathjson.py to calculate the response.

Context obedient question answering

The model was trained to ignore its prior knowledge and use the context to answer questions. It was also tuned to limit values to the provided context to reduce hallucinations.

The format for a closed - context prompt is as follows:

BEGININPUT
BEGINCONTEXT
[key0: value0]
[key1: value1]
... other metdata ...
ENDCONTEXT
[insert your text blocks here]
ENDINPUT
[add as many other blocks, in the exact same format]
BEGININSTRUCTION
[insert your instruction(s).  The model was tuned with single questions, paragraph format, lists, etc.]
ENDINSTRUCTION

It's helpful to add "Don't make up answers if you don't know." to your instruction block to avoid the model making up answers when the context is unrelated.

⚠️ Important Note

The only prompts that need this closed - context formatting are closed - context instructions. Normal questions/instructions do not!

The explicit delimiters help the model understand where to find responses and how to associate specific sources:

• BEGININPUT - denotes a new input block
• BEGINCONTEXT - denotes the block of context (metadata key/value pairs) to associate with the current input block
• ENDCONTEXT - denotes the end of the metadata block for the current input
• [text] - Insert whatever text you want for the input block
• ENDINPUT - denotes the end of the current input block
• [repeat as many input blocks in this format as you want]
• BEGININSTRUCTION - denotes the start of the instruction(s) to respond to for all input blocks above
• [instruction(s)]
• ENDINSTRUCTION - denotes the end of the instruction set

It sometimes works without ENDINSTRUCTION, but including it helps the model better understand the instructions.

⚠️ Important Note

Use a very low temperature!

Example:

BEGININPUT
BEGINCONTEXT
date: 2021 - 01 - 01
url: https://web.site/123
ENDCONTEXT
In a shocking turn of events, blueberries are now green, but will be sticking with the same name.
ENDINPUT
BEGININSTRUCTION
What color are bluberries?  Source?
ENDINSTRUCTION

Response:

Blueberries are now green.
Source:
date: 2021 - 01 - 01
url: https://web.site/123

Summarization

500 samples from [this dataset](https://huggingface.co/datasets/mattpscott/airoboros - summarization) are included, using the same format as contextual question answering, e.g.:

BEGININPUT
{text to summarize}
ENDINPUT
BEGININSTRUCTION
Summarize the input in around 130 words.
ENDINSTRUCTION

Getting longer responses

You can use detailed prompts with explicit word - count requirements:

Please compose a narrative set in the heart of an ancient library, steeped in the scent of old parchment and ink. The protagonist should be a young scholar who is dedicated to studying the art of storytelling and its evolution throughout history. In her pursuit of knowledge, she stumbles upon a forgotten tome that seems to possess an unusual aura. This book has the ability to bring stories to life, literally manifesting characters and scenarios from within its pages into reality.

The main character must navigate through various epochs of storytelling - from oral traditions of tribal societies, through medieval minstrels' tales, to modern - day digital narratives - as they come alive around her. Each era presents its unique challenges and lessons about the power and impact of stories on human civilization.

One such character could be a sentient quill pen, who was once used by renowned authors of yesteryears and now holds their wisdom and experiences. It becomes her mentor, guiding her through this journey with witty remarks and insightful commentary.

Ensure that your tale encapsulates the thrill of adventure, the beauty of learning, and the profound connection between humans and their stories. All characters involved should be non - human entities. Feel free to explore creative liberties but maintain the mentioned elements.

Your response should be approximately 2300 words.

Or a simpler example:

Please create a long, detailed story about a dragon in an old growth forest who, for some reason, begins speaking the words of the source code of linux.

There are also examples of next - chapter completion, e.g.:

Write the next chapter of a historical fiction novel set in Paris during the 20th century.

Here's a summary of the previous chapter:
In the vibrant city of Paris, amid the tumultuous changes of the 20th century, our protagonist Margot, an aspiring fashion designer, has just secured an apprenticeship at a prestigious couture house. She meets Lucien, a charming journalist who covers the fashion industry. Together they navigate the ever - changing world of fashion and society, uncovering secrets that reveal the intricate links between style, politics, and culture. As the chapter concludes, they decide to delve deeper into the hidden corners of the fashion world to unravel its mysteries.

Requirements for the next chapter:

1. Character Development of Margot and Lucien:
- Margot's Evolution: Unfold more about Margot's past, her dreams of revolutionizing fashion, and her struggle to establish herself in a male - dominated industry. Illustrate her growing expertise, innovative ideas, and increasing dependence on Lucien.
- Lucien's Complexity: Introduce uncertainties surrounding Lucien's background and real motives. Increase suspense by suggesting undisclosed information he possesses, while also highlighting his wit and perceptiveness.

2. Exploration of Paris and the Couture House:
- Paris: Elaborate their journey through the bustling streets of Paris, including encounters with iconic figures, social unrest, and relics from different eras of French history.
- The Couture House: Expand on the grandeur of the couture house they work in, filled with artistic masterpieces, intense competition, and cryptic notes hinting at a scandalous past.

3. Emergence of the Subplot: The Lost Collection:
- Discovery: Have Margot and Lucien stumble upon a secret vault containing a lost collection designed before World War II, raising new questions about the previous owner and the influence of war on fashion.
- Revelation: Capture their shock as they realize the designs were plagiarized, the potential repercussions, and the opportunities it presents for Margot's career.
- Twist: End with a twist that suggests there are other stolen collections across Paris, setting up their new mission.

Your response should be approximately 650 words.

Coding

You can ask for complex coding instructions with multiple criteria, e.g.:

Create a python application with the following requirements:
- Asyncio FastAPI webserver
- ping endpoint that returns the current date in JSON format
- file upload endpoint, which calculates the file's sha256 checksum, and checks postgres to deduplicate

Or inline criteria:

Write a multi - threaded TCP server in C that accepts a "GET [key]" input and "SET [key] [value]" input, and uses a binary tree to get and store the input values.

You can optionally add a single space and "PLAINFORMAT" at the end of your prompt to get just the code, e.g.:

Write a websocket application in node.js. PLAINFORMAT

Agent/function calling

The dataset includes many examples of function/args generation based on input criteria, similar to OpenAI function calling, with the output in JSON or YAML.

Example prompt:

As an AI assistant, please select the most suitable function and parameters from the list of available functions below, based on the user's input. Provide your response in JSON format.

Input: I want to know how many times 'Python' is mentioned in my text file.

Available functions:
file_analytics:
  description: This tool performs various operations on a text file.
  params:
    action: The operation we want to perform on the data, such as "count_occurrences", "find_line", etc.
    filters:
      keyword: The word or phrase we want to search for.

Response:

{
  "function": "file_analytics",
  "params": {
    "action": "count_occurrences",
    "filters": {
      "keyword": "Python"
    }
  }
}

📄 License

This project is licensed under the Apache - 2.0 license.