Qwen2 7B Instruct

🚀 Rubra-Qwen2-7B-Instruct

This model is the result of further post - training on Qwen/Qwen2-7B-Instruct. It excels in handling complex multi - turn tool/function calling, offering high - quality text generation capabilities.

🚀 Quick Start

Prerequisites

Before using the model, make sure you have Node.js and npm installed. Also, install the jsonrepair package to fix rare hallucinations from the model.

npm install jsonrepair

Installation

You can use the model with the Hugging Face transformers and the rubra library [rubra - tools](https://github.com/rubra - ai/rubra - tools). Install the necessary Python packages:

pip install rubra_tools torch==2.3.0 transformers accelerate

✨ Features

• Function Calling: Capable of complex multi - turn tool/function calling.
• Multi - Language Support: Supports both English and Chinese.

📦 Installation Guide

The installation steps are as follows:

1. Install the jsonrepair package via npm:

npm install jsonrepair

2. Install the required Python packages:

pip install rubra_tools torch==2.3.0 transformers accelerate

💻 Usage Examples

Basic Usage

1. Load the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
from rubra_tools import preprocess_input, postprocess_output

model_id = "rubra-ai/Qwen2-7B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

2. Define Functions

Here we use 4 functions for a simple math chaining question:

functions = [
    {
            'type': 'function',
            'function': {
                'name': 'addition',
                'description': "Adds two numbers together",
                'parameters': {
                    'type': 'object',
                    'properties': {
                        'a': {
                            'description': 'First number to add',
                            'type': 'string'
                        },
                        'b': {
                            'description': 'Second number to add',
                            'type': 'string'
                        }
                    },
                    'required': []
                }
            }
        },
        {
            'type': 'function',
            'function': {
                'name': 'subtraction',
                'description': "Subtracts two numbers",
                'parameters': {
                    'type': 'object',
                    'properties': {
                        'a': {
                            'description': 'First number to be subtracted from',
                            'type': 'string'
                        },
                        'b': {
                            'description': 'Number to subtract',
                            'type': 'string'
                        }
                    },
                    'required': []
                }
            }
        },
        {
            'type': 'function',
            'function': {
                'name': 'multiplication',
                'description': "Multiply two numbers together",
                'parameters': {
                    'type': 'object',
                    'properties': {
                        'a': {
                            'description': 'First number to multiply',
                            'type': 'string'
                        },
                        'b': {
                            'description': 'Second number to multiply',
                            'type': 'string'
                        }
                    },
                    'required': []
                }
            }
        },
        {
            'type': 'function',
            'function': {
                'name': 'division',
                'description': "Divide two numbers",
                'parameters': {
                    'type': 'object',
                    'properties': {
                        'a': {
                            'description': 'First number to use as the dividend',
                            'type': 'string'
                        },
                        'b': {
                            'description': 'Second number to use as the divisor',
                            'type': 'string'
                        }
                    },
                    'required': []
                }
            }
        },
]

3. Start the conversation

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the result of four plus six? Take the result and add 2? Then multiply by 5 and then divide by two"},
]

def run_model(messages, functions):
    ## Format messages in Rubra's format
    formatted_msgs = preprocess_input(msgs=messages, tools=functions)

    text = tokenizer.apply_chat_template(
        formatted_msgs,
        tokenize=False,
        add_generation_prompt=True
    )
    model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

    generated_ids = model.generate(
        model_inputs.input_ids,
        max_new_tokens=512
    )
    generated_ids = [
        output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
    ]

    response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
    return response


raw_output = run_model(messages, functions)
# Check if there's a function call
function_call = postprocess_output(raw_output)
if function_call:
    print(function_call)
else:
    print(raw_output)

You should see this output, which is a function call made by the AI assistant:

[{'id': 'fc65a533', 'function': {'name': 'addition', 'arguments': '{"a": "4", "b": "6"}'}, 'type': 'function'}]

4. Add Executed Tool Result to Message History & Continue the Conversation

if function_call:
    # append the assistant tool call msg
    messages.append({"role": "assistant", "tool_calls": function_call})
    # append the result of the tool call in openai format, in this case, the value of add 6 to 4 is 10.
    messages.append({'role': 'tool', 'tool_call_id': function_call[0]["id"], 'name': function_call[0]["function"]["name"], 'content': '10'})
    raw_output1 = run_model(messages, functions)
    # Check if there's a function call

    function_call = postprocess_output(raw_output1)
    if function_call:
        print(function_call)
    else:
        print(raw_output)

The LLM will make another call

[{'id': '2ffc3de4', 'function': {'name': 'addition', 'arguments': '{"a": "10", "b": "2"}'}, 'type': 'function'}]

📚 Documentation

Model description

The model is the result of further post - training Qwen/Qwen2-7B-Instruct. It is capable of complex multi - turn tool/function calling.

Training

The model was post - trained (freeze tuned & DPO) on a proprietary dataset consisting of diverse function calling, chat, and instruct data.

🔧 Technical Details

Framework Versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1

📄 License

This model is licensed under the Apache - 2.0 license.

⚠️ Limitations and Bias

While the model performs well on a wide range of tasks, it may still produce biased or incorrect outputs. Users should exercise caution and critical judgment when using the model in sensitive or high - stakes applications. The model's outputs are influenced by the data it was trained on, which may contain inherent biases.

💡 Ethical Considerations

Users should ensure that the deployment of this model adheres to ethical guidelines and consider the potential societal impact of the generated text. Misuse of the model for generating harmful or misleading content is strongly discouraged.

🙏 Acknowledgements

We would like to thank Alibaba Cloud for the model.

📞 Contact Information

For questions or comments about the model, please reach out to the rubra team.

📖 Citation

If you use this work, please cite it as:

@misc {rubra_ai_2024,
	author       = { Sanjay Nadhavajhala and Yingbei Tong },
	title        = { Rubra-Qwen2-7B-Instruct },
	year         = 2024,
	url          = { https://huggingface.co/rubra-ai/Qwen2-7B-Instruct },
	doi          = { 10.57967/hf/2683 },
	publisher    = { Hugging Face }
}

📊 Model Performance