Synapsellm 7b Mistral V0.4 Preview2

🚀 SynapseLLM

SynapseLLM, a remarkable achievement by WebraftAI, is a series of large language AI models crafted to build robust, generalized, and decentralized information systems. This repository specifically stores the finetuned version of Mistral for SynapseLLM. The finetuning is carried out on a custom - albeit limited - dataset, concentrating on code and normal question - answering scenarios. This adaptation demonstrates the model's flexibility and applicability in specific domains, contributing to the broader AI advancement landscape.

🚀 Quick Start

To use the SynapseLLM model, you can follow the example code below. This code uses the transformers library provided by Hugging Face to load the model and generate text.

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("WebraftAI/synapsellm-7b-mistral-v0.4-preview2")
model = AutoModelForCausalLM.from_pretrained("WebraftAI/synapsellm-7b-mistral-v0.4-preview2")

prompt= "<s>[INST] Hello!  [/INST] "

device = "cuda"

model_inputs = tokenizer([prompt], return_tensors="pt").to(device)
model.to(device)

generated_ids = model.generate(**model_inputs, max_new_tokens=100, do_sample=True)
print(tokenizer.batch_decode(generated_ids)[0])

✨ Features

• Versatility: Trained on a diverse dataset including Maths Instruct Q/A, GPT - 3.5 Q/A, General Code, Python code, and General Q/A, the model can handle various tasks such as chat Q/A and code - related instructions.
• Ease of Use: It's a full model merged and compiled with trained adapters, which can be easily loaded through the transformers library.

📦 Installation

The example code uses the transformers library. You can install it using the following command:

pip install transformers

💻 Usage Examples

Basic Usage

The basic usage is shown in the example code above. You just need to replace the prompt with your own text to get the model's response.

Advanced Usage

You can adjust the parameters in the generate method to control the text generation process. For example, you can change the max_new_tokens to control the length of the generated text, and set do_sample to False for more deterministic output.

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("WebraftAI/synapsellm-7b-mistral-v0.4-preview2")
model = AutoModelForCausalLM.from_pretrained("WebraftAI/synapsellm-7b-mistral-v0.4-preview2")

prompt= "<s>[INST] Can you solve this math problem: 2 + 3? [/INST] "

device = "cuda"

model_inputs = tokenizer([prompt], return_tensors="pt").to(device)
model.to(device)

generated_ids = model.generate(**model_inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.batch_decode(generated_ids)[0])

📚 Documentation

Model Details

Model Description

This is a 7b parameter, decoder - only transformer - based finetuned model on Chat Q/A and Code instructions. It's a preview finetune on Mistral 7B v0.1 on a sample dataset of 770k rows comprising of 361k Maths Instruct Q/A, 143k GPT - 3.5 Q/A, 140k General Code, 63k Python code, and 54k General Q/A (Through GPT - 4) [Each row contains one instruction and one response].

Prompt format

This model follows the same prompt format as mistral instruct 7b v0.1. The sample prompt is as follows:

<s>[INST] Hello, how are you? [/INST]

Model Bias

• The model might output factually incorrect information.
• It does not follow system prompts.
• The model does not have any kind of memory, and researchers can experiment with feeding memory.
• Since the model is trained on different data, it can bias information or claim itself as a GPT model.

🔧 Technical Details

This model is a finetuned version of Mistral 7B v0.1. The finetuning process is conducted on a custom dataset with specific hyperparameters. The model uses a Qlora adapter and is trained with a paged_adamw_32bit optimizer. The learning rate is set to 2e - 4, and the batch size is 32. The model is trained for 150 steps and 1 epoch.

📄 License

This model is licensed under the Apache 2.0 license.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here