đ SynapseLLM
SynapseLLM, a remarkable achievement by WebraftAI, is a series of large language AI models crafted to build robust, generalized, and decentralized information systems. This repository specifically stores the finetuned version of Mistral for SynapseLLM. The finetuning is carried out on a custom - albeit limited - dataset, concentrating on code and normal question - answering scenarios. This adaptation demonstrates the model's flexibility and applicability in specific domains, contributing to the broader AI advancement landscape.
đ Quick Start
To use the SynapseLLM model, you can follow the example code below. This code uses the transformers
library provided by Hugging Face to load the model and generate text.
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("WebraftAI/synapsellm-7b-mistral-v0.4-preview2")
model = AutoModelForCausalLM.from_pretrained("WebraftAI/synapsellm-7b-mistral-v0.4-preview2")
prompt= "<s>[INST] Hello! [/INST] "
device = "cuda"
model_inputs = tokenizer([prompt], return_tensors="pt").to(device)
model.to(device)
generated_ids = model.generate(**model_inputs, max_new_tokens=100, do_sample=True)
print(tokenizer.batch_decode(generated_ids)[0])
⨠Features
- Versatility: Trained on a diverse dataset including Maths Instruct Q/A, GPT - 3.5 Q/A, General Code, Python code, and General Q/A, the model can handle various tasks such as chat Q/A and code - related instructions.
- Ease of Use: It's a full model merged and compiled with trained adapters, which can be easily loaded through the
transformers
library.
đĻ Installation
The example code uses the transformers
library. You can install it using the following command:
pip install transformers
đģ Usage Examples
Basic Usage
The basic usage is shown in the example code above. You just need to replace the prompt with your own text to get the model's response.
Advanced Usage
You can adjust the parameters in the generate
method to control the text generation process. For example, you can change the max_new_tokens
to control the length of the generated text, and set do_sample
to False
for more deterministic output.
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("WebraftAI/synapsellm-7b-mistral-v0.4-preview2")
model = AutoModelForCausalLM.from_pretrained("WebraftAI/synapsellm-7b-mistral-v0.4-preview2")
prompt= "<s>[INST] Can you solve this math problem: 2 + 3? [/INST] "
device = "cuda"
model_inputs = tokenizer([prompt], return_tensors="pt").to(device)
model.to(device)
generated_ids = model.generate(**model_inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.batch_decode(generated_ids)[0])
đ Documentation
Model Details
Property |
Details |
Model Type |
Decoder-only Transformer |
Parameters |
7B |
Learning rate |
2e-4 |
Adapter used |
Qlora |
Precision |
float16 |
Batch size |
32 |
Maximum gradient normal |
0.3 |
Optimizer |
paged_adamw_32bit |
Warmup Ratio |
0.03 |
Step(s) (trained) |
150 |
Epoch(s) (trained) |
1 |
Model Description
This is a 7b parameter, decoder - only transformer - based finetuned model on Chat Q/A and Code instructions. It's a preview finetune on Mistral 7B v0.1 on a sample dataset of 770k rows comprising of 361k Maths Instruct Q/A, 143k GPT - 3.5 Q/A, 140k General Code, 63k Python code, and 54k General Q/A (Through GPT - 4) [Each row contains one instruction and one response].
Prompt format
This model follows the same prompt format as mistral instruct 7b v0.1. The sample prompt is as follows:
<s>[INST] Hello, how are you? [/INST]
Model Bias
- The model might output factually incorrect information.
- It does not follow system prompts.
- The model does not have any kind of memory, and researchers can experiment with feeding memory.
- Since the model is trained on different data, it can bias information or claim itself as a GPT model.
đ§ Technical Details
This model is a finetuned version of Mistral 7B v0.1. The finetuning process is conducted on a custom dataset with specific hyperparameters. The model uses a Qlora adapter and is trained with a paged_adamw_32bit optimizer. The learning rate is set to 2e - 4, and the batch size is 32. The model is trained for 150 steps and 1 epoch.
đ License
This model is licensed under the Apache 2.0 license.
Detailed results can be found here
Metric |
Value |
Avg. |
55.93 |
AI2 Reasoning Challenge (25 - Shot) |
52.99 |
HellaSwag (10 - Shot) |
74.54 |
MMLU (5 - Shot) |
54.60 |
TruthfulQA (0 - shot) |
53.79 |
Winogrande (5 - shot) |
73.95 |
GSM8k (5 - shot) |
25.70 |