Vicuna Baichuan 13b Chat

🚀 Vicuna Baichuan-13B Model

A Vicuna fine-tuned dialogue model based on Baichaun-13B-chat

The shibing624/vicuna-baichuan-13b-chat model is evaluated on test data. The overall performance of the model on the QA test is as follows:

This model performs excellently in the Chinese open test set, inheriting the advantages of two aspects:

1. The base model for fine-tuning is the baichuan-inc/Baichuan-13B-Chat model, which is a powerful Chinese-English base model.
2. The fine-tuning uses a high-quality dataset of 100,000 multilingual GPT4 dialogues. After fine-tuning, the model has reached a leading level in general conversations, including the ability to answer medical questions. Its ability to answer general questions is not inferior to that of Baichuan-13B-Chat.

📚 Documentation

Training details

The training arguments are as follows:

{"per_device_train_batch_size": 2,
"per_device_eval_batch_size": 2,
"gradient_accumulation_steps": 1, 
"learning_rate": 2e-05, 
"num_train_epochs": 3.0,
"max_steps": -1,
"lr_scheduler_type": "linear",
"warmup_ratio": 0.0,
"warmup_steps": 50, 
"save_strategy": "steps",
"save_steps": 500,
"save_total_limit": 13,
"seed": 42,
"bf16": true,
"fp16": false, 
"eval_steps": 50,
"remove_unused_columns": false,
"load_best_model_at_end": true,
"metric_for_best_model": "loss", 
"optim": "paged_adamw_32bit",
"report_to": ["tensorboard"]}

• Training loss: 1.3209
• Evaluation loss: 1.4953

The following bitsandbytes quantization configuration was used during training:

• load_in_8bit: False
• load_in_4bit: True
• llm_int8_threshold: 6.0
• llm_int8_skip_modules: None
• llm_int8_enable_fp32_cpu_offload: False
• llm_int8_has_fp16_weight: False
• bnb_4bit_quant_type: nf4
• bnb_4bit_use_double_quant: True
• bnb_4bit_compute_dtype: bfloat16

💻 Usage Examples

Basic Usage

This project is open-sourced on the following GitHub repositories:

• shibing624/textgen
• shibing624/MedicalGPT

You can use the textgen library to call the Baichuan/LLaMA models.

Install the package:

pip install -U textgen

from textgen import GptModel

def generate_prompt(instruction):
    return f"""A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.USER: {instruction} ASSISTANT: """

model = GptModel("baichuan", "shibing624/vicuna-baichuan-13b-chat")
predict_sentence = generate_prompt("What kind of medicine can a one-year-old baby take when having a fever?")
r = model.predict([predict_sentence])
print(r) # ["1. First, most pediatric antipyretics contain antipyretic and analgesic ingredients such as Aspirin or Ibuprofen. Although these drugs have fewer side effects, there is also a risk of gastrointestinal damage or even gastrointestinal ulcer bleeding if used in excess..."]

Advanced Usage

Without using the textgen library, you can use the model as follows:

First, pass your input through the transformer model, and then you can get the generated sentence.

Install the package:

pip install transformers 

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig

model = AutoModelForCausalLM.from_pretrained("shibing624/vicuna-baichuan-13b-chat", device_map='auto', torch_dtype=torch.float16, trust_remote_code=True)
model.generation_config = GenerationConfig.from_pretrained("shibing624/vicuna-baichuan-13b-chat", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("shibing624/vicuna-baichuan-13b-chat", trust_remote_code=True)
device = torch.device(0) if torch.cuda.is_available() else torch.device("cpu")

def generate_prompt(instruction):
    return f"""A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.USER: {instruction} ASSISTANT: """

sents = ['What kind of medicine can a one-year-old baby take when having a fever?', "Who are you?"]
for s in sents:
    q = generate_prompt(s)
    inputs = tokenizer(q, return_tensors="pt")
    inputs = inputs.to(device)

    generate_ids = model.generate(
        **inputs,
        max_new_tokens=512, 
    )

    output = tokenizer.batch_decode(generate_ids, skip_special_tokens=True)[0]
    print(output)
    print()

Output:

What kind of medicine can a one-year-old baby take when having a fever?
A child's body is very fragile. If you don't take good care of yourself, you may get various diseases. Some parents are very anxious when they see their children get sick and want to give them medicine immediately. But in fact, it's not that simple because everyone's sensitivity and tolerance to drugs are different. So we need to choose the appropriate treatment method according to the specific situation to achieve the best effect. 1. First, most pediatric antipyretics contain antipyretic and analgesic ingredients such as Aspirin or Ibuprofen. Although these drugs have fewer side effects, there is also a risk of gastrointestinal damage or even gastrointestinal ulcer bleeding if used in excess. 2. Some pediatric antipyretics contain antihistamines such as Chlorpheniramine. They are prescription drugs and should be taken strictly according to the doctor's advice. Do not increase or decrease the dosage or extend the medication time without permission. 3. In addition, some pediatric antipyretics contain nutritional supplements such as Vitamin C and Calcium Gluconate. They should only be considered when the child has dehydration symptoms (such as thirst and dark urine). Otherwise, it is easy to cause electrolyte imbalance.

Model file composition:

vicuna-baichuan-13b-chat
├── config.json
├── configuration_baichuan.py
├── generation_config.json
├── modeling_baichuan.py
├── pytorch_model-00001-of-00003.bin
├── pytorch_model-00002-of-00003.bin
├── pytorch_model-00003-of-00003.bin
├── pytorch_model.bin.index.json
├── quantizer.py
├── special_tokens_map.json
├── tokenization_baichuan.py
├── tokenizer_config.json
└── tokenizer.model

• Inference GPU: 27G

Inference Examples

📄 License

This project is licensed under the "apache-2.0" license.