Soulchat

🚀 SoulChat: Mental Health Large Model

SoulChat enhances the "empathy" ability of large models through mixed fine - tuning of long - text consultation instructions and multi - round empathetic dialogue datasets.

SoulChat   |   BianQue  |

Based on the six characteristics of proactive health, namely initiative, prevention, precision, personalization, co - construction and sharing, and self - discipline, the Key Laboratory of Digital Twin Humans of Guangdong Province, School of Future Technology, South China University of Technology, has open - sourced the ProactiveHealthGPT, a large - model base for proactive health in the Chinese - language living space. It includes:

• The BianQue, a large - scale health model for living spaces, which has been fine - tuned with tens of millions of Chinese health dialogue data instructions.
• The SoulChat, a large - scale mental health model, which has been fine - tuned with millions of Chinese long - text instructions and multi - round empathetic dialogue data in the field of psychological counseling.

We hope that the ProactiveHealthGPT can help the academic community accelerate the research and application of large models in proactive health fields such as chronic diseases and psychological counseling. This project is about the SoulChat, a large - scale mental health model.

🚀 Quick Start

✨ Features

We investigated common psychological counseling platforms and found that when users seek online psychological help, they usually provide long self - descriptions, and the counselors also give long responses (see https://github.com/scutcyr/SoulChat/blob/main/figure/single_turn.png). There is a lack of a progressive process of pouring out feelings. In actual psychological counseling, there are multiple rounds of communication between users and counselors. During this process, counselors will guide users to talk and provide empathy, such as "Very good", "I understand how you feel", "Of course you can", etc.

Considering the lack of multi - round empathetic dialogue datasets, on the one hand, we constructed over 150,000 single - round long - text psychological counseling instructions and answers (SoulChatCorpus - single_turn), with over 500,000 answers (the number of instructions is 6.7 times that of the current common psychological counseling dataset [PsyQA](https://github.com/thu - coai/PsyQA)). We also used ChatGPT and GPT4 to generate about 1 million rounds of multi - round answer data (SoulChatCorpus - multi_turn). Specifically, in pre - experiments, we found that a psychological counseling model driven purely by single - round long - text data would generate text that is too long and annoying for users and lacks the ability to guide users to talk. A model driven purely by multi - round psychological counseling dialogue data weakens the model's ability to give advice. Therefore, we combined SoulChatCorpus - single_turn and SoulChatCorpus - multi_turn to form the SoulChatCorpus, a mixed empathetic dialogue dataset of single - round and multi - round data with over 1.2 million samples. All data is unified into an instruction format of "User: xxx\nPsychological Counselor: xxx\nUser: xxx\nPsychological Counselor:".

We selected [ChatGLM - 6B](https://huggingface.co/THUDM/chatglm - 6b) as the initial model and performed full - parameter instruction fine - tuning to enhance the model's empathy ability, the ability to guide users to talk, and the ability to provide reasonable advice. More training details will be provided in our subsequent published papers.

📦 Installation

• Clone the project

cd ~
git clone https://github.com/scutcyr/SoulChat.git

• Install dependencies Note that the version of torch needs to be selected according to the actual cuda version of your server. For details, refer to the [pytorch installation guide](https://pytorch.org/get - started/previous - versions/).

cd SoulChat
conda env create -n proactivehealthgpt_py38 --file proactivehealthgpt_py38.yml
conda activate proactivehealthgpt_py38
pip install cpm_kernels
pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra - index - url https://download.pytorch.org/whl/cu116

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoModel, AutoTokenizer
# GPU settings
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# Load the model and tokenizer
model_name_or_path = 'scutcyr/SoulChat'
model = AutoModel.from_pretrained(model_name_or_path, trust_remote_code=True).half()
model.to(device)
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)

# Call the model's chat function for single - round dialogue
user_input = "I'm so sad because I broke up with my girlfriend!"
input_text = "User: " + user_input + "\nPsychological Counselor: "
response, history = model.chat(tokenizer, query=input_text, history=None, max_length=2048, num_beams=1, do_sample=True, top_p=0.75, temperature=0.95, logits_processor=None)

# Call the model's chat function for multi - round dialogue
# Note: This project uses "\nUser: " and "\nPsychological Counselor: " to divide the dialogue history of different rounds
# Note: The length of user_history is one more than that of bot_history
user_history = ['Hello, teacher', 'My girlfriend broke up with me. I feel so sad']
bot_history = ['Hello! I am your personal digital counselor, Teacher Sweetheart. Welcome to talk to me. I look forward to helping you!']
# Concatenate the dialogue history
context = "\n".join([f"User: {user_history[i]}\nPsychological Counselor: {bot_history[i]}" for i in range(len(bot_history))])
input_text = context + "\nUser: " + user_history[-1] + "\nPsychological Counselor: "

response, history = model.chat(tokenizer, query=input_text, history=None, max_length=2048, num_beams=1, do_sample=True, top_p=0.75, temperature=0.95, logits_processor=None)

Advanced Usage

streamlit run soulchat_app.py --server.port 9026

Specifically, in soulchat_app.py, you can modify the following code to change the specified graphics card:

os.environ['CUDA_VISIBLE_DEVICES'] = '2'

You can specify the model path as a local path by changing the following code:

model_name_or_path = 'scutcyr/SoulChat'

📚 Documentation

Recent Updates

• 👏🏻 On June 6, 2023: The BianQue - 2.0 model was open - sourced. For details, see [BianQue - 2.0](https://huggingface.co/scutcyr/BianQue - 2).
• 👏🏻 On June 6, 2023: The SoulChat mental health large model with empathy and listening abilities was released. For details, see: SoulChat: Mental Health Large Model: Enhancing the "Empathy" Ability of Large Models through Mixed Fine - Tuning of Long - Text Consultation Instructions and Multi - Round Empathetic Dialogue Datasets.
• 👏🏻 On April 22, 2023: A medical Q&A system demo based on the BianQue - 1.0 model was released. For details, visit: https://huggingface.co/spaces/scutcyr/BianQue.
• 👏🏻 On April 22, 2023: The BianQue - 1.0 model was released. For details, see: [BianQue - 1.0: Improving the "Question" Ability of Medical Chat Models through Fine - Tuning with Hybrid Instructions and Multi - Turn Doctor QA Datasets](https://huggingface.co/scutcyr/BianQue - 1.0).

Declaration

• This project uses the weights of the ChatGLM - 6B model and needs to follow its [MODEL_LICENSE](https://github.com/THUDM/ChatGLM - 6B/blob/main/MODEL_LICENSE). Therefore, this project can only be used for your non - commercial research purposes.
• The SoulChat model provided in this project aims to improve the empathetic dialogue and listening ability of large models. However, the output text of the model has a certain degree of randomness. While it is suitable as a listener, it is not recommended to replace the diagnosis and advice of psychologists with the output text of the SoulChat model. This project does not guarantee that the output text of the model is completely suitable for users, and users need to bear all risks when using this model!
• You shall not use, copy, modify, merge, publish, distribute, reproduce, or create derivative works of all or part of the SoulChat model for any commercial, military, or illegal purposes.
• You shall not use the SoulChat model to engage in any activities that endanger national security and national unity, harm social and public interests, or infringe on personal rights.
• When using the SoulChat model, you should be aware that it cannot replace professionals such as doctors and psychologists. You should not overly rely on, obey, or trust the model's output, and you should not be addicted to chatting with the SoulChat model for a long time.

Acknowledgments

This project was initiated by the Key Laboratory of Digital Twin Humans of Guangdong Province, School of Future Technology, South China University of Technology (https://www2.scut.edu.cn/ft/main.htm) and supported by the Information Network Engineering Research Center of South China University of Technology. We also thank our cooperation units, such as the Guangdong Women and Children Hospital, Guangzhou Women and Children's Medical Center, and the Third Affiliated Hospital of Sun Yat - sen University.

Citation

@misc{chen2023soulchat,
      title={SoulChat: Mental Health Large Model: Enhancing the "Empathy" Ability of Large Models through Mixed Fine - Tuning of Long - Text Consultation Instructions and Multi - Round Empathetic Dialogue Datasets},
      author={Yirong Chen, Xiaofen Xing, Zhenyu Wang, Xiangmin Xu},
      year={2023},
      month = {6},
      version = {1.0},
      url = {https://github.com/scutcyr/SoulChat}
}

📄 License

The license of this project is Apache - 2.0.

Inference Parameters

Widget Examples

• Text: User: I'm approaching an exam, and I feel like I haven't learned a lot of the content. I'm so nervous.\nPsychological Counselor:
• Text: User: Hello\nPsychological Counselor: Hello! I'm your personal digital counselor, Teacher Sweetheart. Welcome to talk to me. I look forward to helping you!\nUser: I'm approaching an exam, and I feel like I haven't learned a lot of the content. I'm so nervous.\nPsychological Counselor:
• Text: User: What should I do if I'm under a lot of exam pressure?\nPsychological Counselor: