Mentallama Chat 13B

🚀 MentaLLaMA-chat-13B

MentaLLaMA-chat-13B is an open - source large language model for interpretable mental health analysis with instruction - following capability, finetuned on LLaMA2 - chat - 13B and IMHI data.

🚀 Quick Start

MentaLLaMA-chat-13B is part of the MentaLLaMA project, the first open - source large language model (LLM) series for interpretable mental health analysis with instruction - following capability. This model is finetuned based on the Meta LLaMA2 - chat - 13B foundation model and the full IMHI instruction tuning data.

It is expected to conduct complex mental health analyses for various mental health conditions and offer reliable explanations for each of its predictions. The model is fine - tuned on the IMHI dataset with 75K high - quality natural language instructions to enhance its performance in downstream tasks. A comprehensive evaluation on the IMHI benchmark with 20K test samples shows that MentalLLaMA approaches state - of - the - art discriminative methods in correctness and generates high - quality explanations.

✨ Features

• Interpretable Mental Health Analysis: Capable of making complex mental health analyses and providing reliable explanations for predictions.
• Instruction - Following: Can follow natural language instructions for mental health analysis tasks.
• High - Quality Training: Fine - tuned on a large dataset of 75K high - quality natural language instructions.

💻 Usage Examples

Basic Usage

from transformers import LlamaTokenizer, LlamaForCausalLM
tokenizer = LlamaTokenizer.from_pretrained('klyang/MentaLLaMA-chat-13B')
model = LlamaForCausalLM.from_pretrained('klyang/MentaLLaMA-chat-13B', device_map='auto')

In this example, LlamaTokenizer is used to load the tokenizer, and LlamaForCausalLM is used to load the model. The device_map='auto' argument is used to automatically use the GPU if it's available.

📚 Documentation

Ethical Consideration

Although experiments on MentaLLaMA show promising performance on interpretable mental health analysis, we stress that all predicted results and generated explanations should only be used for non - clinical research. Help - seekers should get assistance from professional psychiatrists or clinical practitioners.

Recent studies have indicated that LLMs may introduce some potential bias, such as gender gaps. Meanwhile, some incorrect prediction results, inappropriate explanations, and over - generalization also illustrate the potential risks of current LLMs. Therefore, there are still many challenges in applying the model to real - scenario mental health monitoring systems.

Other Models in MentaLLaMA

In addition to MentaLLaMA - chat - 13B, the MentaLLaMA project includes other models: MentaLLaMA - chat - 7B, MentalBART, MentalT5.

• MentaLLaMA - chat - 7B: Finetuned based on the Meta LLaMA2 - chat - 7B foundation model and the full IMHI instruction tuning data. The training data covers 10 mental health analysis tasks.
• MentalBART: Finetuned based on the BART - large foundation model and the full IMHI - completion data. The training data covers 10 mental health analysis tasks. This model doesn't have instruction - following ability but is more lightweight and performs well in interpretable mental health analysis in a completion - based manner.
• MentalT5: Finetuned based on the T5 - large foundation model and the full IMHI - completion data. The training data covers 10 mental health analysis tasks. This model doesn't have instruction - following ability but is more lightweight and performs well in interpretable mental health analysis in a completion - based manner.

📄 License

MentaLLaMA - chat - 13B is licensed under MIT. For more details, please see the MIT file.

Citation

If you use MentaLLaMA - chat - 7B in your work, please cite our paper:

@misc{yang2023mentalllama,
      title={MentalLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models}, 
      author={Kailai Yang and Tianlin Zhang and Ziyan Kuang and Qianqian Xie and Sophia Ananiadou},
      year={2023},
      eprint={2309.13567},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}