Biomistral 7B

🚀 BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains

Large Language Models (LLMs) have shown great potential in medical domains. BioMistral is an open - source LLM tailored for the biomedical domain, offering superior performance in medical question - answering tasks and multilingual evaluation.

✨ Features

• Domain - Specific Adaptation: BioMistral is an open - source LLM based on Mistral, further pre - trained on PubMed Central for the biomedical domain.
• Multilingual Evaluation: It undergoes large - scale multilingual evaluation on 7 languages in addition to English, marking the first of its kind in the medical domain.
• Model Variants: There are different model variants, including merged models and quantized models, providing flexibility for various use cases.
• Superior Performance: Outperforms existing open - source medical models and competes well against proprietary counterparts in medical question - answering tasks.

📦 Installation

You can use BioMistral with Hugging Face's Transformers library as follows.

Basic Usage

from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("BioMistral/BioMistral-7B")
model = AutoModel.from_pretrained("BioMistral/BioMistral-7B")

💻 Usage Examples

Loading the model and tokenizer

from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("BioMistral/BioMistral-7B")
model = AutoModel.from_pretrained("BioMistral/BioMistral-7B")

📚 Documentation

BioMistral models

Quantized Models

Supervised Fine - tuning Benchmark

Supervised Fine - Tuning (SFT) performance of BioMistral 7B models compared to baselines, measured by accuracy (↑) and averaged across 3 random seeds of 3 - shot. DARE, TIES, and SLERP are model merging strategies that combine BioMistral 7B and Mistral 7B Instruct. Best model in bold, and second - best underlined. *GPT - 3.5 Turbo performances are reported from the 3 - shot results without SFT.

📄 License

The project is licensed under the Apache - 2.0 license.

📚 Citation

Arxiv : https://arxiv.org/abs/2402.10373

@misc{labrak2024biomistral,
      title={BioMistral: A Collection of Open - Source Pretrained Large Language Models for Medical Domains}, 
      author={Yanis Labrak and Adrien Bazoge and Emmanuel Morin and Pierre - Antoine Gourraud and Mickael Rouvier and Richard Dufour},
      year={2024},
      eprint={2402.10373},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

⚠️ Important Note

Although BioMistral is intended to encapsulate medical knowledge sourced from high - quality evidence, it hasn't been tailored to effectively, safely, or suitably convey this knowledge within professional parameters for action. Both direct and downstream users need to be informed about the risks, biases, and constraints inherent in the model. Our exploration of its capabilities and limitations is just beginning. In fields such as medicine, comprehending these limitations is crucial.

💡 Usage Tip

We advise refraining from utilizing BioMistral in medical contexts unless it undergoes thorough alignment with specific use cases and undergoes further testing, notably including randomized controlled trials in real - world medical environments. BioMistral 7B may possess inherent risks and biases that have not yet been thoroughly assessed. Additionally, the model's performance has not been evaluated in real - world clinical settings. Consequently, we recommend using BioMistral 7B strictly as a research tool and advise against deploying it in production environments for natural language generation or any professional health and medical purposes.