Medalpaca 13b

🚀 MedAlpaca 13b

A large language model fine - tuned for medical domain tasks, based on LLaMA with 13 billion parameters, aiming to improve medical question - answering and dialogue.

📚 Table of Contents

• 🔍 Model Description
  • 🏗️ Architecture
  • 📊 Training Data
• 💻 Model Usage
• ⚠️ Limitations

🔍 Model Description

🏗️ Architecture

medalpaca-13b is a large language model specifically fine - tuned for medical domain tasks. It is based on LLaMA (Large Language Model Meta AI) and contains 13 billion parameters. The primary goal of this model is to improve question - answering and medical dialogue tasks.

📊 Training Data

The training data for this project was sourced from various resources.

• Firstly, we used Anki flashcards to automatically generate questions from the front of the cards and answers from the back of the card.
• Secondly, we generated medical question - answer pairs from Wikidoc. We extracted paragraphs with relevant headings, and used Chat - GPT 3.5 to generate questions from the headings and used the corresponding paragraphs as answers. This dataset is still under development and we believe that approximately 70% of these question - answer pairs are factually correct.
• Thirdly, we used StackExchange to extract question - answer pairs, taking the top - rated question from five categories: Academia, Bioinformatics, Biology, Fitness, and Health.
• Additionally, we used a dataset from ChatDoctor consisting of 200,000 question - answer pairs, available at https://github.com/Kent0n - Li/ChatDoctor.

💻 Model Usage

📈 Evaluation

To evaluate the performance of the model on a specific dataset, you can use the Hugging Face Transformers library's built - in evaluation scripts. Please refer to the evaluation guide for more information.

🚀 Inference

You can use the model for inference tasks like question - answering and medical dialogues using the Hugging Face Transformers library.

🔍 Basic Usage

from transformers import pipeline

pl = pipeline("text-generation", model="medalpaca/medalpaca-13b", tokenizer="medalpaca/medalpaca-13b")
question = "What are the symptoms of diabetes?"
context = "Diabetes is a metabolic disease that causes high blood sugar. The symptoms include increased thirst, frequent urination, and unexplained weight loss."
answer = pl(f"Context: {context}\n\nQuestion: {question}\n\nAnswer: ")
print(answer)

⚠️ Limitations

⚠️ Important Note

• The model may not perform effectively outside the scope of the medical domain.
• The training data primarily targets the knowledge level of medical students, which may result in limitations when addressing the needs of board - certified physicians.
• The model has not been tested in real - world applications, so its efficacy and accuracy are currently unknown. It should never be used as a substitute for a doctor's opinion and must be treated as a research tool only.