Tinyllama 1.1B Medical

🚀 TinyLlama 1.1B Medical 🤏🦙

A smaller and more efficient medical language model based on TinyLlama 1.1B, fine - tuned for medical text generation.

🚀 Quick Start

Basic Usage

from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM

config = PeftConfig.from_pretrained("therealcyberlord/TinyLlama-1.1B-Medical")
model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
model = PeftModel.from_pretrained(model, "therealcyberlord/TinyLlama-1.1B-Medical")

✨ Features

This is a smaller version of https://huggingface.co/therealcyberlord/llama2-qlora-finetuned-medical, which originally used Llama 2 7B. It is fine - tuned on <|user|> <|assistant|> instructions.

📚 Documentation

📦 Installation

The installation mainly involves setting up the required Python libraries. Ensure that you have peft and transformers installed. You can install them via pip:

pip install peft transformers

🔧 Technical Details

Training Data

The model uses two data sources:

• BI55/MedText: https://huggingface.co/datasets/BI55/MedText
• MedQuad - MedicalQnADataset: https://huggingface.co/datasets/keivalya/MedQuad-MedicalQnADataset

Training Procedure

The model was trained for 1000 steps on a shuffled combined dataset.

Framework versions

• PEFT 0.7.2.dev0