Llama Medx V3.2

🚀 Llama-medx_v3.2

Llama-medx_v3.2 is a model specifically fine - tuned for the medical domain, offering accurate medical knowledge and reasoning.

🚀 Quick Start

Please ensure transformers>=4.45.2

import torch
import transformers

model_id = "skumar9/Llama-medx_v3.2"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

prompt = "Question: A 20-year-old man comes to the physician because of worsening gait unsteadiness and bilateral hearing loss for 1 month. He has had intermittent tingling sensations on both cheeks over this time period. He has no history of serious medical illness and takes no medications. Audiometry shows bilateral sensorineural hearing loss. Genetic evaluation shows a mutation of a tumor suppressor gene on chromosome 22 that encodes merlin. This patient is at increased risk for which of the following conditions?\nA. Renal cell carcinoma\nB. Meningioma\nC. Astrocytoma\nD. Vascular malformations\nAnswer:\n"
gen_kwargs = {
    "return_full_text": False,
    "max_new_tokens": 100,
}
print(pipeline(prompt, **gen_kwargs))

✨ Features

Llama-medx_v3.2 is specifically fine - tuned for the medical domain.

🔧 Technical Details

Training Process

Built on the Meta - Llama - 3.1 - 8B architecture. Llama - medx_v3.2 has been finetuned using Supervised Fine - Tuning (SFT) and Odds Ratio Preference Optimization (ORPO). This ensures that the model aligns effectively with medical terminology and reasoning while maintaining learning efficiency. Hyperparameter tuning strategies have been carefully implemented to prevent catastrophic forgetting, ensuring consistent performance across various tasks. The benchmarked model was further finetuned for answering mcq questions.

Data Preparation

Publically available datasets, Enriched datasets for reinforcement feedback where the llama base model's knowledge and accuracy was not up to the mark.