Bio Medical Llama 3 8B

🚀 Bio-Medical-Llama-3-8B

A specialized large language model designed for biomedical applications, fine-tuned from Meta-Llama-3-8B-Instruct with a custom high - quality dataset.

🚀 Quick Start

To use the Bio - Medical - Llama - 3 - 8B model, you can follow the code example below:

import transformers
import torch

model_id = "ContactDoctor/Bio-Medical-Llama-3-8B"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are an expert trained on healthcare and biomedical domain!"},
    {"role": "user", "content": "I'm a 35-year-old male and for the past few months, I've been experiencing fatigue, increased sensitivity to cold, and dry, itchy skin. What is the diagnosis here?"},
]

prompt = pipeline.tokenizer.apply_chat_template(
        messages, 
        tokenize=False, 
        add_generation_prompt=True
)

terminators = [
    pipeline.tokenizer.eos_token_id,
    pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = pipeline(
    prompt,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])

✨ Features

• Specialized for Biomedicine: The Bio - Medical - Llama - 3 - 8B model is fine - tuned for biomedical applications, trained to understand and generate text related to various biomedical fields.
• High - Quality Data: It is trained on a custom high - quality biomedical dataset with over 500,000 diverse entries, including a mix of synthetic and manually curated data.
• Good Performance: Outperforms many leading LLMs in biomedical NLP tasks, as evaluated by the Eleuther AI Language Model Evaluation Harness framework.

📚 Documentation

Model details

Model description

Bio - Medical - Llama - 3 - 8B model is a specialized large language model designed for biomedical applications. It is finetuned from the meta - llama/Meta - Llama - 3 - 8B - Instruct model using a custom dataset containing over 500,000 diverse entries. These entries include a mix of synthetic and manually curated data, ensuring high quality and broad coverage of biomedical topics.

The model is trained to understand and generate text related to various biomedical fields, making it a valuable tool for researchers, clinicians, and other professionals in the biomedical domain.

Evaluation Metrics

Bio - Medical - Llama - 3 - 8B model outperforms many of the leading LLMs and find below its metrics evaluated using the Eleuther AI Language Model Evaluation Harness framework against the tasks medmcqa, medqa_4options, mmlu_anatomy, mmlu_clinical_knowledge, mmlu_college_biology, mmlu_college_medicine, mmlu_medical_genetics, mmlu_professional_medicine and pubmedqa.


Intended uses & limitations

Bio - Medical - Llama - 3 - 8B model is intended for a wide range of applications within the biomedical field, including:

1. Research Support: Assisting researchers in literature review and data extraction from biomedical texts.
2. Clinical Decision Support: Providing information to support clinical decision - making processes.
3. Educational Tool: Serving as a resource for medical students and professionals seeking to expand their knowledge base.

Limitations and Ethical Considerations

⚠️ Important Note

• Biases: The model may inherit biases present in the training data. Efforts have been made to curate a balanced dataset, but some biases may persist.
• Accuracy: The model's responses are based on patterns in the data it has seen and may not always be accurate or up - to - date. Users should verify critical information from reliable sources.
• Ethical Use: The model should be used responsibly, particularly in clinical settings where the stakes are high. It should complement, not replace, professional judgment and expertise.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 12
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.03
• training_steps: 2000
• mixed_precision_training: Native AMP

Framework versions

• PEFT 0.11.0
• Transformers 4.40.2
• Pytorch 2.1.2
• Datasets 2.19.1
• Tokenizers 0.19.1

📄 License

This model is licensed under the Bio - Medical - Llama - 3 - 8B (Non - Commercial Use Only). Please review the terms and conditions before using the model.

Contact Information

For further information, inquiries, or issues related to Biomed - LLM, please contact:

• Email: info@contactdoctor.in
• Website: https://www.contactdoctor.in

Citation

If you use Bio - Medical LLM in your research or applications, please cite it as follows:

@misc{ContactDoctor_Bio-Medical-Llama-3-8B,
  author = ContactDoctor,
  title = {Bio - Medical: A High - Performance Biomedical Language Model},
  year = {2024},
  howpublished = {https://huggingface.co/ContactDoctor/Bio - Medical - Llama - 3 - 8B},
}