Bio Medical Llama 3 2 1B CoT 012025

🚀 Bio-Medical-Llama-3-2-1B-CoT-012025

This is a fine - tuned language model optimized for the Healthcare & Lifesciences domain, enhancing reasoning capabilities with chain - of - thought instruction samples.

🚀 Quick Start

The Bio - Medical - Llama - 3 - 2 - 1B - CoT - 012025 model is a powerful tool for healthcare and biomedical applications. Here's a simple way to get started:

import transformers
import torch

model_id = "ContactDoctor/Bio-Medical-Llama-3-2-1B-CoT-012025"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are an expert trained on healthcare and biomedical domain!"},
    {"role": "user", "content": "What are the differential diagnoses for a patient presenting with shortness of breath and chest pain?"},
]

prompt = pipeline.tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

terminators = [
    pipeline.tokenizer.eos_token_id,
    pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = pipeline(
    prompt,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])

✨ Features

• Domain - Specific Content Generation: Capable of generating high - quality content tailored to the healthcare and biomedical fields.
• Enhanced Reasoning: Strengthened reasoning capabilities through 25,000 chain - of - thought (CoT) instruction samples in the training data.
• Versatile Use Cases: Supports research, clinical decision - making, and education in the biomedical domain.

📦 Installation

The installation process mainly involves setting up the necessary Python libraries. You can use the following commands to install the required libraries:

pip install transformers torch datasets tokenizers peft

📚 Documentation

Model details

Model description

The Bio - Medical - Llama - 3 - 2 - 1B - CoT - 012025 model is a lightweight yet powerful language model tailored for generating domain - specific content, answering complex questions requiring step - by - step reasoning, and supporting researchers, clinicians, and students in their biomedical endeavors. It is fine - tuned to provide interpretability and improved logical coherence through its enhanced CoT capabilities.

Evaluation Metrics

This model has been evaluated using the Eleuther AI Language Model Evaluation Harness framework on tasks such as medmcqa, medqa_4options, mmlu_anatomy, etc. Results show consistent performance improvements over general - purpose models of similar size, particularly in tasks requiring reasoning.

Intended uses & limitations

Intended Uses:

1. Research Support: Assisting researchers with reasoning and data extraction from biomedical texts.
2. Clinical Decision Support: Offering logical and evidence - based information to aid decision - making.
3. Educational Tool: Serving as a learning resource for understanding complex biomedical concepts.

Limitations and Ethical Considerations:

• Biases: The model may reflect biases from the training data, despite efforts to mitigate them.
• Accuracy: Responses should be cross - verified with reliable sources in critical scenarios.
• Ethical Use: The model should augment professional expertise and not replace it, especially in high - stakes applications.

Training hyperparameters

The following hyperparameters were used during training:

• Learning Rate: 0.0002
• Train Batch Size: 8
• Eval Batch Size: 4
• Seed: 42
• Gradient Accumulation Steps: 8
• Total Train Batch Size: 32
• Optimizer: Adam with betas=(0.9, 0.999) and epsilon = 1e - 08
• LR Scheduler Type: Cosine
• LR Scheduler Warmup Ratio: 0.03
• Training Steps: 2000
• Mixed Precision Training: Native AMP

Framework versions

• PEFT: 0.11.0
• Transformers: 4.40.2
• Pytorch: 2.1.2
• Datasets: 2.19.1
• Tokenizers: 0.19.1

📄 License

This model is licensed under the Bio - Medical - Llama - 3 - 2 - 1B - CoT - 012025 (Non - Commercial Use Only). Please review the terms and conditions before using the model.

Contact Information

For further information, inquiries, or issues related to Bio - Medical - Llama - 3 - 2 - 1B - CoT - 012025, please contact: Email: info@contactdoctor.in Website: https://www.contactdoctor.in

Citation

If you use Bio - Medical - Llama - 3 - 2 - 1B - CoT - 012025 in your research or applications, please cite it as follows:

@misc{ContactDoctor_Bio-Medical-Llama-3.2-1B-CoT-012025,
  author = {ContactDoctor},
  title = {Bio-Medical-Llama-3-2-1B-CoT-012025: A Reasoning-Enhanced Biomedical Language Model},
  year = {2025},
  howpublished = {https://huggingface.co/ContactDoctor/Bio-Medical-Llama-3-2-1B-CoT-012025},
}