Dermatech-Qwen2-VL-2B Open-source Dermatological Diagnosis Model - Precise Analysis of Skin Images for Professional Diagnosis

Dermatech Qwen2 VL 2B

Developed by Rewatiramans

A dermatology-specific diagnostic model fine-tuned using LoRA technology based on Qwen2-VL-2B-Instruct, capable of analyzing skin condition images and providing professional diagnostic descriptions.

Image-to-Text

Transformers

#Dermatological Diagnosis #Multimodal Healthcare #LoRA Fine-tuning

Downloads 60

Release Time : 10/10/2024

Model Overview

This model is the core component of the DermaTech project, specializing in the field of dermatology. It was fine-tuned using a curated dataset of 1500 dermatological images paired with expert diagnostic texts, enabling it to simulate the professional judgment of dermatologists.

Model Features

Dermatology-Specific Fine-tuning

Trained on a targeted dataset of 1500 dermatology-related image-text pairs to enhance the accuracy of skin condition analysis.

Efficient LoRA Fine-tuning

Utilizes low-rank adaptation technology for efficient fine-tuning of the base model, retaining its multimodal capabilities while incorporating dermatological expertise.

Multimodal Understanding

Capable of processing both image and text inputs to understand dermatological images and generate professional diagnostic descriptions.

Model Capabilities

Dermatological Image Analysis

Medical Diagnostic Description Generation

Multimodal Input Processing

Professional Medical Information Provision

Use Cases

Medical Auxiliary Diagnosis

Preliminary Skin Condition Assessment

Upload images of skin lesions to obtain preliminary diagnostic descriptions and recommendations.

Provides professional judgments similar to those of dermatologists.

Medical Education Assistance

Used as a learning aid for medical students studying dermatology.

Helps understand the visual characteristics and diagnostic points of various skin conditions.

🚀 DermaTech Qwen 2 VL LoRA Model

This model is a fine - tuned version of Qwen 2 VL for specialized dermatological diagnosis, providing insightful diagnostic descriptions for skin conditions.

🚀 Quick Start

You can use this model with the transformers library. Refer to the DermaTech GitHub repository (GitHub Repo) for detailed usage instructions and integration with the full application.

✨ Features

This model is a fine - tuned version of [Qwen/Qwen2 - VL - 2B - Instruct](https://huggingface.co/Qwen/Qwen2 - VL - 2B - Instruct) using Low - Rank Adaptation (LoRA) for dermatological diagnosis.
It can analyze images of skin conditions and provide diagnostic descriptions, similar to a dermatologist's expertise.

📦 Installation

The README doesn't provide installation steps, so this section is skipped.

💻 Usage Examples

Basic Usage

from transformers import Qwen2VLForConditionalGeneration, Qwen2VLProcessor, GenerationConfig

# Load model and processor
model = Qwen2VLForConditionalGeneration.from_pretrained("Rewatiramans/Dermatech-Qwen2-VL-2B") 
processor = Qwen2VLProcessor.from_pretrained("Rewatiramans/Dermatech-Qwen2-VL-2B")

# Load and process image
... 

# Prepare inputs
...

# Generate text
generation_config = GenerationConfig(do_sample=True, temperature=0.7) # Adjust parameters as needed
outputs = model.generate(**inputs, generation_config=generation_config)

# Decode output
generated_text = processor.batch_decode(outputs, skip_special_tokens=True)[0]
print(generated_text)

Advanced Usage

The README doesn't provide advanced usage examples, so this part is skipped.

📚 Documentation

Model Description

The base model, Qwen 2 VL, is a powerful multimodal model pre - trained on a massive dataset of text and image pairs. This LoRA adaptation focuses its capabilities on dermatology by training on a curated dataset of 1500 image - text pairs, where images depict various skin conditions and the text provides corresponding expert diagnoses. This targeted training allows for more accurate and relevant responses in the context of skincare analysis.

Training Procedure

Property	Details
Base Model	Qwen/Qwen2 - VL - 2B - Instruct
Training Dataset	A custom dataset of 1500 image - text pairs related to dermatological conditions (Dermatech_vlm.json).
Fine - tuning Method	LoRA (Low - Rank Adaptation)
Optimizer	LoRA+ with a learning rate ratio of 16.00.
Learning Rate Schedule	Cosine Decay
Batch Size	2 per device, effective batch size of 8 with gradient accumulation of 4.
Training Steps	618
Training Epochs	4 (though training completed early due to cosine decay schedule)
Hardware	Trained on a single GPU (cuda:0).
Training Loss	0.5477
Training Time	Approximately 9 minutes.

The model was resumed from checkpoint - 205 and saved at checkpoint - 618.

Intended Uses & Limitations

This model is intended for research and prototyping purposes as part of the DermaTech project. It's designed to assist in preliminary skin condition assessment and provide information, but it should not be considered a substitute for professional medical advice. Always consult with a qualified dermatologist for accurate diagnoses and treatment plans.

The model's performance might be limited by the size and diversity of the training data. It may not accurately diagnose all skin conditions, and its output should be interpreted with caution.

Evaluation Results

Formal evaluation is still pending, but initial results show promising performance on the training data. Further testing and benchmarking will be conducted to assess generalization and accuracy on unseen data.

Citation

If you use this model in your research, please cite the DermaTech project and the original Qwen 2 VL model.

Contact

For any inquiries regarding the model or the DermaTech project, please contact [your_email_address].

🔧 Technical Details

The README doesn't provide in - depth technical details, so this section is skipped.

📄 License

The README doesn't provide license information, so this section is skipped.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご