Pmc Vit L 14

🚀 Open Clip Model for Text-to-Image

This model is designed for text-to-image tasks in the fields of biology, chemistry, and medical science, leveraging fine-tuning techniques on pre - trained models.

🚀 Quick Start

This model is fine - tuned from OpenAI's ViT - L - 14. It uses the PMC_OA_beta and roco datasets, with the help of the open_clip tool (https://github.com/mlfoundations/open_clip).

✨ Features

• Domain - Specific: Specialized for biology, chemistry, and medical applications.
• Fine - Tuned: Based on a well - known pre - trained model (OpenAI's ViT - L - 14).

📦 Installation

Not provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

The following is the code for training the model:

python -m training.main \
    --save-frequency 2 \
    --zeroshot-frequency 1 \
    --report-to tensorboard \
    --train-data="/home/data1/ryanyip/huggingface-models/pmc_oa_beta/train.csv"  \
    --val-data="/home/data1/ryanyip/huggingface-models/pmc_oa_beta/sample_valid.csv"  \
    --csv-separator "," \
    --csv-img-key image \
    --csv-caption-key caption \
    --warmup 10000 \
    --batch-size=128 \
    --lr=1e-5 \
    --wd=0.2 \
    --epochs=30 \
    --workers=8 \
    --model "ViT-L-14" \
    --name "pmc_vit_l_14" \
    --pretrained  "ViT-L-14_state_dict.pt" \
    --save-most-recent

Note: ViT - L - 14_state_dict.pt is the pretrained weight from openai/ViT - L - 14

📚 Documentation

Not provided in the original document, so this section is skipped.

🔧 Technical Details

Not provided in the original document, so this section is skipped.

📄 License

Not provided in the original document, so this section is skipped.

📋 Model Information