Vit Base Patch8 224.kaiko Ai Towards Large Pathology Fms

🚀 Model card for vit_base_patch8_224.kaiko_ai_towards_large_pathology_fms

This model card provides details about the vit_base_patch8_224.kaiko_ai_towards_large_pathology_fms model, including its features, usage, and performance metrics.

🚀 Quick Start

The vit_base_patch8_224.kaiko_ai_towards_large_pathology_fms model can be used for image classification tasks. You can load the model using the timm library and perform image embedding extraction as shown in the usage example below.

✨ Features

• Feature Backbone: Serves as a feature backbone for various image - related tasks.
• Multiple Datasets Support: Demonstrates performance on multiple image - classification datasets such as BACH, CRC - NCT - HE, MHIST, PCam, TP53, and CoNSeP.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from torchvision.transforms import v2
from PIL import Image
import requests
import torch
import timm
import io

# get example histology image
url = "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQc7_xZpGOfQT7sxKwf2w5lL4GAq6IX_CbTzP1NGeenzA&s"
image = Image.open(io.BytesIO(requests.get(url).content))

# load model from the hub
model = timm.create_model(
  model_name="hf-hub:1aurent/vit_base_patch8_224.kaiko_ai_towards_large_pathology_fms",
  dynamic_img_size=True,
  pretrained=True,
).eval()

# get image transform
preprocessing = v2.Compose(
  [
    v2.ToImage(),
    v2.Resize(size=224),
    v2.CenterCrop(size=224),
    v2.ToDtype(torch.float32, scale=True),
    v2.Normalize(
      mean=(0.5, 0.5, 0.5),
      std=(0.5, 0.5, 0.5),
    ),
  ]
)

data = preprocessing(image).unsqueeze(0) # input is a (batch_size, num_channels, img_size, img_size) shaped tensor
output = model(data)  # output is a (batch_size, num_features) shaped tensor

📚 Documentation

Model Details

Model Performance

The model has been evaluated on several image - classification datasets. The following table shows the accuracy metrics:

📄 License

The model is released under the kaiko-non-commercial license.

📚 Citation

@misc{ai2024largescale,
  title         = {Towards Large-Scale Training of Pathology Foundation Models}, 
  author        = {kaiko.ai and Nanne Aben and Edwin D. de Jong and Ioannis Gatopoulos and Nicolas Känzig and Mikhail Karasikov and Axel Lagré and Roman Moser and Joost van Doorn and Fei Tang},
  year          = {2024},
  eprint        = {2404.15217},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}