SDO VT1

🚀 NASA Solar Dynamics Observatory Vision Transformer v.1 (SDO_VT1)

A Vision Transformer model fine - tuned on Solar Dynamics Observatory (SDO) data for active region classification.

🚀 Quick Start

This Vision Transformer model has been fine - tuned on Solar Dynamics Observatory (SDO) data. The images used are available at Solar Dynamics Observatory Gallery. This model aims to showcase the ease of use of the HuggingFace platform, its integration with popular deep - learning frameworks like PyTorch, TensorFlow, or JAX, performance monitoring with Weights and Biases, and the ability to easily use pre - trained large - scale Transformer models for targeted fine - tuning.

The data used was provided courtesy of NASA/SDO and the AIA, EVE, and HMI science teams. The authors gratefully acknowledge the entire NASA Solar Dynamics Observatory Mission Team.

Here is a quick snippet to use this model on Google Colab (comment the pip install for local use if you have transformers already installed):

!pip install transformers --quiet
from transformers import AutoFeatureExtractor, AutoModelForImageClassification
from PIL import Image
import requests

url = 'https://sdo.gsfc.nasa.gov/assets/gallery/preview/211_coronalhole.jpg'
image = Image.open(requests.get(url, stream=True).raw)

feature_extractor = AutoFeatureExtractor.from_pretrained("kenobi/SDO_VT1")
model = AutoModelForImageClassification.from_pretrained("kenobi/SDO_VT1")
inputs = feature_extractor(images=image, return_tensors="pt")

outputs = model(**inputs)
logits = outputs.logits
# model predicts one of the three fine - tuned classes (NASA_SDO_Coronal_Hole, NASA_SDO_Coronal_Loop or NASA_SDO_Solar_Flare) 
predicted_class_idx = logits.argmax(-1).item()
print("Predicted class:", model.config.id2label[predicted_class_idx])

✨ Features

• Fine - tuned on SDO data: Specifically adapted for Solar Dynamics Observatory data in an active region classification task.
• HuggingFace integration: Demonstrates the ease of use of the HuggingFace platform and its integration with popular deep - learning frameworks.
• Performance monitoring: Allows for performance monitoring with Weights and Biases.
• Pre - trained model utilization: Enables effortless use of pre - trained large - scale Transformer models for targeted fine - tuning.

📦 Installation

If you want to use this model, you need to install the transformers library. You can use the following command:

!pip install transformers --quiet

💻 Usage Examples

Basic Usage

!pip install transformers --quiet
from transformers import AutoFeatureExtractor, AutoModelForImageClassification
from PIL import Image
import requests

url = 'https://sdo.gsfc.nasa.gov/assets/gallery/preview/211_coronalhole.jpg'
image = Image.open(requests.get(url, stream=True).raw)

feature_extractor = AutoFeatureExtractor.from_pretrained("kenobi/SDO_VT1")
model = AutoModelForImageClassification.from_pretrained("kenobi/SDO_VT1")
inputs = feature_extractor(images=image, return_tensors="pt")

outputs = model(**inputs)
logits = outputs.logits
# model predicts one of the three fine - tuned classes (NASA_SDO_Coronal_Hole, NASA_SDO_Coronal_Loop or NASA_SDO_Solar_Flare) 
predicted_class_idx = logits.argmax(-1).item()
print("Predicted class:", model.config.id2label[predicted_class_idx])

Advanced Usage

There is no advanced usage example provided in the original document.

📚 Documentation

Example Images

Use one of the images below for the inference API field on the upper right.

Additional images for testing can be found at: Solar Dynamics Observatory Gallery. You can use the following tags to further select images for testing: "coronal holes", "loops" or "flares". You can also choose "active regions" to get a general pool for testing.

NASA_SDO_Coronal_Hole

NASA_SDO_Coronal_Loop

NASA_SDO_Solar_Flare

Training data

The ViT model was pretrained on a dataset consisting of 14 million images and 21k classes (ImageNet - 21k). More information on the base model used can be found here: (https://huggingface.co/google/vit - base - patch16 - 224 - in21k)

BibTeX & References

A publication on this work is currently in preparation. In the meantime, please refer to this model by using the following citation:

@misc{sdovt2022,
  author = {Frank Soboczenski and Paul J Wright},
  title = {SDOVT: A Vision Transformer Model for Solar Dynamics Observatory (SDO) Data},
  url = {https://huggingface.co/kenobi/SDO_VT1/},
  version = {1.0},
  year = {2022},
}

For the base ViT model used please refer to:

@misc{wu2020visual,
      title={Visual Transformers: Token - based Image Representation and Processing for Computer Vision}, 
      author={Bichen Wu and Chenfeng Xu and Xiaoliang Dai and Alvin Wan and Peizhao Zhang and Zhicheng Yan and Masayoshi Tomizuka and Joseph Gonzalez and Kurt Keutzer and Peter Vajda},
      year={2020},
      eprint={2006.03677},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

For referring to Imagenet:

@inproceedings{deng2009imagenet,
  title={Imagenet: A large - scale hierarchical image database},
  author={Deng, Jia and Dong, Wei and Socher, Richard and Li, Li - Jia and Li, Kai and Fei - Fei, Li},
  booktitle={2009 IEEE conference on computer vision and pattern recognition},
  pages={248--255},
  year={2009},
  organization={Ieee}
}

⚠️ Important Note

The data used was provided courtesy of NASA/SDO and the AIA, EVE, and HMI science teams. The authors gratefully acknowledge the entire NASA Solar Dynamics Observatory Mission Team.

💡 Usage Tip

You can use the tags "coronal holes", "loops", "flares" or "active regions" to select images from Solar Dynamics Observatory Gallery for testing.