đ TabPFNMix Classifier
The TabPFNMix classifier is a tabular foundation model. It's pre - trained on purely synthetic datasets sampled from a mix of random classifiers, offering a new approach to tabular data classification.
đ Quick Start
To use the TabPFNMix classifier, you first need to install AutoGluon. You can do this by running the following command:
pip install autogluon
⨠Features
TabPFNMix is based on a 12 - layer encoder - decoder Transformer with 37 M parameters. It uses a pre - training strategy that incorporates in - context learning, similar to the strategies used by TabPFN and TabForestPFN.
đģ Usage Examples
Basic Usage
The following is a minimal example showing how to perform fine - tuning and inference using the TabPFNMix classifier:
import pandas as pd
from autogluon.tabular import TabularPredictor
if __name__ == '__main__':
train_data = pd.read_csv('https://autogluon.s3.amazonaws.com/datasets/Inc/train.csv')
subsample_size = 5000
if subsample_size is not None and subsample_size < len(train_data):
train_data = train_data.sample(n=subsample_size, random_state=0)
test_data = pd.read_csv('https://autogluon.s3.amazonaws.com/datasets/Inc/test.csv')
tabpfnmix_default = {
"model_path_classifier": "autogluon/tabpfn-mix-1.0-classifier",
"model_path_regressor": "autogluon/tabpfn-mix-1.0-regressor",
"n_ensembles": 1,
"max_epochs": 30,
}
hyperparameters = {
"TABPFNMIX": [
tabpfnmix_default,
],
}
label = "class"
predictor = TabularPredictor(label=label)
predictor = predictor.fit(
train_data=train_data,
hyperparameters=hyperparameters,
verbosity=3,
)
predictor.leaderboard(test_data, display=True)
đ Documentation
If you find TabPFNMix useful for your research, please consider citing the associated papers:
@article{erickson2020autogluon,
title={Autogluon-tabular: Robust and accurate automl for structured data},
author={Erickson, Nick and Mueller, Jonas and Shirkov, Alexander and Zhang, Hang and Larroy, Pedro and Li, Mu and Smola, Alexander},
journal={arXiv preprint arXiv:2003.06505},
year={2020}
}
@article{hollmann2022tabpfn,
title={Tabpfn: A transformer that solves small tabular classification problems in a second},
author={Hollmann, Noah and M{\"u}ller, Samuel and Eggensperger, Katharina and Hutter, Frank},
journal={arXiv preprint arXiv:2207.01848},
year={2022}
}
@article{breejen2024context,
title={Why In-Context Learning Transformers are Tabular Data Classifiers},
author={Breejen, Felix den and Bae, Sangmin and Cha, Stephen and Yun, Se-Young},
journal={arXiv preprint arXiv:2405.13396},
year={2024}
}
đ License
This project is licensed under the Apache - 2.0 License.