Tab Transformer

🚀 tf-keras

This is a Keras implementation for structured data learning with TabTransformer, suitable for tabular classification tasks.

🚀 Quick Start

This repo contains the trained model of Structured data learning with TabTransformer. The full credit goes to: Khalid Salama.

Spaces Link:

✨ Features

• The trained model uses self-attention based Transformers structure followed by multiple feed forward layers for supervised and semi-supervised learning.
• The model can accept both numerical and categorical features as inputs.
• All categorical features are encoded into embedding vectors with the same embedding dimensions, then added point-wise and fed into a stack of Transformer blocks.
• After the final Transformer layer, the contextual embeddings of categorical features are concatenated with the input numerical features and fed into a final MLP block.
• A SoftMax function is applied at the end of the model.

📚 Documentation

Intended uses & limitations

This model can be used for both supervised and semi-supervised tasks on tabular data.

Training and evaluation data

• This model was trained using the United States Census Income Dataset provided by the UC Irvine Machine Learning Repository. The task of the dataset is to predict whether a person is likely to be making over USD 50,000 a year (binary classification).
• The dataset consists of 14 input features: 5 numerical features and 9 categorical features.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

Training Metrics

Model history needed

Model Plot