Bigbird Roberta Base Finetuned App

🚀 Mobile App Classification

This project uses the BigBird model to classify mobile app descriptions into 6 Play Store categories, offering a practical solution for app categorization.

🚀 Quick Start

The model is based on BigBird, a sparse - attention based transformer that extends Transformer - based models like BERT to handle much longer sequences. It can process input sequences up to 4,096 tokens in length. The [google/bigbird - roberta - base](https://huggingface.co/google/bigbird - roberta - base) model has been fine - tuned to classify mobile app descriptions into one of 6 play store categories. It was trained on 9000 samples of English App Descriptions and their associated categories from Google Play.

✨ Features

• Long - sequence handling: Can handle input sequences up to 4,096 tokens.
• App categorization: Classify mobile app descriptions into 6 Play Store categories.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

tokenizer = AutoTokenizer.from_pretrained("nsi319/bigbird-roberta-base-finetuned-app")  
model = AutoModelForSequenceClassification.from_pretrained("nsi319/bigbird-roberta-base-finetuned-app")

classifier = pipeline('sentiment-analysis', model=model, tokenizer=tokenizer)

classifier("From scores to signings, the ESPN App is here to keep you updated. Never miss another sporting moment with up-to-the-minute scores, latest news & a range of video content. Sign in and personalise the app to receive alerts for your teams and leagues. Wherever, whenever; the ESPN app keeps you connected.")

'''Output'''
[{'label': 'Sports', 'score': 0.9983325600624084}]

📚 Documentation

Fine - tuning

The model was fine - tuned for 5 epochs with a batch size of 16, a learning rate of 2e - 05, and a maximum sequence length of 1024. Since this was a classification task, the model was trained with a cross - entropy loss function. The best evaluation f1 score achieved by the model was 0.8964259037209702, found after 4 epochs. The accuracy of the model on the test set was 0.8966.

Limitations

Training data consists of apps from 6 play store categories namely Education, Entertainment, Productivity, Sports, News & Magazines and Photography.

📄 License

This project is licensed under the MIT license.