Sportsbert

🚀 SportsBERT

SportsBERT is a BERT-based transformer model specifically trained on sports articles, aiming to enhance performance in sports-related natural language processing tasks.

🚀 Quick Start

Pretraining large natural language processing models like BERT and RoBERTa has become the state-of-the-art approach for natural language understanding and processing tasks. However, these models are typically trained on general corpora from the web or repositories such as Quora and Wikipedia, which cover articles from all domains. Training domain-specific language models has been proven to outperform general pre-trained models in certain fields, like medicine. Based on this, we trained a sports-specific BERT-based transformer model, SportsBERT.

✨ Features

• Domain-Specific Training: SportsBERT is trained from scratch using a corpus of sports news articles collected over the past four years. These articles cover a wide range of sports, including Football, Basketball, Hockey, Cricket, Soccer, Baseball, Olympics, Tennis, Golf, MMA, etc.
• Custom Tokenizer: A tokenizer was trained from scratch to include more sports-related tokens in the vocabulary, enabling better handling of sports-specific language.
• BERT Base Uncased Architecture: The model adopts the BERT base uncased architecture and was trained on four V100 GPUs.
• MLM-Based Model: As a Masked Language Model (MLM), its primary task is to predict missing masked tokens. For example, given the input "Anthony Davis is a [MASK]", it can output tokens like "legend", "superstar", "rookie", "star", "king" in descending order of confidence.
• Fine-Tuning Capability: The model can be fine-tuned for other natural language processing tasks, such as classification and entity extraction.

💻 Usage Examples

Basic Usage

# Assume there is a library to load the SportsBERT model
from sportsbert_library import SportsBERTModel

model = SportsBERTModel.from_pretrained('sportsbert_model_path')
input_text = "Anthony Davis is a [MASK]"
output = model.predict_masked_tokens(input_text)
print(output)

Advanced Usage

# Fine-tuning the model for a classification task
from sportsbert_library import SportsBERTModel, SportsBERTClassifier
from datasets import load_dataset

# Load the dataset
dataset = load_dataset('sports_classification_dataset')

# Initialize the model and classifier
model = SportsBERTModel.from_pretrained('sportsbert_model_path')
classifier = SportsBERTClassifier(model)

# Fine-tune the classifier
classifier.train(dataset['train'], epochs=5)

# Evaluate the classifier
accuracy = classifier.evaluate(dataset['test'])
print(f"Accuracy: {accuracy}")

📚 Documentation

Model Information

Authors

• Prithvishankar Srinivasan (prsrini@microsoft.com)

Language

English

Pipeline Tag

fill-mask