CryptoBERT Open-source Model - Free Analysis of Sentiment and Language in Cryptocurrency Social Media Posts

Cryptobert

Developed by ElKulako

CryptoBERT is a pre-trained natural language processing model specifically designed for analyzing sentiment and language in cryptocurrency-related social media posts.

Text Classification

Transformers

EnglishOpen Source License:MIT #Cryptocurrency Sentiment Analysis #Social Media Text Processing #Fine-tuned BERT Model

Downloads 276.93k

Release Time : 6/20/2022

Model Overview

This model is based on the BERT architecture and has been optimized for the cryptocurrency domain, effectively identifying bullish, bearish, and neutral sentiments in social media discussions about cryptocurrencies.

Model Features

Cryptocurrency-Specific

Trained specifically on cryptocurrency-related social media content, enabling better understanding of domain-specific terminology and expressions.

Multi-platform Training Data

Trained on 3.2 million cryptocurrency-related posts from multiple platforms including StockTwits, Telegram, Reddit, and Twitter.

Three-Class Sentiment Analysis

Accurately identifies 'bearish', 'neutral', and 'bullish' sentiments related to cryptocurrencies.

Model Capabilities

Cryptocurrency social media text analysis

Sentiment classification

Natural language understanding

Use Cases

Market Sentiment Analysis

Cryptocurrency Market Sentiment Monitoring

Real-time analysis of sentiment trends regarding specific cryptocurrencies on social media.

Accurately identifies bullish/bearish sentiments, with example accuracy rates: 87.3% for bullish identification, 98.9% for bearish identification.

Investment Decision Support

Provides cryptocurrency investors with market sentiment reference data.

Academic Research

Cryptocurrency Community Language Research

Analyzes unique linguistic patterns and expressions in cryptocurrency communities.

🚀 CryptoBERT

CryptoBERT is a pre - trained NLP model designed to analyze the language and sentiments of cryptocurrency - related social media posts and messages, offering valuable insights into the cryptocurrency market.

🚀 Quick Start

For academic reference, cite the following paper: https://ieeexplore.ieee.org/document/10223689

CryptoBERT is a pre - trained NLP model to analyse the language and sentiments of cryptocurrency - related social media posts and messages. It was built by further training the [vinai's bertweet - base](https://huggingface.co/vinai/bertweet - base) language model on the cryptocurrency domain, using a corpus of over 3.2M unique cryptocurrency - related social media posts. (A research paper with more details will follow soon.)

✨ Features

Classification Training

The model was trained on the following labels: "Bearish" : 0, "Neutral": 1, "Bullish": 2

CryptoBERT's sentiment classification head was fine - tuned on a balanced dataset of 2M labelled StockTwits posts, sampled from [ElKulako/stocktwits - crypto](https://huggingface.co/datasets/ElKulako/stocktwits - crypto).

CryptoBERT was trained with a max sequence length of 128. Technically, it can handle sequences of up to 514 tokens, however, going beyond 128 is not recommended.

💻 Usage Examples

Basic Usage

from transformers import TextClassificationPipeline, AutoModelForSequenceClassification, AutoTokenizer
model_name = "ElKulako/cryptobert"
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels = 3)
pipe = TextClassificationPipeline(model=model, tokenizer=tokenizer, max_length=64, truncation=True, padding = 'max_length')
# post_1 & post_3 = bullish, post_2 = bearish
post_1 = " see y'all tomorrow and can't wait to see ada in the morning, i wonder what price it is going to be at. 😎🐂🤠💯😴, bitcoin is looking good go for it and flash by that 45k. "
post_2 = "  alright racers, it’s a race to the bottom! good luck today and remember there are no losers (minus those who invested in currency nobody really uses) take your marks... are you ready? go!!" 
post_3 = " i'm never selling. the whole market can bottom out. i'll continue to hold this dumpster fire until the day i die if i need to." 
df_posts = [post_1, post_2, post_3]
preds = pipe(df_posts)
print(preds)

[{'label': 'Bullish', 'score': 0.8734585642814636}, {'label': 'Bearish', 'score': 0.9889495372772217}, {'label': 'Bullish', 'score': 0.6595883965492249}]

🔧 Technical Details

Training Corpus

CryptoBERT was trained on 3.2M social media posts regarding various cryptocurrencies. Only non - duplicate posts of length above 4 words were considered. The following communities were used as sources for our corpora:

(1) StockTwits - 1.875M posts about the top 100 cryptos by trading volume. Posts were collected from the 1st of November 2021 to the 16th of June 2022. [ElKulako/stocktwits - crypto](https://huggingface.co/datasets/ElKulako/stocktwits - crypto)

(2) Telegram - 664K posts from top 5 telegram groups: Binance, Bittrex, huobi global, Kucoin, OKEx. Data from 16.11.2020 to 30.01.2021. Courtesy of [Anton](https://www.kaggle.com/datasets/aagghh/crypto - telegram - groups).

(3) Reddit - 172K comments from various crypto investing threads, collected from May 2021 to May 2022

(4) Twitter - 496K posts with hashtags XBT, Bitcoin or BTC. Collected for May 2018. Courtesy of [Paul](https://www.kaggle.com/datasets/paul92s/bitcoin - tweets - 14m).

📄 License

This project is licensed under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご