đ CryptoBERT
CryptoBERT is a fine - tuned model based on ProsusAI/finbert. It is designed for sentiment analysis of cryptocurrency - related text, offering valuable insights into the crypto market sentiment.
đ Quick Start
You can use the following code to quickly start using the CryptoBERT model for sentiment analysis:
from transformers import BertTokenizer, BertForSequenceClassification
from transformers import pipeline
tokenizer = BertTokenizer.from_pretrained("kk08/CryptoBERT")
model = BertForSequenceClassification.from_pretrained("kk08/CryptoBERT")
classifier = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)
text = "Bitcoin (BTC) touches $29k, Ethereum (ETH) Set To Explode, RenQ Finance (RENQ) Crosses Massive Milestone"
result = classifier(text)
print(result)
The output will be like:
[{'label': 'LABEL_1', 'score': 0.9678454399108887}]
⨠Features
- Fine - tuned on Crypto Data: CryptoBERT fine - tunes the pre - trained ProsusAI/finbert on a custom cryptocurrency market sentiment dataset, enabling it to better understand and analyze crypto - related text.
- Sentiment Prediction: It can predict whether the given text related to the crypto market is positive (LABEL_1) or negative (LABEL_0).
đģ Usage Examples
Basic Usage
from transformers import BertTokenizer, BertForSequenceClassification
from transformers import pipeline
tokenizer = BertTokenizer.from_pretrained("kk08/CryptoBERT")
model = BertForSequenceClassification.from_pretrained("kk08/CryptoBERT")
classifier = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)
text = "Bitcoin (BTC) touches $29k, Ethereum (ETH) Set To Explode, RenQ Finance (RENQ) Crosses Massive Milestone"
result = classifier(text)
print(result)
Advanced Usage
from transformers import BertTokenizer, BertForSequenceClassification
from transformers import pipeline
tokenizer = BertTokenizer.from_pretrained("kk08/CryptoBERT")
model = BertForSequenceClassification.from_pretrained("kk08/CryptoBERT")
classifier = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)
texts = ["Bitcoin price is rising", "Ethereum has some technical issues"]
results = classifier(texts)
for result in results:
print(result)
đ Documentation
Model description
This model fine - tunes the ProsusAI/finbert, which is a pre - trained NLP model to analyze the sentiment of financial text. CryptoBERT fine - tunes this model by training it as a downstream task on custom crypto sentiment data to predict whether the given text related to the crypto market is positive (LABEL_1) or negative (LABEL_0).
Intended uses & limitations
The model can perform well on crypto - related data. The main limitation is that the fine - tuning was done using only a small corpus of data.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e - 05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9, 0.999) and epsilon = 1e - 08
- lr_scheduler_type: linear
- num_epochs: 10
Training results
Training Loss |
Epoch |
Step |
Validation Loss |
0.4077 |
1.0 |
27 |
0.4257 |
0.2048 |
2.0 |
54 |
0.2479 |
0.0725 |
3.0 |
81 |
0.3068 |
0.0028 |
4.0 |
108 |
0.4120 |
0.0014 |
5.0 |
135 |
0.3566 |
0.0007 |
6.0 |
162 |
0.3495 |
0.0006 |
7.0 |
189 |
0.3645 |
0.0005 |
8.0 |
216 |
0.3754 |
0.0004 |
9.0 |
243 |
0.3804 |
0.0004 |
10.0 |
270 |
0.3823 |
Framework versions
- Transformers 4.28.0
- Pytorch 2.0.0+cu118
- Datasets 2.11.0
- Tokenizers 0.13.3
đ§ Technical Details
- Base Model: ProsusAI/finbert, a pre - trained NLP model for financial text sentiment analysis.
- Fine - tuning: The model is fine - tuned on a custom cryptocurrency market sentiment dataset to predict the sentiment of crypto - related text.
- Hyperparameters: Specific hyperparameters are used during training, including learning rate, batch size, optimizer, etc., to ensure the model's performance.
đ License
The license information is not provided in the original document. If you have the license details, please add them here.
đĻ Installation
The installation steps are not provided in the original document. If you want to install the necessary libraries, you can use the following commands:
pip install transformers datasets torch