Batterybert Cased Squad V1

🚀 BatteryBERT-cased for QA

BatteryBERT-cased is a language model designed for extractive question answering tasks, trained on SQuAD v1 data.

🚀 Quick Start

BatteryBERT-cased is an English language model tailored for the downstream task of extractive question answering. It was trained on SQuAD v1 data and evaluated on both the SQuAD v1.0 dev set and a battery device dataset.

✨ Features

• Language Model: batterybert-cased
• Language: English
• Downstream-task: Extractive QA
• Training data: SQuAD v1
• Eval data: SQuAD v1
• Code: See example
• Infrastructure: 8x DGX A100

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline

model_name = "batterydata/batterybert-cased-squad-v1"
# a) Get predictions
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
    'question': 'What is the electrolyte?',
    'context': 'The typical non-aqueous electrolyte for commercial Li-ion cells is a solution of LiPF6 in linear and cyclic carbonates.'
}
res = nlp(QA_input)
# b) Load model & tokenizer
model = AutoModelForQuestionAnswering.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

📚 Documentation

Hyperparameters

batch_size = 16
n_epochs = 4
base_LM_model = "batterybert-cased"
max_seq_len = 386
learning_rate = 2e-5
doc_stride=128
max_query_length=64

Performance

Evaluated on the SQuAD v1.0 dev set.

"exact": 81.54,
"f1": 89.16,

Evaluated on the battery device dataset.

"precision": 70.74,
"recall": 84.19,

🔧 Technical Details

The model was trained using an infrastructure of 8x DGX A100 with specific hyperparameters set for optimal performance on the extractive question answering task.

📄 License

This project is licensed under the Apache-2.0 license.

📝 Authors

• Shu Huang: sh2009 [at] cam.ac.uk
• Jacqueline Cole: jmc61 [at] cam.ac.uk

📚 Citation

BatteryBERT: A Pre-trained Language Model for Battery Database Enhancement