Biobert Nli

🚀 BioBERT-NLI

This model fine-tunes BioBERT [1] on the SNLI and MultiNLI datasets using the sentence-transformers library to generate universal sentence embeddings [2].

🚀 Quick Start

This is the model BioBERT [1] fine - tuned on the SNLI and the MultiNLI datasets using the sentence-transformers library to produce universal sentence embeddings [2].

✨ Features

• The model uses the original BERT wordpiece vocabulary.
• It was trained using the average pooling strategy and a softmax loss.

🔧 Technical Details

Base Model

The base model is monologg/biobert_v1.1_pubmed from HuggingFace's AutoModel.

Training Time

It took approximately 6 hours to train on the NVIDIA Tesla P100 GPU provided in Kaggle Notebooks.

Parameters

Performances

The performance was evaluated on the test portion of the STS dataset using Spearman rank correlation and compared to the performances of a general BERT base model obtained with the same procedure to verify their similarity.

💻 Usage Examples

An example usage for similarity - based scientific paper retrieval is provided in the Covid Papers Browser repository.

📄 License

No license information is provided in the original document.

📚 References

[1] J. Lee et al, BioBERT: a pre - trained biomedical language representation model for biomedical text mining

[2] A. Conneau et al., Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

[3] N. Reimers et I. Gurevych, Sentence - BERT: Sentence Embeddings using Siamese BERT - Networks