Distilroberta Base Squad V2

🚀 distilroberta-base-squad_v2

This model, fine - tuned on the squad_v2 dataset, is a powerful tool for extractive question - answering tasks. It can handle mismatched question - context pairs and is compatible with PyTorch, Tensorflow and ONNX frameworks.

✨ Features

• Fine - tuned on the SQuAD2.0 dataset for extractive question answering.
• Compatible with PyTorch, Tensorflow and ONNX frameworks.
• Capable of handling mismatched question - context pairs.

📦 Installation

The original README does not provide installation steps, so this section is skipped.

💻 Usage Examples

Basic Usage

>>> from transformers import AutoModelForQuestionAnswering, AutoTokenizer, QuestionAnsweringPipeline
>>> model = AutoModelForQuestionAnswering.from_pretrained("squirro/distilroberta-base-squad_v2")
>>> tokenizer = AutoTokenizer.from_pretrained("squirro/distilroberta-base-squad_v2")
>>> qa_model = QuestionAnsweringPipeline(model, tokenizer)
>>> qa_model(
>>>    question="What's your name?",
>>>    context="My name is Clara and I live in Berkeley.",
>>>    handle_impossible_answer=True  # important!
>>> )
{'score': 0.9498472809791565, 'start': 11, 'end': 16, 'answer': 'Clara'}

Advanced Usage

The original README does not have advanced usage examples, so this part is not added.

📚 Documentation

Model description

This model is fine - tuned on the extractive question answering task -- The Stanford Question Answering Dataset -- SQuAD2.0. For convenience, this model is prepared to be used with the frameworks PyTorch, Tensorflow and ONNX.

Intended uses & limitations

This model can handle mismatched question - context pairs. Make sure to specify handle_impossible_answer=True when using QuestionAnsweringPipeline.

Training and evaluation data

Training and evaluation was done on SQuAD2.0.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e - 05
• train_batch_size: 64
• eval_batch_size: 8
• seed: 42
• distributed_type: tpu
• num_devices: 8
• total_train_batch_size: 512
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• num_epochs: 3.0

Training results

Framework versions

• Transformers 4.17.0.dev0
• Pytorch 1.9.0+cu111
• Datasets 1.18.3
• Tokenizers 0.11.6

About Us

Squirro marries data from any source with your intent, and your context to intelligently augment decision - making - right when you need it!

An Insight Engine at its core, Squirro works with global organizations, primarily in financial services, public sector, professional services, and manufacturing, among others. Customers include Bank of England, European Central Bank (ECB), Deutsche Bundesbank, Standard Chartered, Henkel, Armacell, Candriam, and many other world - leading firms.

Founded in 2012, Squirro is currently present in Zürich, London, New York, and Singapore. Further information about AI - driven business insights can be found at http://squirro.com.

Social media profiles:

• Redefining AI Podcast (Spotify): https://open.spotify.com/show/6NPLcv9EyaD2DcNT8v89Kb
• Redefining AI Podcast (Apple Podcasts): https://podcasts.apple.com/us/podcast/redefining-ai/id1613934397
• Squirro LinkedIn: https://www.linkedin.com/company/squirroag
• Squirro Academy LinkedIn: https://www.linkedin.com/showcase/the-squirro-academy
• Twitter: https://twitter.com/Squirro
• Facebook: https://www.facebook.com/squirro
• Instagram: https://www.instagram.com/squirro/

📄 License

This model is licensed under the Apache 2.0 license.

📊 Model Information

⚠️ Important Note

This model can handle mismatched question - context pairs. Make sure to specify handle_impossible_answer=True when using QuestionAnsweringPipeline.