Bart Paraphrase

🚀 BART Paraphrase Model (Large)

A large BART seq2seq (text2text generation) model fine-tuned on 3 paraphrase datasets.

🚀 Quick Start

You can use the pre - trained model for paraphrasing an input sentence. Here is a simple example of how to use it:

import torch
from transformers import BartForConditionalGeneration, BartTokenizer

input_sentence = "They were there to enjoy us and they were there to pray for us."

model = BartForConditionalGeneration.from_pretrained('eugenesiow/bart-paraphrase')
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
tokenizer = BartTokenizer.from_pretrained('eugenesiow/bart-paraphrase')
batch = tokenizer(input_sentence, return_tensors='pt')
generated_ids = model.generate(batch['input_ids'])
generated_sentence = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)

print(generated_sentence)

Output

['They were there to enjoy us and to pray for us.']

✨ Features

• Bart uses a standard seq2seq/machine translation architecture with a bidirectional encoder (like BERT) and a left - to - right decoder (like GPT).
• The pretraining task involves randomly shuffling the order of the original sentences and a novel in - filling scheme, where spans of text are replaced with a single mask token.
• BART is particularly effective when fine - tuned for text generation. This model is fine - tuned on 3 paraphrase datasets (Quora, PAWS and MSR paraphrase corpus).

📚 Documentation

Model description

The BART model was proposed in BART: Denoising Sequence - to - Sequence Pre - training for Natural Language Generation, Translation, and Comprehension by Lewis et al. (2019).

The original BART code is from this repository.

Intended uses & limitations

You can use the pre - trained model for paraphrasing an input sentence.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

🔧 Technical Details

Training data

The model was fine - tuned on a pretrained [facebook/bart - large](https://huggingface.co/facebook/bart - large), using the Quora, PAWS and [MSR paraphrase corpus](https://www.microsoft.com/en - us/download/details.aspx?id = 52398).

Training procedure

We follow the training procedure provided in the simpletransformers seq2seq example.

📄 License

The model is released under the Apache 2.0 license.

📚 BibTeX entry and citation info

@misc{lewis2019bart,
      title={BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension}, 
      author={Mike Lewis and Yinhan Liu and Naman Goyal and Marjan Ghazvininejad and Abdelrahman Mohamed and Omer Levy and Ves Stoyanov and Luke Zettlemoyer},
      year={2019},
      eprint={1910.13461},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}