Roberta2roberta L 24 Wikisplit

🚀 Roberta2Roberta_L-24_wikisplit EncoderDecoder model

An encoder-decoder model initialized on roberta-large checkpoints and fine-tuned for sentence splitting.

🚀 Quick Start

The model was introduced in this paper by Sascha Rothe, Shashi Narayan, Aliaksei Severyn and first released in this repository. It is an encoder-decoder model initialized on the roberta-large checkpoints for both the encoder and decoder, and fine-tuned on sentence splitting using the WikiSplit dataset.

Disclaimer: The model card has been written by the Hugging Face team.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("google/roberta2roberta_L-24_wikisplit")
model = AutoModelForSeq2SeqLM.from_pretrained("google/roberta2roberta_L-24_wikisplit")

long_sentence = """Due to the hurricane, Lobsterfest has been canceled, making Bob very happy about it and he decides to open Bob 's Burgers for customers who were planning on going to Lobsterfest."""

input_ids = tokenizer(tokenizer.bos_token + long_sentence + tokenizer.eos_token, return_tensors="pt").input_ids
output_ids = model.generate(input_ids)[0]
print(tokenizer.decode(output_ids, skip_special_tokens=True))
# should output
# Due to the hurricane, Lobsterfest has been canceled, making Bob very happy about it. He decides to open Bob's Burgers for customers who were planning on going to Lobsterfest. 

⚠️ Important Note

The model was not trained on the " (double quotation mark) character. So before tokenizing the text, it is advised to replace all " (double quotation marks) with two single ' (single quotation mark).

📄 License

This model is released under the Apache-2.0 license.