Update Summarization Bart Large Longformer

🚀 Update Summarization with BART Large and Longformer Encoder Decoder

This is a Transformer - based model for long - document generative sequence - to - sequence tasks, leveraging BART Large and Longformer Encode Decoder for longer inputs.

🚀 Quick Start

The model is designed to handle long - document generative sequence - to - sequence tasks. It combines the power of BART Large and Longformer Encode Decoder to support longer inputs.

✨ Features

• Long - document support: Capable of handling long documents for generative sequence - to - sequence tasks.
• Transformer - based: Built on the Transformer architecture, ensuring high performance.

📦 Installation

The README does not provide specific installation steps, so this section is skipped.

💻 Usage Examples

Basic Usage

import torch
from transformers import LEDTokenizer, LEDForConditionalGeneration

tokenizer = LEDTokenizer.from_pretrained("hyesunyun/update-summarization-bart-large-longformer")
model = LEDForConditionalGeneration.from_pretrained("hyesunyun/update-summarization-bart-large-longformer")

input = "<EV> <t> Hypoglycemic effect of bitter melon compared with metformin in newly diagnosed type 2 diabetes patients. <abs> ETHNOPHARMACOLOGICAL RELEVANCE: Bitter melon (Momordica charantia L.) has been widely used as an traditional medicine treatment for diabetic patients in Asia. In vitro and animal studies suggested its hypoglycemic activity, but limited human studies are available to support its use. AIM OF STUDY: This study was conducted to assess the efficacy and safety of three doses of bitter melon compared with metformin. MATERIALS AND METHODS: This is a 4-week, multicenter, randomized, double-blind, active-control trial. Patients were randomized into 4 groups to receive bitter melon 500 mg/day, 1,000 mg/day, and 2,000 mg/day or metformin 1,000 mg/day. All patients were followed for 4 weeks. RESULTS: There was a significant decline in fructosamine at week 4 of the metformin group (-16.8; 95% CI, -31.2, -2.4 mumol/L) and the bitter melon 2,000 mg/day group (-10.2; 95% CI, -19.1, -1.3 mumol/L). Bitter melon 500 and 1,000 mg/day did not significantly decrease fructosamine levels (-3.5; 95% CI -11.7, 4.6 and -10.3; 95% CI -22.7, 2.2 mumol/L, respectively). CONCLUSIONS: Bitter melon had a modest hypoglycemic effect and significantly reduced fructosamine levels from baseline among patients with type 2 diabetes who received 2,000 mg/day. However, the hypoglycemic effect of bitter melon was less than metformin 1,000 mg/day. <EV> <t> Momordica charantia for type 2 diabetes mellitus. <abs> There is insufficient evidence to recommend momordica charantia for type 2 diabetes mellitus. Further studies are therefore required to address the issues of standardization and the quality control of preparations. For medical nutritional therapy, further observational trials evaluating the effects of momordica charantia are needed before RCTs are established to guide any recommendations in clinical practice."
inputs_dict = tokenizer(input, padding="max_length", max_length=10240, return_tensors="pt", truncation=True)
input_ids = inputs_dict.input_ids
attention_mask = inputs_dict.attention_mask
global_attention_mask = torch.zeros_like(attention_mask)
# put global attention on <s> token
global_attention_mask[:, 0] = 1

predicted_summary_ids = model.generate(input_ids, attention_mask=attention_mask, global_attention_mask=global_attention_mask)
print(tokenizer.batch_decode(predicted_summary_ids, skip_special_tokens=True))

📚 Documentation

Intended uses

Format your data so that each new article or evidence to add have <EV> token in front with each title prefixed by <t> and each abstract prefixed by <abs>. The original summary should also be in the same format. The list of articles and original summary can be concatenated in any order as long as they have the correct separator tokens.

Limitations and bias

Provide examples of latent issues and potential remediations.

Training data

Used pre - trained LED model and fine - tuned using the dataset found in this github repo.

Training procedure

Preprocessing, hardware used, hyperparameters...

Eval results

The README does not provide detailed evaluation results, so specific content is not added here.

BibTeX entry and citation info

@inproceedings{...,
  year={2021}
}

🔧 Technical Details

The README does not provide specific technical details, so this section is skipped.

📄 License

The model is released under the apache - 2.0 license.