Wikihow T5 Small

🚀 Wikihow T5-small

This is a T5-small model trained on the Wikihow dataset. It can effectively summarize text, offering a practical solution for information extraction and content condensation.

🚀 Quick Start

Prerequisites

Before using the model, ensure you have installed the necessary libraries, such as transformers and torch.

Installation

pip install transformers torch

Usage Example

from transformers import AutoTokenizer, AutoModelWithLMHead
import torch

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("deep-learning-analytics/wikihow-t5-small")
model = AutoModelWithLMHead.from_pretrained("deep-learning-analytics/wikihow-t5-small")

# Set the device
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model = model.to(device)

# Example text for summarization
text = """
Lack of fluids can lead to dry mouth, which is a leading cause of bad breath. Water
can also dilute any chemicals in your mouth or gut that are causing bad breath., Studies show that
eating 6 ounces of yogurt a day reduces the level of odor-causing compounds in the mouth. In
particular, look for yogurt containing the active bacteria Streptococcus thermophilus or
Lactobacillus bulgaricus., The abrasive nature of fibrous fruits and vegetables helps to clean
teeth, while the vitamins, antioxidants, and acids they contain improve dental health.Foods that can
be particularly helpful include:Apples — Apples contain vitamin C, which is necessary for health
gums, as well as malic acid, which helps to whiten teeth.Carrots — Carrots are rich in vitamin A,
which strengthens tooth enamel.Celery — Chewing celery produces a lot of saliva, which helps to
neutralize bacteria that cause bad breath.Pineapples — Pineapples contain bromelain, an enzyme that
cleans the mouth., These teas have been shown to kill the bacteria that cause bad breath and
plaque., An upset stomach can lead to burping, which contributes to bad breath. Don’t eat foods that
upset your stomach, or if you do, use antacids. If you are lactose intolerant, try lactase tablets.,
They can all cause bad breath. If you do eat them, bring sugar-free gum or a toothbrush and
toothpaste to freshen your mouth afterwards., Diets low in carbohydrates lead to ketosis — a state
in which the body burns primarily fat instead of carbohydrates for energy. This may be good for your
waistline, but it also produces chemicals called ketones, which contribute to bad breath.To stop the
problem, you must change your diet. Or, you can combat the smell in one of these ways:Drink lots of
water to dilute the ketones.Chew sugarless gum or suck on sugarless mints.Chew mint leaves.
"""

# Preprocess the text
preprocess_text = text.strip().replace("\n","")
tokenized_text = tokenizer.encode(preprocess_text, return_tensors="pt").to(device)

# Generate the summary
summary_ids = model.generate(
            tokenized_text,
            max_length=150, 
            num_beams=2,
            repetition_penalty=2.5, 
            length_penalty=1.0, 
            early_stopping=True
        )

# Decode the summary
output = tokenizer.decode(summary_ids[0], skip_special_tokens=True)

print ("\n\nSummarized text: \n",output)

✨ Features

• Trained on Wikihow Dataset: The model is trained on the Wikihow dataset, enabling it to handle various types of text summaries.
• Good Performance Metrics: It achieved a Rouge1 score of 31.2 and a RougeL score of 24.5, indicating its effectiveness in summarization.

📦 Installation

pip install transformers torch

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelWithLMHead
import torch

tokenizer = AutoTokenizer.from_pretrained("deep-learning-analytics/wikihow-t5-small")
model = AutoModelWithLMHead.from_pretrained("deep-learning-analytics/wikihow-t5-small")

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model = model.to(device)

text = "Your input text here"
preprocess_text = text.strip().replace("\n","")
tokenized_text = tokenizer.encode(preprocess_text, return_tensors="pt").to(device)

summary_ids = model.generate(
            tokenized_text,
            max_length=150, 
            num_beams=2,
            repetition_penalty=2.5, 
            length_penalty=1.0, 
            early_stopping=True
        )

output = tokenizer.decode(summary_ids[0], skip_special_tokens=True)

print ("\n\nSummarized text: \n",output)

📚 Documentation

Model Training

The model was trained for 3 epochs using a batch size of 16 and a learning rate of 3e-4. The max_input_length is set to 512, and the max_output_length is 150.

Metrics

Blog Post

We have written a blog post that covers the training procedure. Please find it here.

🔧 Technical Details

The model is based on the T5-small architecture, which is a pre - trained language model. It is fine - tuned on the Wikihow dataset for the summarization task. During training, a batch size of 16 and a learning rate of 3e - 4 are used for 3 epochs. The input length is set to a maximum of 512 tokens, and the output length is set to a maximum of 150 tokens.