Lacia Sum Small V1

🚀 Russian Text Summarization Model - LaciaSUM V1 (small)

This model is a fine - tuned version of d0rj/rut5 - base - summ, specifically designed for automatic text summarization. It's tailored for processing Russian texts and fine - tuned on a custom CSV dataset with original texts and their summaries. It also has potential support for English, though this hasn't been tested.

✨ Features

• Objective: Automatic abstractive summarization of texts.
• Base Model: d0rj/rut5 - base - summ.
• Dataset: A custom CSV file with columns "Text" (original text) and "Summarize" (summary).
• Preprocessing: The prefix "summarize: " is added to the original text before tokenization to help the model focus on summarization.
• Training Settings:
  • Number of epochs: 9.
  • Batch size: 4 per device.
  • Warmup steps: 1000.
  • FP16 training: Enabled if CUDA is available.
  • Hardware: Trained on an RTX 3070 (about 40 minutes of training).

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("LaciaStudio/Lacia_sum_small_v1")
model = AutoModelForSeq2SeqLM.from_pretrained("LaciaStudio/Lacia_sum_small_v1")

text = "Современные технологии оказывают значительное влияние на нашу повседневную жизнь и рабочие процессы. Искусственный интеллект становится важным инструментом, помогающим оптимизировать задачи и открывающим новые перспективы в различных областях."

# "summarize: " prefix
input_text = "summarize: " + text
inputs = tokenizer(input_text, return_tensors="pt", max_length=512, truncation=True)

summary_ids = model.generate(inputs["input_ids"], max_length=150, num_beams=4, early_stopping=True)
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)

print("Summary:", summary)

Example of summarization

Russian

Main text:

Современные технологии оказывают значительное влияние на нашу повседневную жизнь и рабочие процессы.
Искусственный интеллект становится важным инструментом, помогающим оптимизировать задачи и открывающим
новые перспективы в различных областях.

Summarized text:

Современные технологии оказывают значительное влияние на повседневную жизнь и рабочие процессы, включая
искусственный интеллект, который помогает оптимизировать задачи и открывать новые перспективы.

English

Main text:

Modern technologies have a significant impact on our daily lives and work processes. Artificial intelligence
is becoming an important tool that helps optimize tasks and opens up new opportunities in various fields.

Summarized text:

Matern technologies have a controration on our daily lives and work processes. Artificial intelligence
is becoming an important tool and helps and opens up new opportunities.

📚 Documentation

The model was fine - tuned using the Transformers library and the Seq2SeqTrainer from Hugging Face. The training script includes:

• Custom Dataset: The SummarizationDataset class reads the CSV file (ensuring correct encoding and separator), trims extra spaces from column names, and tokenizes both the source text and the target summary.
• Token Processing: To improve loss computation, padding tokens in the target text are replaced with - 100.

This model is suitable for rapid prototyping and practical applications in automatic summarization of Russian documents, news articles, and other text formats.

🔧 Technical Details

The model is based on the fine - tuning of d0rj/rut5 - base - summ. It uses the Transformers library and Seq2SeqTrainer from Hugging Face. The custom dataset and token processing methods are designed to enhance the model's performance in text summarization tasks. The training settings such as the number of epochs, batch size, and warmup steps are carefully chosen to optimize the training process.

📄 License

The model is licensed under cc - by - nc - 4.0.

Finetuned by LaciaStudio | LaciaAI