Trocr Base Handwritten Hist Swe 2

🚀 Swedish Lion Libre

An HTR model for historical Swedish, developed through a collaboration between the Swedish National Archives, the Stockholm City Archives, the Finnish National Archives, and Jämtlands Fornskriftsällskap. This model is trained on Swedish handwriting from around 1600 - 1900.

🚀 Quick Start

Use the code below to get started with the model, but keep in mind that the image has to be a single text - line.

from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image
import requests

img_path = 'path/to/image'
image = Image.open(img_path)

processor = TrOCRProcessor.from_pretrained("microsoft/trocr - base - handwritten")
model = VisionEncoderDecoderModel.from_pretrained('Riksarkivet/trocr - base - handwritten - hist - swe - 2')
pixel_values = processor(images=image, return_tensors="pt").pixel_values

generated_ids = model.generate(pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

If you want to transcribe entire pages, consider using HTRflow, a package developed by The Swedish National Archives and intended for streamlining large and small scale HTR/OCR - projects. Install the package, write a pipeline config yaml, where you specify the models to use by their huggingface id, add preprocessing or post - processing steps, and then run the pipeline with htrflow pipeline <path/to/yaml> <path/to/image/images>. A .yaml file for an entire pipeline, transcribing full pages, could look like this:

# Demo pipeline for running text

steps:

# Region segmentation
- step: Segmentation
  settings:
    model: yolo
    model_settings:
       model: Riksarkivet/yolov9 - regions - 1
    generation_settings:
       conf: 0.3
       batch_size: 32

# Line segmentation
- step: Segmentation
  settings:
    model: yolo
    model_settings:
       model: Riksarkivet/yolov9 - lines - within - regions - 1
    generation_settings:
       conf: 0.3
       batch_size: 16

- step: TextRecognition
  settings:
    model: WordLevelTrocr
    model_settings:
       model: Riksarkivet/trocr - base - handwritten - hist - swe - 2
    generation_settings:
       batch_size: 16
       num_beams: 1

- step: ReadingOrderMarginalia
  settings:
    two_page: always

- step: RemoveLowTextConfidencePages
  settings:
    threshold: 0.95

- step: RemoveLowTextConfidenceLines
  settings:
    threshold: 0.95

# Export to Alto and Page XML 
- step: Export
  settings:
    dest: outputs/new_models/alto
    format: alto

- step: Export
  settings:
    dest: outputs/new_models/page
    format: page

# Sets label format to regionX_lineY_wordZ
labels:
  level_labels:
    - region
    - line
    - word
  sep: _
  template: "{label}{number}"

See the documentation for the HTRflow package for further instructions on specific steps and customizations

✨ Features

• Historical Swedish HTR: Specifically trained on Swedish handwriting from around 1600 - 1900, making it suitable for transcribing historical Swedish documents.
• Flexible Usage: Can be used directly without fine - tuning or fine - tuned on other types of handwriting within its domain.
• Page - level Transcription: With the help of the HTRflow package, it can transcribe entire pages efficiently.

📦 Installation

The README does not provide specific installation steps, so this section is skipped.

📚 Documentation

Model Details

Model Description

Uses

Direct Use

The model can be used without fine - tuning on all handwriting but performs best on the type of handwriting it was trained on, which is Swedish handwriting from 1600 - 1900. See below for detailed test and evaluation results.

Downstream Use

The model can be fine - tuned on other types of handwriting, or if you plan to use it to transcribe some specific material that is within its domain but not included in the training data, for instance if you got a large letter collection dating from the 17th century, it can be fine - tuned on a small amount of manually transcribed in - domain data, say 20 - 50 letters, and then used to transcribe the entire collection.

Out - of - Scope Use

The model won't work well out - of - the - box for other languages than Swedish, and it won't work well for printed text.

Training Details

Training Data

We cannot publicly release all data the model was trained on, since we ourselves haven't created all the data, but below are links to the datasets we can release publicly:

Göteborgs poliskammare 1850 - 1900
Krigshovrättens domböcker
Svea hovrätt
Bergskollegium
Frihetstidens utskottshandlingar
Carl - Fredrik Påhlmans resejournaler
Trolldomskommissionen
Göta hovrätt
Bergmästaren i Nora
Älvsborgs lösen
Jönköpings rådhusrätt magistrat

Training Procedure

Preprocessing

The text - line polygons were masked out and placed against a white backgroundy, with dimensions decided by the polygon's bounding box.

Training Hyperparameters

See config.json at [model repo](https://huggingface.co/Riksarkivet/trocr - base - handwritten - hist - swe - 2/tree/main)
training regime: bf16
learning rate: 5e - 5
weight decay: 0.01

Evaluation

In - Domain Evaluation Data (Sorted by CER)

These are the character and word error rates on evaluation data taken from the same archives that was included in the training set. Of course the evaluation samples aren't part of the training data. The number of samples included in the training - set give an indication of how the model improves by fine - tuning it on some specific material within the model's range.

Testing Data

Out - of - Domain Test Data (Sorted by CER)

These are all test - sets taken from archives that we're not at all included in the training data. So these are the results one would expect if one uses this model out - of - the - box on just any running text document within the models time - span. The entire test - suite is available here: test - suite for htr

Metrics

Character Error Rate (CER)

Character Error Rate (CER) is a metric used to evaluate the performance of a Handwritten Text Recognition (HTR) system by comparing the recognized text to the reference (ground truth) text at the character level.

The CER is calculated using the following formula:

$$ CER = \frac{S + D + I}{N} $$

Where:

• ( S ) = Number of substitutions (incorrect characters)
• ( D ) = Number of deletions (missing characters)
• ( I ) = Number of insertions (extra characters)
• ( N ) = Total number of characters in the reference text

A lower CER indicates better recognition accuracy.

Word Error Rate (WER)

Word Error Rate (WER) is a metric used to assess the accuracy of an HTR system at the word level by comparing the recognized text to the reference text.

The WER is calculated using the following formula:

$$ WER = \frac{S + D + I}{N} $$

Where:

• ( S ) = Number of substitutions (incorrect words)
• ( D ) = Number of deletions (missing words)
• ( I ) = Number of insertions (extra words)
• ( N ) = Total number of words in the reference text

A lower WER indicates better recognition accuracy.

🔧 Technical Details

The text - line polygons were masked out and placed against a white backgroundy, with dimensions decided by the polygon's bounding box. See config.json at [model repo](https://huggingface.co/Riksarkivet/trocr - base - handwritten - hist - swe - 2/tree/main) for training hyperparameters. Training regime is bf16, learning rate is 5e - 5, and weight decay is 0.01.

📄 License

The model is licensed under the apache - 2.0 license.