Tablecell Htr

🚀 Handwritten text recognition for table cell images

This model performs handwritten text recognition from text line images, trained on specific Finnish table data.

🚀 Quick Start

The model can be used for predicting the text content of images. It is recommended to use GPU for inference if available.

from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image
import torch

# Use GPU if available
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Model location in Huggingface Hub
model_checkpoint = "Kansallisarkisto/tablecell-htr"
# Path to textline image
line_image_path = "/path/to/textline_image.jpg"

# Initialize processor and model
processor = TrOCRProcessor.from_pretrained(model_checkpoint)
model = VisionEncoderDecoderModel.from_pretrained(model_checkpoint).to(device)

# Open image file and extract pixel values
image = Image.open(line_image_path).convert("RGB")
pixel_values = processor(image, return_tensors="pt").pixel_values

# Use the model to generate predictions 
generated_ids = model.generate(pixel_values.to(device))
# Use the processor to decode ids to text
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(generated_text)

The model that is downloaded from the HuggingFace Hub is saved locally to ~/.cache/huggingface/hub/.

✨ Features

• The model performs handwritten text recognition from text line images.
• It was trained by fine-tuning the National Archives' Multicentury HTR model and Microsoft's TrOCR model using text line images from Finnish death record and census record tables from the 1930s.

📦 Installation

No specific installation steps provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image
import torch

# Use GPU if available
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Model location in Huggingface Hub
model_checkpoint = "Kansallisarkisto/tablecell-htr"
# Path to textline image
line_image_path = "/path/to/textline_image.jpg"

# Initialize processor and model
processor = TrOCRProcessor.from_pretrained(model_checkpoint)
model = VisionEncoderDecoderModel.from_pretrained(model_checkpoint).to(device)

# Open image file and extract pixel values
image = Image.open(line_image_path).convert("RGB")
pixel_values = processor(image, return_tensors="pt").pixel_values

# Use the model to generate predictions 
generated_ids = model.generate(pixel_values.to(device))
# Use the processor to decode ids to text
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(generated_text)

📚 Documentation

Intended uses & limitations

The model has been trained to recognize handwritten text from a specific type of table cell data, and may generalize poorly to other datasets. The model takes as input text line images, and the use of other types of inputs are not recommended.

Training data

Model was trained using 6704 text line images, while the validation dataset contained 836 text line images.

Training procedure

This model was trained using a NVIDIA RTX A6000 GPU with the following hyperparameters:

• train batch size: 16
• epochs: 15
• optimizer: AdamW
• maximum length of text sequence: 64

For other parameters, the default values were used (find more information here). The training code is available in the train_trocr.py code file.

Evaluation results

Evaluation results using the validation dataset are listed below:

The metrics were calculated using the Evaluate library. More information on the CER metric can be found here. More information on the WER metric can be found here.

🔧 Technical Details

The model is based on fine - tuning the National Archives' Multicentury HTR model and Microsoft's TrOCR model. It uses specific text line images from Finnish death record and census record tables from the 1930s for training.

📄 License

This project is licensed under the MIT license.

📋 Information Table