Trocr Base Printed

🚀 TrOCR (base-sized model, fine-tuned on SROIE)

TrOCR model fine-tuned on the SROIE dataset for optical character recognition.

🚀 Quick Start

The TrOCR model presented here is a fine - tuned version on the SROIE dataset. It was introduced in the paper TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models by Li et al. and first released in this repository.

Disclaimer: The team releasing TrOCR did not write a model card for this model, so this model card has been written by the Hugging Face team.

✨ Features

• Encoder - Decoder Architecture: The TrOCR model is an encoder - decoder model. It uses an image Transformer as the encoder and a text Transformer as the decoder. The image encoder is initialized from the weights of BEiT, and the text decoder is initialized from the weights of RoBERTa.
• Patch - based Image Input: Images are presented to the model as a sequence of fixed - size patches (resolution 16x16), which are linearly embedded. Absolute position embeddings are added before feeding the sequence to the Transformer encoder layers. Then, the Transformer text decoder autoregressively generates tokens.

📚 Documentation

Intended uses & limitations

You can use the raw model for optical character recognition (OCR) on single text - line images. Check the model hub to find fine - tuned versions for tasks that interest you.

💻 Usage Examples

Basic Usage

from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image
import requests

# load image from the IAM database (actually this model is meant to be used on printed text)
url = 'https://fki.tic.heia-fr.ch/static/img/a01-122-02-00.jpg'
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")

processor = TrOCRProcessor.from_pretrained('microsoft/trocr-base-printed')
model = VisionEncoderDecoderModel.from_pretrained('microsoft/trocr-base-printed')
pixel_values = processor(images=image, return_tensors="pt").pixel_values

generated_ids = model.generate(pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

BibTeX entry and citation info

@misc{li2021trocr,
      title={TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models}, 
      author={Minghao Li and Tengchao Lv and Lei Cui and Yijuan Lu and Dinei Florencio and Cha Zhang and Zhoujun Li and Furu Wei},
      year={2021},
      eprint={2109.10282},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}