MinerU Open-Source Model - Free Convert PDF to Markdown, Accurately Identify Formulas and Tables While Preserving Layout

Mineru

Developed by kitjesen

This model converts PDF documents into Markdown format while preserving the original document layout structure and accurately recognizing mathematical formulas and tables.

Image-to-Text

Transformers

Supports Multiple LanguagesOpen Source License:Apache-2.0 #PDF structured parsing #Academic document conversion #Multimodal processing

Downloads 122

Release Time : 11/7/2024

Model Overview

MinerU employs a multi-model ensemble architecture, suitable for automated PDF-to-Markdown scenarios, with capabilities to maintain original document layout, accurately identify mathematical formulas, and fully reconstruct table frameworks.

Model Features

Multi-model ensemble architecture

Combines layout analysis, formula detection, formula recognition, and table processing models to ensure conversion accuracy.

Preserves original layout

Maintains the original document's layout structure, ensuring the converted Markdown document matches the original PDF.

Mathematical formula recognition

Accurately identifies and converts mathematical formulas, making it suitable for academic papers and technical documents.

Table reconstruction

Fully reconstructs table frameworks to ensure data accuracy and readability.

Model Capabilities

PDF to Markdown

Layout analysis

Formula detection

Formula recognition

Table processing

Use Cases

Academic research

Academic paper conversion

Converts academic paper PDFs into Markdown format for easy editing and sharing.

Text conversion accuracy: 95%, Formula conversion accuracy: 90%

Technical documentation

Technical document conversion

Converts technical document PDFs into Markdown format for version control and collaboration.

Table conversion accuracy: 85%

Education

Textbook conversion

Converts textbook PDFs into Markdown format for online learning and editing.

🚀 MinerU PDF to Markdown Model

This model can convert PDF documents into Markdown format, offering a seamless solution for document transformation.

🚀 Quick Start

The MinerU PDF to Markdown Model enables you to effortlessly convert PDF documents into Markdown format. Here's a simple example to get you started:

from transformers import pipeline

converter = pipeline("document-conversion", model="kitjesen/MinerU")
markdown = converter("document.pdf")

✨ Features

Multi - model Architecture: MinerU employs a multi - model combination architecture, including document layout analysis, math formula detection and recognition, and table structure reconstruction.
Comprehensive Functionality: It supports text layout analysis, math formula recognition, and table structure reconstruction during the PDF - to - Markdown conversion process.

📚 Documentation

Model Description

MinerU uses a multi - model combination architecture:

Layout: Document layout analysis (Detectron2)
MFD: Math formula detection (PyTorch)
MFR: Math formula recognition (BERT - based)
TabRec: Table recognition and reconstruction (T5 - based)

Intended Uses

This model is designed for automatically converting PDF documents into Markdown format, supporting:

Text layout analysis
Math formula recognition
Table structure reconstruction

Limitations and Bias

Maximum Page Support: 100 pages
PDF File Size Limit: 50MB
Supported Languages: Chinese, English

Training Data

The model is trained using the following datasets:

Academic paper dataset
Textbook document dataset
Technical document dataset

Training Procedure

It uses a multi - stage training process:

Pretrain each sub - model.
Jointly train and optimize.
End - to - end fine - tuning.

Evaluation Results

Text Recognition Accuracy: 95%
Formula Recognition Accuracy: 90%
Table Reconstruction Accuracy: 85%

Environmental Impact

Hardware Requirement: GPU with 8GB+ VRAM
Inference Time: ~2s/page

Technical Specifications

Property	Details
Model Architecture	- Layout: Detectron2 (FasterRCNN) - MFD: Custom CNN - MFR: BERT - based - TabRec: T5 - based
Hardware Requirements	- RAM: 16GB+ - GPU: 8GB+ VRAM - Storage: 5GB
Software Requirements	- Python >= 3.7 - PyTorch >= 1.9.0 - transformers >= 4.28.0 - detectron2

📄 License

This project is licensed under the Apache 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご