EraX-VL-7B-V1.5 Open-Source Multi-Modal Model - Free Deployment with Support for OCR and Multilingual Visual Question Answering, Especially Good for Vietnamese

Erax VL 7B V1.5

Developed by mxw1998

EraX-VL-7B-V1.5 is a powerful multimodal model specializing in Optical Character Recognition (OCR) and Visual Question Answering (VQA), excelling in multilingual environments with particular expertise in Vietnamese.

Image-to-Text

Transformers

Supports Multiple LanguagesOpen Source License:Apache-2.0 #Vietnamese OCR Specialist #Multimodal Medical Document Processing #Multi-turn Visual Reasoning

Downloads 26

Release Time : 4/23/2025

Model Overview

A multimodal model fine-tuned based on Qwen/Qwen2-VL-7B-Instruct, proficient in document recognition, visual question answering, and multi-turn reasoning, with special optimization for Vietnamese language processing.

Model Features

Exceptional Vietnamese OCR Capability

Optimized for Vietnamese documents (such as medical forms, invoices, ID cards, etc.) with high recognition accuracy.

Multi-turn Visual Question Answering

Supports complex multi-turn question answering and reasoning based on images.

Multilingual Support

In addition to Vietnamese, supports processing in English and Chinese.

Open-Source Model

Open-sourced under Apache 2.0 license, freely available for use and modification.

Model Capabilities

Document Information Extraction

Image Content Description

Multi-turn Visual Question Answering

Multilingual Text Recognition

Medical Form Processing

Business Document Analysis

Use Cases

Healthcare

Medical Prescription Recognition

Extract patient information, medication lists, and dosages from medical prescriptions.

Accurately identifies key information in Vietnamese medical documents.

Medical Report Analysis

Parse various indicators and diagnostic results in medical examination reports.

Structured output of medical examination data.

Financial Services

Invoice Processing

Automatically recognize amounts, tax numbers, and product information in invoices.

Improves financial processing efficiency.

Identity Verification

Extract personal information from identification documents.

Simplifies KYC processes.

Government Services

Document Processing

Process various government documents such as driver's licenses and vehicle registration certificates.

Automates document information entry.

🚀 EraX-VL-7B-V1.5

EraX-VL-7B-V1.5 is a robust multimodal model for OCR and VQA, excelling in various languages, especially Vietnamese. It offers precise recognition for diverse documents and is beneficial for multiple industries.

Logo

✨ Features

Hot on the heels of the popular EraX-VL-7B-V1.0 model, we proudly present EraX-VL-7B-V1.5, another robust multimodal model for OCR (optical character recognition) and VQA (visual question-answering) that excels in various languages 🌍, with a particular focus on Vietnamese 🇻🇳. This model stands out for its precise recognition capabilities across a range of documents 📝, including medical forms 🩺, invoices 🧾, bills of sale 💳, quotes 📄, and medical records 💊. This functionality is expected to be highly beneficial for hospitals 🏥, clinics 💉, insurance companies 🛡️, and other similar applications 📋. Built on the solid foundation of the Qwen/Qwen2-VL-2B-Instruct[1], which we found to be of high quality and fluent in Vietnamese, EraX-VL-7B-V1.5 has been fine-tuned to enhance its performance. We plan to continue improving and releasing new versions for free, along with sharing performance benchmarks in the near future.

One standing-out feature of EraX-VL-7B-V1.5 is the capability to do multi-turn Q&A with impressive reasoning capability!

NOTA BENE:

EraX-VL-7B-V1.5 is NOT a typical OCR-only tool likes Tesseract but is a Multimodal LLM-based model. To use it effectively, you may have to twist your prompt carefully depending on your tasks.
This model was NOT finetuned with medical (X-ray) dataset or car accidences (yet). Stay tune for updated version coming up sometime early 2025.

EraX-VL-7B-V1.5 is a young member of our EraX's LànhGPT collection of LLM models.

Developed by:
- Nguyễn Anh Nguyên (nguyen@erax.ai)
- Nguyễn Hồ Nam (BCG)
- Phạm Huỳnh Nhật (nhat.ph@erax.ai)
- Phạm Đình Thục (thuc.pd@erax.ai)
Funded by: Bamboo Capital Group and EraX
Model type: Multimodal Transformer with over 7B parameters
Languages (NLP): Primarily Vietnamese with multilingual capabilities
License: Apache 2.0
Fine-tuned from: Qwen/Qwen2-VL-7B-Instruct
Prompt examples: Some popular prompt examples on Github.

📊 Benchmarks

🏆 LeaderBoard

The EraX-VL-7B-V1.5 achieved exceptionally high performance compared to other models of equal size or even 10 times larger, and we open-source! You can re-run the benchmark at any time.

Models	Open-Source	VI-MTVQA
EraX-VL-7B-V1.5 🥇	✅	47.2
Qwen2-VL 72B 🥈	✘	41.6
ViGPT-VL 🥉	✘	39.1
EraX-VL-2B-V1.5	✅	38.2
EraX-VL-7B-V1	✅	37.6
Vintern-1B-V2	✅	37.4
Qwen2-VL 7B	✅	30.0
Claude3 Opus	✘	29.1
GPT-4o mini	✘	29.1
GPT-4V	✘	28.9
Gemini Ultra	✘	28.6
InternVL2 76B	✅	26.9
QwenVL Max	✘	23.5
Claude3 Sonnet	✘	20.8
QwenVL Plus	✘	18.1
MiniCPM-V2.5	✅	15.3

The test code for evaluating models in the paper can be found in: EraX-JS-Company/EraX-MTVQA-Benchmark

🎉 API trial

Please contact nguyen@erax.ai for API access inquiry.

💻 Usage Examples

📋 Basic Usage - OCR for Multi-Images

Example 01: Citizen identification card

Front View

Back View

Source: Google Support

{
    "Số thẻ": "037094012351",
    "Họ và tên": "TRỊNH QUANG DUY",
    "Ngày sinh": "04/09/1994",
    "Giới tính": "Nam",
    "Quốc tịch": "Việt Nam",
    "Quê quán": "Tân Thành, Kim Sơn, Ninh Bình",
    "Nơi thường trú": "Xóm 6\nTân Thành, Kim Sơn, Ninh Bình",
    "Có giá trị đến": "04/09/2034",
    "Đặc điểm nhân dạng": "sẹo chấm c. 1cm trên đuôi mắt trái",
    "Nơi cấp": "CỤC TRƯỞNG CỤC CẢNH SÁT\nQUẢN LÝ HÀNH CHÍNH VỀ TRẬT TỰ XÃ HỘI",
    "Ngày cấp": "10/12/2022",
    "Cán bộ ký tên": "Nguyễn Quốc Hùng",
    "Mã định danh": "IDVNM0940123513037094012351"
}

Example 02: Driver's License

Front View

Back View

Source: Báo Pháp luật

{
  "No.":"400116012313",
  "Fullname":"NGUYỄN VĂN DŨNG",
  "Date_of_birth":"08/06/1979",
  "Nationality":"VIỆT NAM",
  "Address":"X. Quỳnh Hầu, H. Quỳnh Lưu, T. Nghệ An Nghệ An, ngày/date 23 tháng/month 04 năm/year 2022",
  "Hang_Class":"FC",
  "Expires":"23/04/2027",
  "Place_of_issue":"Nghệ An",
  "Date_of_issue":"ngày/date 23 tháng/month 04 năm/year 2022",
  "Signer":"Trần Anh Tuấn",
  "Các loại xe được phép":"Ô tô hạng C kéo rơmoóc, đầu kéo kéo sơmi rơmoóc và xe hạng B1, B2, C, FB2 (Motor vehicle of class C with a trailer, semi-trailer truck and vehicles of classes B1, B2, C, FB2)",
  "Mã số":""
}

Example 03: Vehicle Registration Certificate

Source: Báo Vietnamnet

{
    "Tên chủ xe": "NGUYỄN TÔN NHUẬN",
    "Địa chỉ": "KE27 Kp3 P.TTTây Q7",
    "Nhãn hiệu": "HONDA",
    "Số loại": "DYLAN",
    "Màu sơn": "Trắng",
    "Năm sản xuất": "2012",
    "Số máy": "F03E-0057735",
    "Số khung": "SA04F-070410",
    "Dung tích": "152",
    "Số chỗ ngồi": "02",
    "Biển số đăng ký": "59V1-498.89",
    "Đăng ký lần đầu ngày": "08/06/2004",
    "Chức vụ": "Thượng tá",
    "Người ký": "Trần Văn Hiểu"
}

Example 04: Vehicle Registration

Source: https://llumar.com.vn

{
    "vehicle": {
        "registration_number": "30A-072.36",
        "vehicle_inspection_number": "2903V-093515",
        "type": "ô tô con",
        "mark": "MERCEDES-BENZ",
        "model_code": "C300 W204",
        "engine_number": "27294732096079",
        "chassis_number": "RLMGF5EX3DV005333",
        "manufactured_year_and_country": "2013, Việt Nam",
        "life_time_limit_to": "",
        "commercial_use": "",
        "modification": ""
    },
    "specifications": {
        "wheel_formula": "4x2",
        "wheel_tread": "1521/1512 (mm)",
        "overall_dimension": "4650 x 1770 x 1429 (mm)",
        "largest_luggage_container_dimension": "",
        "wheelbase": "2760 (mm)",
        "kerb_mass": "1575 (kg)",
        "design_authorized_pay_load": "",
        "design_authorized_total_mass": "2090/2090 (kg)",
        "design_authorized_towed_mass": "",
        "permissible_number_of_pers_carried": "5 chỗ ngồi, 0 chỗ đứng, 0 chỗ nằm",
        "type_of_fuel_used": "Xăng",
        "engine_displacement": "2996 (cm3)",
        "max_output_per_rpm": "170(kW)/6000vph",
        "number": "KC-1292285"
    },
    "inspection_report_number": "2905V-20953/16",
    "valid_until": "31/01/2018",
    "place_date_of_issue": "Hà Nội, ngày 1 tháng 8 năm 2016",
    "inspection_center": "ĐƠN VỊ KIỂM ĐỊNH XE CƠ GIỚI",
    "signature": "Ngọc Tuấn",
    "equipped_with_tachograph": "",
    "inspection_stamp_was_not_issued": "",
    "notes": "Biển đăng ký nền trắng"
}

Example 05: Receipt

Source: https://tintucketoan.com/

{
    'Mẫu số': '01GKTKT3/001', 
    'Ký hiệu': 'TC/18P', 
    'Số': '0000030', 
    'Họ tên người mua hàng': None, 
    'Tên đơn vị': 'Công Ty TNHH Kế Toán Hà Nội', 
    'Mã số thuế': '0106235869', 
    'Địa chỉ': 'Số 49 Ngõ 322 Lê Trọng Tấn, phường Khương Mai, quận Thanh Xuân, Hà Nội', 
    'Hình thức thanh toán': 'TM', 
    'STT': None, 
    'Tên hàng hóa, dịch vụ': 'Tra cứu phần mềm thư viện pháp luật trực tuyến', 
    'Đơn vị tính': None, 
    'Số lượng': None, 
    'Đơn giá': '168.000', 
    'Thành tiền': '2.016.000', 
    'Thuế suất GTGT': None, 
    'Tiền thuế GTGT': None, 
    'Tổng cộng tiền thanh toán': 
}

📄 License

The model is licensed under the Apache 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご