Jpharmatron 7B

🚀 JPharmatron-7B

JPharmatron-7B是一款專為製藥應用和研究設計的70億參數大語言模型，能在製藥文書和研究領域發揮重要作用。

✨ 主要特性

• 基於Qwen2.5 - 7B持續預訓練，使用來自日語和英語數據集的88億個標記，相比JPharmatron - 7B - base模型，通過Qwen2.5 - 7B - Instruct的聊天向量增強了聊天能力。
• 專為製藥文書和研究應用而設計，不適用於醫療用途或其他對風險敏感的用途。
• 在與同規模的其他通用/特定領域模型的評估對比中，在五個基準測試中均取得了最高分。

📚 詳細文檔

模型詳情

模型描述

JPharmatron - 7B基於Qwen2.5 - 7B，使用來自日語和英語數據集的88億個標記進行持續預訓練。與JPharmatron - 7B - base模型相比，它從Qwen2.5 - 7B - Instruct的聊天向量中獲得了增強的聊天能力。

• 開發者：EQUES Inc.
• 資助方：GENIAC項目
• 模型類型：自迴歸解碼器
• 支持語言：日語、英語
• 許可證：CC - BY - SA - 4.0

模型資源

• 倉庫地址：https://github.com/EQUES-Inc/pharma-LLM-eval
• 論文：A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLP

應用場景

此模型旨在用於製藥文書和研究應用，未經過醫療用途或其他對風險敏感用途的驗證。

評估

我們將JPharmatron - 7B模型與其他同規模的通用/特定領域模型進行了評估對比。

測試數據

使用了JPharmaBench和兩個現有基準測試（JMMLU (pharma) 和IgakuQA）。

評估結果

與Meditron3 - Qwen2.5 - 7B和Llama3.1 - Swallow - 8B - Instruct - v0.3相比，JPharmatron - 7B在所有五個基準測試中均取得了最高分。

引用信息

BibTeX：

@misc{sukeda_japanese_2025,
  title     = {A {Japanese} {Language} {Model} and {Three} {New} {Evaluation} {Benchmarks} for {Pharmaceutical} {NLP}},
  url       = {http://arxiv.org/abs/2505.16661},
  doi       = {10.48550/arXiv.2505.16661},
  abstract  = {We present a Japanese domain-specific language model for the pharmaceutical field, developed through continual pretraining on 2 billion Japanese pharmaceutical tokens and 8 billion English biomedical tokens. To enable rigorous evaluation, we introduce three new benchmarks: YakugakuQA, based on national pharmacist licensing exams; NayoseQA, which tests cross-lingual synonym and terminology normalization; and SogoCheck, a novel task designed to assess consistency reasoning between paired statements. We evaluate our model against both open-source medical LLMs and commercial models, including GPT-4o. Results show that our domain-specific model outperforms existing open models and achieves competitive performance with commercial ones, particularly on terminology-heavy and knowledge-based tasks. Interestingly, even GPT-4o performs poorly on SogoCheck, suggesting that cross-sentence consistency reasoning remains an open challenge. Our benchmark suite offers a broader diagnostic lens for pharmaceutical NLP, covering factual recall, lexical variation, and logical consistency. This work demonstrates the feasibility of building practical, secure, and cost-effective language models for Japanese domain-specific applications, and provides reusable evaluation resources for future research in pharmaceutical and healthcare NLP. Our model, codes, and datasets are released at https://github.com/EQUES-Inc/pharma-LLM-eval.},
  urldate   = {2025-05-30},
  publisher = {arXiv},
  author    = {Sukeda, Issey and Fujii, Takuro and Buma, Kosei and Sasaki, Shunsuke and Ono, Shinnosuke},
  month     = may,
  year      = {2025},
  note      = {arXiv:2505.16661 [cs]},
  annote    = {Comment: 15 pages, 9 tables, 5 figures}
}

更多信息

請參閱我們的預印本：A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLP。

模型卡片作者

@shinnosukeono