Ppace V1.0

## 🚀 パンデミックPACT高度分類エンジン (PPACE)

PPACEは、世界保健機関（WHO）に準拠した研究優先事項に基づいて、資金提供を受けた生物医学プロジェクトの研究要約を自動分類するために微調整された80億パラメータの大規模言語モデル（LLM）です。GLOPID - RパンデミックPACTプロジェクトの一環として開発され、PPACEは、発生可能性のある幅広い疾病に関する研究資金と臨床エビデンスの追跡と分析を支援します。

## 🚀 クイックスタート
PPACEモデルを使用することで、生物医学研究の抽象化を分類することができます。以下に、使用方法の概要を説明します。

## ✨ 主な機能
- WHOに準拠した研究優先事項に基づいて、生物医学研究の要約を自動分類します。
- 人間によるアノテーションと大規模LLMによる根拠生成を組み合わせたデータセットを利用し、モデルの解釈可能性と精度を向上させます。
- Low - Rank Adaptation (LoRA)を使用して微調整され、高性能を維持しながら効率的なトレーニングを実現します。

## 📦 インストール
このモデルを使用するには、必要なライブラリをインストールする必要があります。以下のコードでは、モデルを読み込むために必要な`transformers`ライブラリなどを使用しています。

```python
# 必要なライブラリのインストール
# ここでは仮にpipでのインストールを想定
pip install transformers torch

💻 使用例

基本的な使用法

まず、入力プロンプトを作成し、モデルを読み込み、推論を行います。

import torch
import transformers as ts

def construct_input_prompt(title, abstract):
    categories = """We have a project in the area of biomedical research which we want to classify in terms of the research priorities it related to. We have 12 possible research priorities and a project can be mapped to 1 or more of these priorities. The following is a guide on what each of these 12 categories are alongside the specific areas that they cover. 

1. Pathogen: Natural History, Transmission, and Diagnostics:
    Development of diagnostic tools, understanding pathogen morphology, genomics, and genotyping, studying immunity, using disease models, and assessing the environmental stability of pathogens.

2. Animal and Environmental Research & Research on Diseases Vectors:
    Animal sources, transmission routes, vector biology, and control strategies for vectors.

3. Epidemiological Studies:
    Research on disease transmission dynamics, susceptibility, control measure effectiveness, and disease mapping through surveillance and reporting.

4. Clinical Characterisation and Management in Humans:
    Prognostic factors for disease severity, disease pathogenesis, supportive care and management, long - term health consequences, and clinical trials for disease management.

5. Infection Prevention and Control:
    Research on community restriction measures, barriers and PPE, infection control in healthcare settings, and measures at the human - animal interface.

6. Therapeutics Research, Development, and Implementation:
    Pre - clinical studies for therapeutic development, clinical trials for therapeutic safety and efficacy, development of prophylactic treatments, logistics and supply chain management for therapeutics, clinical trial design for therapeutics, and research on adverse events related to therapeutic administration.

7. Vaccines Research, Development, and Implementation:
    Pre - clinical studies for vaccine development, clinical trials for vaccine safety and efficacy, logistics and distribution strategies for vaccines, vaccine design and administration, clinical trial design for vaccines, research on adverse events related to immunisation, and characterisation of vaccine - induced immunity.

8. Research to Inform Ethical Issues:
    Ethical considerations in research design, ethical issues in public health measures, ethical clinical decision - making, ethical resource allocation, ethical governance, and ethical considerations in social determinants of health.

9. Policies for public health, disease control and  community resilience:
    Approaches to public health interventions, community engagement, communication and infodemic management, vaccine/therapeutic hesitancy, and policy research and interventions.

10. Secondary Impacts of Disease, Response, and Control Measures:
    Indirect health impacts, social impacts, economic impacts, and other secondary impacts such as environmental effects, food security, and infrastructure.

11. Health Systems Research:
    Health service delivery, health financing, access to medicines and technologies, health information systems, health leadership and governance, and health workforce management.

12. Capacity Strengthening:
    Individual capacity building, institutional capacity strengthening, systemic/environmental components, and cross - cutting activities across all levels of capacity building."""

    prompt = "Based on the research categorization guidelines, classify the following project into the appropriate primary research priorities using the categories 1 to 12."
    prompt += f"\n\n{categories.strip()}\n\nProject Information:\n\n"
    prompt += f"### Title:\n'''\n{title.strip()}\n'''\n\n### Abstract:\n'''\n{abstract.strip()}\n'''\n\n"
    prompt += "Based on this information, identify the relevant research categories for this project. Provide clear explanation for your choices. Section your response in the following format:"
    prompt += "\n\n### Explanation: ...\n\n### Categories: ..."
    
    return prompt

title = "Neutralization of Primate Immunodeficiency Viruses"
abstract = "We will repurpose existing assays, techniques and expertise that are central to our project team's virology, structural biology, vaccine development and protein production skill - sets for HIV research, to now also work on SARS - CoV - 2 during the COVID - 19 pandemic emergency. These interactive research efforts will draw on our established methodologies and should represent a productive use of our existing NIH grant resources. We note that there continue to be institutional restrictions at all three performance sites on the effort that can be applied to our original goals relating to HIV - 1 vaccine research and development. Those goals will be unchanged, but will be pursued at a reduced effort during the period when we also work on the new SARS - CoV - 2 projects for which we have fewer institutional restrictions due to the COVID - 19 pandemic."

input_prompt = construct_input_prompt(title, abstract)

model_id = "nlpie/ppace - v1.0"

tokenizer = ts.AutoTokenizer.from_pretrained(model_id)

tokenizer.padding_side = 'right'
tokenizer.pad_token = tokenizer.eos_token
tokenizer.add_eos_token = True
tokenizer.bos_token, tokenizer.eos_token

model = ts.AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    attn_implementation="flash_attention_2",
    torch_dtype=torch.bfloat16)

def generateOutput(input):
    inputs = tokenizer(
        [
            tokenizer.decode(tokenizer.apply_chat_template([{"content": input, "role": "user"}]))
        ], return_tensors = "pt").to("cuda")

    output = model.generate(**inputs, max_new_tokens = 512, num_beams=4, eos_token_id=tokenizer("<|eot_id|>", add_special_tokens=False)["input_ids"][0])

    return tokenizer.decode(output[0])

output = generateOutput(input_prompt)
print(output)

🔧 技術詳細

PPACEは、Low - Rank Adaptation (LoRA)を使用して微調整されています。これにより、高性能を維持しながら効率的なトレーニングが可能になります。微調整プロセスでは、5142件のプロジェクトのデータセットで4エポックのトレーニングを行い、8台のA100 GPUを使用し、GPUごとのバッチサイズを1、勾配累積ステップを4としています。

ハイパーパラメータ

📄 ライセンス

このプロジェクトはMITライセンスの下で公開されています。

以上のREADMEでは、元の文書の内容を日本語に翻訳し、必要な要素を追加して構造化し、視覚的に分かりやすくなるように整形しました。コードブロックやテーブルなどの形式はそのままに、必要な箇所にemojiを追加しています。