Oneke

🚀 OneKE：用於知識提取的雙語大語言模型

OneKE是由螞蟻集團和浙江大學聯合開發的大規模知識提取模型框架。它具備中英文雙語、跨多領域和多任務的通用知識提取能力，並提供全面的工具鏈支持。OneKE以開源方式為OpenKG開放知識圖譜社區做出了貢獻。

🚀 快速開始

建議在訓練和推理時至少擁有 20GB的顯存。

基礎用法

import torch
from transformers import (
    AutoConfig,
    AutoTokenizer,
    AutoModelForCausalLM,
    GenerationConfig,
    BitsAndBytesConfig
)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model_path = 'zjunlp/OneKE'
config = AutoConfig.from_pretrained(model_path, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)


# 4-bit Quantized OneKE
quantization_config=BitsAndBytesConfig(     
    load_in_4bit=True,
    llm_int8_threshold=6.0,
    llm_int8_has_fp16_weight=False,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
)

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    config=config,
    device_map="auto",  
    quantization_config=quantization_config,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)
model.eval()


system_prompt = '<<SYS>>\nYou are a helpful assistant. 你是一個樂於助人的助手。\n<</SYS>>\n\n'
sintruct = "{\"instruction\": \"You are an expert in named entity recognition. Please extract entities that match the schema definition from the input. Return an empty list if the entity type does not exist. Please respond in the format of a JSON string.\", \"schema\": [\"person\", \"organization\", \"else\", \"location\"], \"input\": \"284 Robert Allenby ( Australia ) 69 71 71 73 , Miguel Angel Martin ( Spain ) 75 70 71 68 ( Allenby won at first play-off hole )\"}"
sintruct = '[INST] ' + system_prompt + sintruct + '[/INST]'

input_ids = tokenizer.encode(sintruct, return_tensors="pt").to(device)
input_length = input_ids.size(1)
generation_output = model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_length=1024, max_new_tokens=512, return_dict_in_generate=True))
generation_output = generation_output.sequences[0]
generation_output = generation_output[input_length:]
output = tokenizer.decode(generation_output, skip_special_tokens=True)

print(output)

如需更詳細的推理說明，請參考 DeepKE-llm/InstructKGC/6.1.2IE專用模型。

✨ 主要特性

• 雙語通用知識提取：具備中英文雙語、跨多領域和多任務的通用知識提取能力。
• 全面工具鏈支持：提供全面的工具鏈支持，方便用戶使用。
• 開源貢獻：以開源方式為OpenKG開放知識圖譜社區做出了貢獻。

📦 安裝指南

文檔未提及具體安裝步驟，暫無法提供。

💻 使用示例

OneKE指令格式

OneKE中的指令採用類似於JSON的字典類型字符串格式。它由三個字段組成： (1) 'instruction'：任務描述，用自然語言指定模型扮演的角色和要完成的任務。 (2) 'schema'：待提取的標籤列表，明確指出要提取的信息的關鍵字段，反映用戶需求，並且是動態可變的。 (3) 'input'：指用於信息提取的源文本。

以下是各種任務的指令示例：

{
    "instruction": "You are an expert specializing in entity extraction. Please extract entities that comply with the schema definition from the input; return an empty list for non-existent entity types. Please respond in the JSON string format.",
    "schema": ["Person Name", "Education", "Position", "Nationality"],
    "input": "Mr. Liu Zhijian: Born in 1956, Chinese nationality, no permanent residency abroad, member of the Communist Party, associate degree, senior economist."
}

{
    "instruction": "You are an expert specializing in relation extraction. Please extract relationship triples that comply with the schema definition from the input; return an empty list for non-existent relationships. Please respond in the JSON string format.",
    "schema": ["Father", "Husband", "Postal Code", "Mother"],
    "input": "Ding Long took out his life savings of $12,000, which without a doubt was a substantial amount at the end of the 19th century, plus Carpentier's donation, they both funded Columbia University's sinology research together."
}

{
    "instruction": "You are an expert in structuring knowledge about graph entities. Based on the schema description of the input entity type, extract the corresponding entity instances and their property information from the text; do not output non-existent properties, return a list if there are multiple values for a property, and provide the output in a parseable json format.", 
    "schema": [
        {
            "entity_type": "Person", 
            "attributes": ["Chinese Name", "English Name", "Ancestral Home", "Date of Birth", "Place of Birth", "Occupation", "Alma Mater", "Works", "Awards"]
        }
    ], 
    "input": "Jay Chou (Jay Chou), born on January 18, 1979, in New Taipei City, Taiwan Province, ancestral home in Yongchun County, Quanzhou City, Fujian Province, Chinese pop singer, musician, actor, director, screenwriter, graduated from Tamkang High School. In 2000, he released his debut album 'Jay'. In 2001, he cemented his style of blending Eastern and Western music with the album 'Fantasy'. In 2002, he held ‘The One’ world tour; the same year, he won the Best Composer award at the 13th Taiwan Golden Melody Awards with the song 'Love Before the Century'."
}

{
    "instruction": "You are an expert specializing in event extraction. Please extract events that match the defined schema from the input; return an empty list for non-existent events, NAN for non-existent arguments, and a list if there are multiple values for an argument. Please provide your response in JSON string format.",
    "schema": [
        {
            "event_type": "Finance/Trading - Interest Rate Hike",
            "trigger": true,
            "arguments": [
                "Time"
            ]
        },
        {
            "event_type": "Finance/Trading - Interest Rate Cut",
            "trigger": true,
            "arguments": [
                "Cut Magnitude"
            ]
        },
        {
            "event_type": "Finance/Trading - Price Increase",
            "trigger": true,
            "arguments": [
                "Price Raiser"
            ]
        },
        {
            "event_type": "Finance/Trading - Price Cut",
            "trigger": true,
            "arguments": [
                "Price Cutter",
                "Time"
            ]
        }
    ],
    "input": "AI risk control solution provider Vezetech secures tens of millions of dollars in Series C+ funding"
}

{
  "instruction": "You are an expert specializing in event trigger identification. Please extract the event types and triggers that match the defined schema from the input; return an empty list if the event type doesn't exist. Please provide your response in JSON string format.", 
  "schema": ["Organizational Relationship - Dissolve", "Organizational Relationship - Layoff", "Organizational Relationship - Dismiss", "Competition Behavior - Promotion"], 
  "input": "Nestlé lays off 4,000 employees: When the times leave you behind, they won't even say goodbye!"
}

{
  "instruction": "You are an expert specializing in event argument extraction. Please extract the event arguments and their roles that match the defined schema from the input; return NAN or an empty dictionary for non-existent arguments, and a list if there are multiple values for an argument. Please provide your response in JSON string format.", 
  "schema": [{"event_type": "Organizational Relationship - Resignation/Departure", "arguments": ["Resigner", "Time", "Former Organization"]}], 
  "input": "Nestlé lays off 4,000 employees: When the times leave you behind, they won't even say goodbye!"
}

⚠️ 重要提示

考慮到特定領域內信息提取的複雜性以及對提示的高度依賴，我們支持在指令中集成Schema描述和示例，以提高提取任務的有效性。詳情請參考 自定義Schema描述指令 和 自定義示例指令。由於模型規模有限，模型輸出依賴於提示，不同的提示可能會產生不一致的結果。

OneKE指令格式轉換

指令列表：

instruction_mapper = {
    'NERzh': "你是專門進行實體抽取的專家。請從input中抽取出符合schema定義的實體，不存在的實體類型返回空列表。請按照JSON字符串的格式回答。",
    'REzh': "你是專門進行關係抽取的專家。請從input中抽取出符合schema定義的關係三元組，不存在的關係返回空列表。請按照JSON字符串的格式回答。",
    'EEzh': "你是專門進行事件提取的專家。請從input中抽取出符合schema定義的事件，不存在的事件返回空列表，不存在的論元返回NAN，如果論元存在多值請返回列表。請按照JSON字符串的格式回答。",
    'EETzh': "你是專門進行事件提取的專家。請從input中抽取出符合schema定義的事件類型及事件觸發詞，不存在的事件返回空列表。請按照JSON字符串的格式回答。",
    'EEAzh': "你是專門進行事件論元提取的專家。請從input中抽取出符合schema定義的事件論元及論元角色，不存在的論元返回NAN或空字典，如果論元存在多值請返回列表。請按照JSON字符串的格式回答。",
    'KGzh': '你是一個圖譜實體知識結構化專家。根據輸入實體類型(entity type)的schema描述，從文本中抽取出相應的實體實例和其屬性信息，不存在的屬性不輸出, 屬性存在多值就返回列表，並輸出為可解析的json格式。',
    'NERen': "You are an expert in named entity recognition. Please extract entities that match the schema definition from the input. Return an empty list if the entity type does not exist. Please respond in the format of a JSON string.",
    'REen': "You are an expert in relationship extraction. Please extract relationship triples that match the schema definition from the input. Return an empty list for relationships that do not exist. Please respond in the format of a JSON string.",
    'EEen': "You are an expert in event extraction. Please extract events from the input that conform to the schema definition. Return an empty list for events that do not exist, and return NAN for arguments that do not exist. If an argument has multiple values, please return a list. Respond in the format of a JSON string.",
    'EETen': "You are an expert in event extraction. Please extract event types and event trigger words from the input that conform to the schema definition. Return an empty list for non-existent events. Please respond in the format of a JSON string.",
    'EEAen': "You are an expert in event argument extraction. Please extract event arguments and their roles from the input that conform to the schema definition, which already includes event trigger words. If an argument does not exist, return NAN or an empty dictionary. Please respond in the format of a JSON string.", 
    'KGen': 'You are an expert in structured knowledge systems for graph entities. Based on the schema description of the input entity type, you extract the corresponding entity instances and their attribute information from the text. Attributes that do not exist should not be output. If an attribute has multiple values, a list should be returned. The results should be output in a parsable JSON format.',
}

各任務推薦的分割數量：

split_num_mapper = {
    'NER':6, 'RE':4, 'EE':4, 'EET':4, 'EEA':4, 'KG':1
}

由於一次性預測標籤集中的所有Schema過於困難且不易擴展，OneKE在訓練時採用分批處理的方式。它將指令中詢問的Schema數量進行分割，每次查詢固定數量的Schema。因此，如果一條數據的標籤集過長，它將被分割成多個指令，模型將依次處理。

Schema格式：

NER: ["Person Name", "Education", "Position", "Nationality"]   # 字符串列表
RE: ["Father", "Husband", "Postal Code", "Mother"]   # 字符串列表
EE: [{"event_type": "Finance/Trading - Interest Rate Hike", "trigger": True, "arguments": ["Time"]}, {"event_type": "Finance/Trading - Interest Rate Cut", "trigger": True, "arguments": ["Cut Magnitude"]}]  # 字典列表，"event_type" 是字符串，"trigger" 是布爾值，"arguments" 是列表
EET: ["Organizational Relationship - Dissolution", "Organizational Relationship - Layoff", "Organizational Relationship - Dismissal", "Competition Behavior - Advancement"]    # 字符串列表
EEA: [{"event_type": "Finance/Trading - Interest Rate Hike", "arguments": ["Time"]}, {"event_type": "Finance/Trading - Interest Rate Cut", "arguments": ["Cut Magnitude"]}]  # 字典列表，"event_type" 是字符串，"arguments" 是列表

以下是一個簡單的分批指令生成腳本：

def get_instruction(language, task, schema, input):
    sintructs = []
    split_num = split_num_mapper[task]
    if type(schema) == dict:
        sintruct = json.dumps({'instruction':instruction_mapper[task+language], 'schema':schema, 'input':input}, ensure_ascii=False)
        sintructs.append(sintruct)
    else:
        split_schemas = [schema[i:i+split_num] for i in range(0, len(schema), split_num)]
        for split_schema in split_schemas:
            sintruct = json.dumps({'instruction':instruction_mapper[task+language], 'schema':split_schema, 'input':input}, ensure_ascii=False)
            sintructs.append(sintruct)
    return sintructs

以下是使用上述簡單腳本的示例：

task = 'NER'
language = 'en'
schema = ['person', 'organization', 'else', 'location']
split_num = split_num_mapper[task]
split_schemas = [schema[i:i+split_num] for i in range(0, len(schema), split_num)]
input = '284 Robert Allenby ( Australia ) 69 71 71 73 , Miguel Angel Martin ( Spain ) 75 70 71 68 ( Allenby won at first play-off hole )'
sintructs = []
for split_schema in split_schemas:
    sintruct = json.dumps({'instruction':instruction_mapper[task+language], 'schema':split_schema, 'input':input}, ensure_ascii=False)
    sintructs.append(sintruct)

如需更詳細的數據轉換說明，請參考 DeepKE-llm/InstructKGC/README_CN.md/2.3測試數據轉換。

自定義Schema描述指令

{
  "instruction": "You are an expert specializing in entity extraction. Please extract entities that comply with the defined schema from the input; return an empty list for non-existent entity types. Please respond in JSON string format.",
  "schema": {
    "Position": "The entity type describes the occupation or official position of an individual or group, including specific role names such as 'producer', 'scorekeeper', 'ascetic', 'oil painter'.",
    "Attraction": "The entity type of attraction includes buildings, museums, memorials, art galleries, rivers, peaks, etc. Representative entities include the Pentagon, Tate Modern, Zheng Chenggong Memorial Hall, Duxi Palace, Barikasa, Robo River, Gunung Batur, Yugong Yishan LIVE, Xu Beihong Memorial Hall, Madame Tussauds, etc.",
    "Company": "Company is an entity type representing any legal entity or business organization. This type of entity can be a catering group, manufacturer, retailer, hotel, bank, design institute, etc. Examples include: 'Shangri-La Hotel Group', 'JVC', 'Shanghai Coolray Professional eSports Peripheral Store', 'K2•Haitang Bay', 'Wuhan Iron and Steel', 'louisvuitton', 'Bank of Scotland', 'Beijing Institute of Architectural Design', '7 Days Inn', 'Vanke Group'.",
    "Address": "Address entities refer to entities with geographical location information, representing specific places such as a country, city, region, street, or abstract geographic areas. Examples include: 'the river dock at the southeast tip of downtown Manhattan', 'Tuapse', 'Venice, Italy', 'Huzhou Hot Spring Golf Course', 'North Carolina', 'Beijing-Tianjin region', 'Happy Internet Cafe', 'Yinian Nursing Home', 'Shangtang Town Pudong', 'Inner Mongolia Autonomous Region Chifeng City', etc.",
    "Organization": "Organizational entities refer to collective organizations such as companies, shops, clubs, schools, etc. They play a certain role in social and economic activities and have certain personality rights.",
    "Movie": "Movie entities include titles of movies in Chinese or English, and sometimes also include names of characters in films."
  },
  "input": "It is difficult for me to imagine setting up another Haifishing Plaza. When we obtained this project, I just happened to be in Sanya."
}

自定義示例指令

由於示例實例通常較長，並且由於模型訓練的最大長度有限，過多的示例可能會對模型性能產生負面影響。因此，我們建議提供2個示例：一個正例和一個反例，同時將Schema數量保持為一個。

{
    "instruction": "You are an expert in entity extraction. Please extract entities from the input that fit the defined schema; return an empty list for non-existent entity types. Please respond in the format of a JSON string. You may refer to the example to guide your extraction.",
    "schema": [
        "Biomarker"
    ],
    "example": [
        {
            "input": "Diagnostic criteria for CKD include: 1. Any of the following indicators persisting for more than 3 months; and meeting at least one criterion.(1) Signs of renal damage: Albuminuria [Albumin excretion rate (AER)≥30mg/24h; Albumin to creatinine ratio (ACR)≥3mg/mmol]; abnormal urinary sediment; tubular pathology; histological anomalies; structural abnormities found in imaging; history of kidney transplantation.(2) Decline in glomerular filtration rate: eGFR≤60ml·min-1·1.73m-2",
            "output": {
                "Biomarker": [
                    "Albumin excretion rate (AER)",
                    "Albumin to creatinine ratio (ACR)",
                    "Glomerular filtration rate",
                    "eGFR"
                ]
            }
        },
        {
            "input": "Application of DPP-4 inhibitors in specific populations",
            "output": {
                "Biomarker": []
            }
        }
    ],
    "input": "Currently, all sulfonylurea drugs' leaflets list severe liver dysfunction as a contraindication. Alanine transaminase (ALT)> 3 times the upper limit of the reference value can serve as a sensitive and specific indicator of liver damage. If ALT>8-10 times the upper limit of the reference value or ALT>3 times with total serum bilirubin (TBIL)>2 times the reference value, it is considered a specific predictor of severe liver damage, indicating substantial injury to hepatic parenchymal cells; sulfonylureas should be contraindicated at this stage. Clinically, patients with decompensated liver cirrhosis accompanied by hepatic encephalopathy, ascites, or coagulation disorders should avoid this class of drugs to prevent hypoglycemia."
}

📚 詳細文檔

如需從輸出文本中提取結構化內容並進行評估，請參考 DeepKE-llm/InstructKGC/README_CN.md/7.評估。

如需繼續訓練OneKE，請參考 DeepKE-llm/InstructKGC/4.9領域內數據繼續訓練。

🔧 技術細節

OneKE主要專注於Schema可泛化的信息提取。由於現有提取指令數據存在格式不規範、數據噪聲和缺乏多樣性等問題，OneKE採用了提取指令的規範化和清理、困難負樣本收集以及基於Schema的批量指令構建等技術，如圖所示。如需更詳細的信息，請參考論文 "IEPile: Unearthing Large-Scale Schema-Based Information Extraction Corpus [Github]"。

OneKE與其他大模型的零樣本泛化比較結果如下：

• NER-en：CrossNER_AI、CrossNER_literature、CrossNER_music、CrossNER_politics、CrossNER_science
• NER-zh：WEIBONER、boson
• RE-zh：COAE2016、IPRE、SKE2020
• RE-en：FewRel、Wiki-ZSL
• EE-en：CrudeOilNews、WikiEvents、RAMS
• EE-zh：FewFC、CCF Law

📄 許可證

本項目採用 CC BY-NC-SA 4.0 許可證。

📚 引用

如果您在工作中使用了OneKE，請引用以下論文：

@article{DBLP:journals/corr/abs-2402-14710,
  author       = {Honghao Gui and
                  Lin Yuan and
                  Hongbin Ye and
                  Ningyu Zhang and
                  Mengshu Sun and
                  Lei Liang and
                  Huajun Chen},
  title        = {IEPile: Unearthing Large-Scale Schema-Based Information Extraction
                  Corpus},
  journal      = {CoRR},
  volume       = {abs/2402.14710},
  year         = {2024},
  url          = {https://doi.org/10.48550/arXiv.2402.14710},
  doi          = {10.48550/ARXIV.2402.14710},
  eprinttype    = {arXiv},
  eprint       = {2402.14710},
  timestamp    = {Tue, 09 Apr 2024 07:32:43 +0200},
  biburl       = {https://dblp.org/rec/journals/corr/abs-2402-14710.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}