Gollie 7B

🚀 GoLLIE 7B：遵循指南的信息提取大語言模型

GoLLIE是一個經過訓練以遵循註釋指南的大語言模型。它在零樣本信息提取方面優於以往的方法，允許用戶使用即時定義的註釋模式進行推理。與以往的方法不同，GoLLIE能夠遵循詳細的定義，而不僅僅依賴於大語言模型中已編碼的知識。

🚀 快速開始

若要開始使用GoLLIE，請閱讀我們的 🚀 示例Jupyter筆記本。

加載模型的最佳方式是使用我們自定義的 load_model 函數。不過，你也可以使用 AutoModelForCausalLM 類來加載。

⚠️ 重要提示

我們的閃存注意力實現與Huggingface中的注意力實現存在細微的數值差異。你必須使用 trust_remote_code=True 標誌，否則結果會較差。閃存注意力需要可用的CUDA GPU，不支持在CPU上運行GOLLIE預訓練模型，我們計劃在未來版本中解決這個問題。首先，安裝閃存注意力2：

pip install flash-attn --no-build-isolation
pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary

然後你可以使用以下代碼加載模型：

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("HiTZ/GoLLIE-7B")
model = AutoModelForCausalLM.from_pretrained("HiTZ/GoLLIE-7B", trust_remote_code=True, torch_dtype=torch.bfloat16)
model.to("cuda")

閱讀我們的 🚀 示例Jupyter筆記本，瞭解如何輕鬆定義指南、生成模型輸入並解析輸出！

✨ 主要特性

• 能夠遵循註釋指南，在零樣本信息提取任務中表現出色。
• 允許用戶使用即時定義的註釋模式進行推理。
• 不侷限於大語言模型中已編碼的知識，可遵循詳細定義。

📦 安裝指南

首先，安裝閃存注意力2：

pip install flash-attn --no-build-isolation
pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary

💻 使用示例

基礎用法

標籤以Python類的形式表示，指南或說明以文檔字符串的形式引入。模型在 result = [ 行之後開始生成。

# Entity definitions
@dataclass
class Launcher(Template):
    """Refers to a vehicle designed primarily to transport payloads from the Earth's 
    surface to space. Launchers can carry various payloads, including satellites, 
    crewed spacecraft, and cargo, into various orbits or even beyond Earth's orbit. 
    They are usually multi-stage vehicles that use rocket engines for propulsion."""

    mention: str  
    """
    The name of the launcher vehicle. 
    Such as: "Sturn V", "Atlas V", "Soyuz", "Ariane 5"
    """
    space_company: str # The company that operates the launcher. Such as: "Blue origin", "ESA", "Boeing", "ISRO", "Northrop Grumman", "Arianespace"
    crew: List[str] # Names of the crew members boarding the Launcher. Such as: "Neil Armstrong", "Michael Collins", "Buzz Aldrin"
    

@dataclass
class Mission(Template):
    """Any planned or accomplished journey beyond Earth's atmosphere with specific objectives, 
    either crewed or uncrewed. It includes missions to satellites, the International 
    Space Station (ISS), other celestial bodies, and deep space."""
    
    mention: str
    """
    The name of the mission. 
    Such as: "Apollo 11", "Artemis", "Mercury"
    """
    date: str # The start date of the mission
    departure: str # The place from which the vehicle will be launched. Such as: "Florida", "Houston", "French Guiana"
    destination: str # The place or planet to which the launcher will be sent. Such as "Moon", "low-orbit", "Saturn"

# This is the text to analyze
text = (
    "The Ares 3 mission to Mars is scheduled for 2032. The Starship rocket build by SpaceX will take off from Boca Chica,"
    "carrying the astronauts Max Rutherford, Elena Soto, and Jake Martinez."
)

# The annotation instances that take place in the text above are listed here
result = [
    Mission(mention='Ares 3', date='2032', departure='Boca Chica', destination='Mars'),
    Launcher(mention='Starship', space_company='SpaceX', crew=['Max Rutherford', 'Elena Soto', 'Jake Martinez'])
]

📚 詳細文檔

模型信息

訓練數據

以下是用於訓練和評估GoLLIE的任務列表。不過，正如在 🚀 創建自定義任務筆記本 中所展示的，GoLLIE可以執行各種未見任務。更多信息請閱讀我們的 📖 論文。

評估結果

環境影響

🔧 技術細節

GoLLIE是由 Oscar Sainz、[Iker García - Ferrero](https://ikergarcia1996.github.io/Iker - Garcia - Ferrero/)、Rodrigo Agerri、Oier Lopez de Lacalle、German Rigau 和 Eneko Agirre 開發，由 HiTZ巴斯克語言技術中心 - Ixa、[巴斯克大學UPV/EHU](https://www.ehu.eus/en/en - home) 支持。

📄 許可證

基礎模型和合並模型使用LLaMA2許可證，預訓練LoRA適配器使用Apache 2.0許可證。

📖 引用

@misc{sainz2023gollie,
      title={GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction}, 
      author={Oscar Sainz and Iker García-Ferrero and Rodrigo Agerri and Oier Lopez de Lacalle and German Rigau and Eneko Agirre},
      year={2023},
      eprint={2310.03668},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

其他資源

• 💻 代碼：https://github.com/osainz59/CoLLIE/
• 📒 博客文章：GoLLIE: 遵循指南的信息提取大語言模型
• 🐕 GoLLIE在🤗HuggingFace Hub上的集合：[HiTZ/gollie](https://huggingface.co/collections/HiTZ/gollie - 651bf19ee315e8a224aacc4f)