Gollie 7B

🚀 GoLLIE 7B：遵循指南的信息提取大语言模型

GoLLIE是一个经过训练以遵循注释指南的大语言模型。它在零样本信息提取方面优于以往的方法，允许用户使用即时定义的注释模式进行推理。与以往的方法不同，GoLLIE能够遵循详细的定义，而不仅仅依赖于大语言模型中已编码的知识。

🚀 快速开始

若要开始使用GoLLIE，请阅读我们的 🚀 示例Jupyter笔记本。

加载模型的最佳方式是使用我们自定义的 load_model 函数。不过，你也可以使用 AutoModelForCausalLM 类来加载。

⚠️ 重要提示

我们的闪存注意力实现与Huggingface中的注意力实现存在细微的数值差异。你必须使用 trust_remote_code=True 标志，否则结果会较差。闪存注意力需要可用的CUDA GPU，不支持在CPU上运行GOLLIE预训练模型，我们计划在未来版本中解决这个问题。首先，安装闪存注意力2：

pip install flash-attn --no-build-isolation
pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary

然后你可以使用以下代码加载模型：

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("HiTZ/GoLLIE-7B")
model = AutoModelForCausalLM.from_pretrained("HiTZ/GoLLIE-7B", trust_remote_code=True, torch_dtype=torch.bfloat16)
model.to("cuda")

阅读我们的 🚀 示例Jupyter笔记本，了解如何轻松定义指南、生成模型输入并解析输出！

✨ 主要特性

• 能够遵循注释指南，在零样本信息提取任务中表现出色。
• 允许用户使用即时定义的注释模式进行推理。
• 不局限于大语言模型中已编码的知识，可遵循详细定义。

📦 安装指南

首先，安装闪存注意力2：

pip install flash-attn --no-build-isolation
pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary

💻 使用示例

基础用法

标签以Python类的形式表示，指南或说明以文档字符串的形式引入。模型在 result = [ 行之后开始生成。

# Entity definitions
@dataclass
class Launcher(Template):
    """Refers to a vehicle designed primarily to transport payloads from the Earth's 
    surface to space. Launchers can carry various payloads, including satellites, 
    crewed spacecraft, and cargo, into various orbits or even beyond Earth's orbit. 
    They are usually multi-stage vehicles that use rocket engines for propulsion."""

    mention: str  
    """
    The name of the launcher vehicle. 
    Such as: "Sturn V", "Atlas V", "Soyuz", "Ariane 5"
    """
    space_company: str # The company that operates the launcher. Such as: "Blue origin", "ESA", "Boeing", "ISRO", "Northrop Grumman", "Arianespace"
    crew: List[str] # Names of the crew members boarding the Launcher. Such as: "Neil Armstrong", "Michael Collins", "Buzz Aldrin"
    

@dataclass
class Mission(Template):
    """Any planned or accomplished journey beyond Earth's atmosphere with specific objectives, 
    either crewed or uncrewed. It includes missions to satellites, the International 
    Space Station (ISS), other celestial bodies, and deep space."""
    
    mention: str
    """
    The name of the mission. 
    Such as: "Apollo 11", "Artemis", "Mercury"
    """
    date: str # The start date of the mission
    departure: str # The place from which the vehicle will be launched. Such as: "Florida", "Houston", "French Guiana"
    destination: str # The place or planet to which the launcher will be sent. Such as "Moon", "low-orbit", "Saturn"

# This is the text to analyze
text = (
    "The Ares 3 mission to Mars is scheduled for 2032. The Starship rocket build by SpaceX will take off from Boca Chica,"
    "carrying the astronauts Max Rutherford, Elena Soto, and Jake Martinez."
)

# The annotation instances that take place in the text above are listed here
result = [
    Mission(mention='Ares 3', date='2032', departure='Boca Chica', destination='Mars'),
    Launcher(mention='Starship', space_company='SpaceX', crew=['Max Rutherford', 'Elena Soto', 'Jake Martinez'])
]

📚 详细文档

模型信息

训练数据

以下是用于训练和评估GoLLIE的任务列表。不过，正如在 🚀 创建自定义任务笔记本 中所展示的，GoLLIE可以执行各种未见任务。更多信息请阅读我们的 📖 论文。

评估结果

环境影响

🔧 技术细节

GoLLIE是由 Oscar Sainz、[Iker García - Ferrero](https://ikergarcia1996.github.io/Iker - Garcia - Ferrero/)、Rodrigo Agerri、Oier Lopez de Lacalle、German Rigau 和 Eneko Agirre 开发，由 HiTZ巴斯克语言技术中心 - Ixa、[巴斯克大学UPV/EHU](https://www.ehu.eus/en/en - home) 支持。

📄 许可证

基础模型和合并模型使用LLaMA2许可证，预训练LoRA适配器使用Apache 2.0许可证。

📖 引用

@misc{sainz2023gollie,
      title={GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction}, 
      author={Oscar Sainz and Iker García-Ferrero and Rodrigo Agerri and Oier Lopez de Lacalle and German Rigau and Eneko Agirre},
      year={2023},
      eprint={2310.03668},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

其他资源

• 💻 代码：https://github.com/osainz59/CoLLIE/
• 📒 博客文章：GoLLIE: 遵循指南的信息提取大语言模型
• 🐕 GoLLIE在🤗HuggingFace Hub上的集合：[HiTZ/gollie](https://huggingface.co/collections/HiTZ/gollie - 651bf19ee315e8a224aacc4f)