Redpajama INCITE 7B Instruct

🚀 RedPajama-INCITE-7B-Instruct

RedPajama-INCITE-7B-Instruct 是由 Together 以及开源 AI 社区的领导者们共同开发的，这些领导者来自 Ontocord.ai、ETH DS3Lab、AAI CERC、蒙特利尔大学、魁北克人工智能研究所（MILA）、斯坦福基础模型研究中心（CRFM）、斯坦福 Hazy Research 研究小组和 LAION。该模型在 GPT-JT 的数据上针对少样本应用进行了微调，同时排除了与 HELM 核心场景重叠的任务。

🚀 快速开始

请注意，该模型需要 transformers 版本 >= 4.25.1。

GPU 推理

这需要一个具有 16GB 内存的 GPU。

import torch
import transformers
from transformers import AutoTokenizer, AutoModelForCausalLM

MIN_TRANSFORMERS_VERSION = '4.25.1'

# 检查 transformers 版本
assert transformers.__version__ >= MIN_TRANSFORMERS_VERSION, f'请将 transformers 升级到 {MIN_TRANSFORMERS_VERSION} 或更高版本。'

# 初始化
tokenizer = AutoTokenizer.from_pretrained("togethercomputer/RedPajama-INCITE-7B-Instruct")
model = AutoModelForCausalLM.from_pretrained("togethercomputer/RedPajama-INCITE-7B-Instruct", torch_dtype=torch.float16)
model = model.to('cuda:0')
# 推理
prompt = "Q: The capital of France is?\nA:"
inputs = tokenizer(prompt, return_tensors='pt').to(model.device)
input_length = inputs.input_ids.shape[1]
outputs = model.generate(
    **inputs, max_new_tokens=128, do_sample=True, temperature=0.7, top_p=0.7, top_k=50, return_dict_in_generate=True
)
token = outputs.sequences[0, input_length:]
output_str = tokenizer.decode(token)
print(output_str)
"""
Paris
"""

Int8 格式的 GPU 推理

这需要一个具有 12GB 内存的 GPU。

要使用 int8 进行推理，请确保你已经安装了 accelerate 和 bitandbytes。你可以使用以下命令安装它们：

pip install accelerate
pip install bitsandbytes

然后你可以按如下方式使用 int8 进行推理：

import torch
import transformers
from transformers import AutoTokenizer, AutoModelForCausalLM

MIN_TRANSFORMERS_VERSION = '4.25.1'

# 检查 transformers 版本
assert transformers.__version__ >= MIN_TRANSFORMERS_VERSION, f'请将 transformers 升级到 {MIN_TRANSFORMERS_VERSION} 或更高版本。'

# 初始化
tokenizer = AutoTokenizer.from_pretrained("togethercomputer/RedPajama-INCITE-7B-Instruct")
model = AutoModelForCausalLM.from_pretrained("togethercomputer/RedPajama-INCITE-7B-Instruct", device_map='auto', torch_dtype=torch.float16, load_in_8bit=True)

# 推理
prompt = "Q: The capital of France is?\nA:"
inputs = tokenizer(prompt, return_tensors='pt').to(model.device)
input_length = inputs.input_ids.shape[1]
outputs = model.generate(
    **inputs, max_new_tokens=128, do_sample=True, temperature=0.7, top_p=0.7, top_k=50, return_dict_in_generate=True
)
token = outputs.sequences[0, input_length:]
output_str = tokenizer.decode(token)
print(output_str)
"""
Paris
"""

CPU 推理

import torch
import transformers
from transformers import AutoTokenizer, AutoModelForCausalLM

MIN_TRANSFORMERS_VERSION = '4.25.1'

# 检查 transformers 版本
assert transformers.__version__ >= MIN_TRANSFORMERS_VERSION, f'请将 transformers 升级到 {MIN_TRANSFORMERS_VERSION} 或更高版本。'

# 初始化
tokenizer = AutoTokenizer.from_pretrained("togethercomputer/RedPajama-INCITE-7B-Instruct")
model = AutoModelForCausalLM.from_pretrained("togethercomputer/RedPajama-INCITE-7B-Instruct", torch_dtype=torch.bfloat16)
# 推理
prompt = "Q: The capital of France is?\nA:"
inputs = tokenizer(prompt, return_tensors='pt').to(model.device)
input_length = inputs.input_ids.shape[1]
outputs = model.generate(
    **inputs, max_new_tokens=128, do_sample=True, temperature=0.7, top_p=0.7, top_k=50, return_dict_in_generate=True
)
token = outputs.sequences[0, input_length:]
output_str = tokenizer.decode(token)
print(output_str)
"""
Paris
"""

请注意，由于 LayerNormKernelImpl 未针对 CPU 的 fp16 进行实现，因此我们在 CPU 推理时使用 bfloat16。

✨ 主要特性

模型详情

模型版本

• 基础模型：RedPajama-INCITE-7B-Base
• 指令微调版本：RedPajama-INCITE-7B-Instruct
• 聊天版本：RedPajama-INCITE-7B-Chat

示例展示

情感分析

Label the sentences as either 'positive', 'negative', 'mixed', or 'neutral': 

Sentence: I can say that there isn't anything I would change.
Label: positive

Sentence: I'm not sure about this.
Label: neutral

Sentence: I liked some parts but I didn't like other parts.
Label: mixed

Sentence: I think the background image could have been better.
Label: negative

Sentence: I really like it.
Label: positive

问答系统

Please answer the following question:

Question: What is the capital of Canada?
Answer: Ottawa

Question: What is the currency of Switzerland?
Answer: Swiss franc

Question: In which country is Wisconsin located?
Answer: United States

主题分类

Given a news article, classify its topic.

Possible labels: 1. World 2. Sports 3. Business 4. Sci/Tech

Article: A nearby star thought to harbor comets and asteroids now appears to be home to planets, too.
Label: Sci/Tech

Article: Soaring crude prices plus worries about the economy and the outlook for earnings are expected to hang over the stock market next week during the depth of the summer doldrums.
Label: Business

Article: Murtagh a stickler for success Northeastern field hockey coach Cheryl Murtagh doesn't want the glare of the spotlight that shines on her to detract from a team that has been the America East champion for the past three years and has been to the NCAA tournament 13 times.
Label: Sports

句子改写

Paraphrase the given sentence into a different sentence.

Input: Can you recommend some upscale restaurants in New York?
Output: What upscale restaurants do you recommend in New York?

Input: What are the famous places we should not miss in Paris?
Output: Recommend some of the best places to visit in Paris?

Input: Could you recommend some hotels that have cheap price in Zurich?
Output: Can you recommend some inexpensive hotels in Zurich?

文本摘要

Given a review from Amazon's food products, the task is to generate a short summary of the given review in the input.

Input: I have bought several of the Vitality canned dog food products and have found them all to be of good quality. The product looks more like a stew than a processed meat and it smells better. My Labrador is finicky and she appreciates this product better than most.
Output: Good Quality Dog Food

Input: Product arrived labeled as Jumbo Salted Peanuts...the peanuts were actually small sized unsalted. Not sure if this was an error or if the vendor intended to represent the product as 'Jumbo'.
Output: Not as Advertised

Input: My toddler loves this game to a point where he asks for it. That's a big thing for me. Secondly, no glitching unlike one of their competitors (PlayShifu). Any tech I don’t have to reach out to support for help is a good tech for me. I even enjoy some of the games and activities in this. Overall, this is a product that shows that the developers took their time and made sure people would not be asking for refund. I’ve become bias regarding this product and honestly I look forward to buying more of this company’s stuff. Please keep up the great work.
Output: Great Kids' Game

词义消歧

Identify which sense of a word is meant in a given context.

Context: The river overflowed the bank.
Word: bank
Sense: river bank

Context: A mouse takes much more room than a trackball.
Word: mouse
Sense: computer mouse

Context: The bank will not be accepting cash on Saturdays.
Word: bank
Sense: commercial (finance) banks

Context: Bill killed the project
Word: kill
Sense: terminate

自然语言推理

Given a pair of sentences, choose whether the two sentences agree (entailment)/disagree (contradiction) with each other.

Possible labels: 1. entailment 2. contradiction

Sentence 1: The skier was on the edge of the ramp. Sentence 2: The skier was dressed in winter clothes.
Label: entailment

Sentence 1: The boy skated down the staircase railing. Sentence 2: The boy is a newbie skater.
Label: contradiction

Sentence 1: Two middle-aged people stand by a golf hole. Sentence 2: A couple riding in a golf cart.
Label: contradiction

推理参数

📚 详细文档

使用方式

直接使用

以下是被排除的使用场景说明。

不当使用、恶意使用和超出范围的使用

最终用户有责任确保模型以负责任和符合道德的方式使用。

超出范围的使用

RedPajama-INCITE-7B-Instruct 是一个语言模型，对于其预期范围之外的其他用例可能表现不佳。例如，它可能不适用于安全关键型应用或用于做出对个人或社会有重大影响的决策。重要的是要考虑模型的局限性，仅将其用于预期目的。

不当使用和恶意使用

RedPajama-INCITE-7B-Instruct 是为语言建模而设计的。对模型的不当使用，例如使用它从事非法或不道德的活动，是严格禁止的，并且违反了项目的原则。

使用该模型生成对个人残酷的内容是对该模型的不当使用。这包括但不限于：

• 生成假新闻、错误信息或宣传内容
• 宣扬仇恨言论、歧视或对个人或群体的暴力行为
• 在未经同意的情况下冒充个人或组织
• 进行网络欺凌或骚扰
• 诽谤性内容
• 垃圾邮件或诈骗
• 在没有适当授权的情况下分享机密或敏感信息
• 违反模型或用于训练它的数据的使用条款
• 创建用于恶意目的的自动化机器人，如传播恶意软件、钓鱼诈骗或垃圾邮件

局限性

RedPajama-INCITE-7B-Instruct 与其他语言模型一样，有一些需要考虑的局限性。例如，该模型可能并不总是提供准确或相关的答案，特别是对于复杂、模糊或超出其训练数据范围的问题。因此，我们欢迎个人和组织的贡献，并鼓励合作以创建一个更强大和包容的聊天机器人。

训练信息

训练数据

请参考 togethercomputer/RedPajama-Data-1T。

训练过程

• 硬件：8 块 A100 GPU
• 优化器：Adam
• 梯度累积：1
• 令牌数量：10 亿个令牌
• 学习率：1e - 5

社区参与

加入我们的 Together Discord 社区。

📄 许可证

本项目采用 Apache 2.0 许可证。