🚀 CyberAgentLM2-7B (CALM2-7B)
CyberAgentLM2是一个仅解码器的语言模型,在1.3T公开可用的日语和英语数据集上进行了预训练。它能为日语和英语相关的自然语言处理任务提供强大支持。
🚀 快速开始
环境准备
确保你已经安装了以下依赖:
- transformers >= 4.34.1
- accelerate
代码示例
import transformers
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
assert transformers.__version__ >= "4.34.1"
model = AutoModelForCausalLM.from_pretrained("cyberagent/calm2-7b", device_map="auto", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("cyberagent/calm2-7b")
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
prompt = "AIによって私達の暮らしは、"
token_ids = tokenizer.encode(prompt, return_tensors="pt")
output_ids = model.generate(
input_ids=token_ids.to(model.device),
max_new_tokens=100,
do_sample=True,
temperature=0.9,
streamer=streamer,
)
✨ 主要特性
📦 安装指南
安装所需的依赖库:
pip install transformers>=4.34.1 accelerate
💻 使用示例
基础用法
import transformers
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
assert transformers.__version__ >= "4.34.1"
model = AutoModelForCausalLM.from_pretrained("cyberagent/calm2-7b", device_map="auto", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("cyberagent/calm2-7b")
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
prompt = "AIによって私達の暮らしは、"
token_ids = tokenizer.encode(prompt, return_tensors="pt")
output_ids = model.generate(
input_ids=token_ids.to(model.device),
max_new_tokens=100,
do_sample=True,
temperature=0.9,
streamer=streamer,
)
📚 详细文档
模型详情
属性 |
详情 |
模型大小 |
7B |
训练token数 |
1.3T tokens |
上下文长度 |
4096 |
模型类型 |
基于Transformer的语言模型 |
支持语言 |
日语、英语 |
开发者 |
CyberAgent, Inc. |
许可证 |
Apache-2.0 |
📄 许可证
本项目使用Apache-2.0许可证。
👨💻 作者
Ryosuke Ishigami
📚 引用
@article{touvron2023llama,
title={LLaMA: Open and Efficient Foundation Language Models},
author={Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timoth{\'e}e and Rozi{\`e}re, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and Rodriguez, Aurelien and Joulin, Armand and Grave, Edouard and Lample, Guillaume},
journal={arXiv preprint arXiv:2302.13971},
year={2023}
}