Internlm Xcomposer2 Vl 7b

🚀 InternLM-XComposer2

InternLM-XComposer2是一款基於InternLM2的視覺語言大模型（VLLM），可實現高級的文本 - 圖像理解與合成。

InternLM-XComposer2

我們發佈了兩個版本的InternLM-XComposer2系列模型：

• InternLM-XComposer2-VL：以InternLM2作為大語言模型（LLM）初始化的預訓練VLLM模型，在各種多模態基準測試中表現出色。
• InternLM-XComposer2：針對自由形式的文本 - 圖像交錯合成進行微調的VLLM。

🚀 快速開始

我們提供了一個簡單的示例，展示如何使用🤗 Transformers庫來使用InternLM-XComposer。

基礎用法

import torch
from transformers import AutoModel, AutoTokenizer

torch.set_grad_enabled(False)

# init model and tokenizer
model = AutoModel.from_pretrained('internlm/internlm-xcomposer2-vl-7b', trust_remote_code=True).cuda().eval()
tokenizer = AutoTokenizer.from_pretrained('internlm/internlm-xcomposer2-vl-7b', trust_remote_code=True)

query = '<ImageHere>Please describe this image in detail.'
image = './image1.webp'
with torch.cuda.amp.autocast():
  response, _ = model.chat(tokenizer, query=query, image=image, history=[], do_sample=False)
print(response)
#The image features a quote by Oscar Wilde, "Live life with no excuses, travel with no regret,"
# set against a backdrop of a breathtaking sunset. The sky is painted in hues of pink and orange,
# creating a serene atmosphere. Two silhouetted figures stand on a cliff, overlooking the horizon.
# They appear to be hiking or exploring, embodying the essence of the quote.
# The overall scene conveys a sense of adventure and freedom, encouraging viewers to embrace life without hesitation or regrets.

📦 安裝指南

從Transformers庫導入

要使用Transformers庫加載InternLM-XComposer2-VL-7B模型，請使用以下代碼：

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
ckpt_path = "internlm/internlm-xcomposer2-vl-7b"
tokenizer = AutoTokenizer.from_pretrained(ckpt_path, trust_remote_code=True).cuda()
# Set `torch_dtype=torch.float16` to load model in float16, otherwise it will be loaded as float32 and might cause OOM Error.
model = AutoModelForCausalLM.from_pretrained(ckpt_path, torch_dtype=torch.float16, trust_remote_code=True).cuda()
model = model.eval()

📄 許可證

代碼採用Apache 2.0許可證，而模型權重完全開放用於學術研究，也允許免費商業使用。如需申請商業許可證，請填寫申請表（英文/中文）。如有其他問題或合作需求，請聯繫 internlm@pjlab.org.cn。