Internlm Xcomposer2 7b 4bit

🚀 InternLM-XComposer2

InternLM-XComposer2 是一款基於 InternLM2 的視覺語言大模型（VLLM），具備先進的圖文理解與組合能力。

InternLM-XComposer2

我們發佈了兩個版本的 InternLM-XComposer2 系列模型：

• InternLM-XComposer2-VL：以 InternLM2 作為大語言模型（LLM）初始化的預訓練 VLLM 模型，在各種多模態基準測試中表現出色。
• InternLM-XComposer2：針對 自由交錯圖文組合 進行微調的 VLLM 模型。

這是 InternLM-XComposer2 的 4 位版本，使用前請安裝最新版本的 auto_gptq。

🚀 快速開始

這是 InternLM-XComposer2 的 4 位版本，使用前請安裝最新版本的 auto_gptq。

💻 使用示例

基礎用法

import torch, auto_gptq
from PIL import Image
from transformers import AutoModel, AutoTokenizer 
from auto_gptq.modeling import BaseGPTQForCausalLM

auto_gptq.modeling._base.SUPPORTED_MODELS = ["internlm"]
torch.set_grad_enabled(False)

class InternLMXComposer2QForCausalLM(BaseGPTQForCausalLM):
    layers_block_name = "model.layers"
    outside_layer_modules = [
        'vit', 'vision_proj', 'model.tok_embeddings', 'model.norm', 'output', 
    ]
    inside_layer_modules = [
        ["attention.wqkv.linear"],
        ["attention.wo.linear"],
        ["feed_forward.w1.linear", "feed_forward.w3.linear"],
        ["feed_forward.w2.linear"],
    ]
 
# init model and tokenizer
model = InternLMXComposer2QForCausalLM.from_quantized(
  'internlm/internlm-xcomposer2-7b-4bit', trust_remote_code=True, device="cuda:0").eval()
tokenizer = AutoTokenizer.from_pretrained(
  'internlm/internlm-xcomposer2-7b-4bit', trust_remote_code=True)

img_path_list = [
    'panda.jpg',
    'bamboo.jpeg',
]
images = []
for img_path in img_path_list:
    image = Image.open(img_path).convert("RGB")
    image = model.vis_processor(image)
    images.append(image)
image = torch.stack(images)
query = '<ImageHere> <ImageHere>please write an article based on the images. Title: my favorite animal.'
with torch.cuda.amp.autocast():
    response, history = model.chat(tokenizer, query=query, image=image, history=[], do_sample=False)
print(response)

#My Favorite Animal: The Panda
#The panda, also known as the giant panda, is one of the most beloved animals in the world. These adorable creatures are native to China and can be found in the wild in a few select locations, but they are more commonly seen in captivity at zoos or wildlife reserves.
#Pandas have a distinct black-and-white coloration that makes them instantly recognizable. They are known for their love of bamboo, which they eat almost exclusively. In fact, pandas spend up to 14 hours a day eating, with the majority of their diet consisting of bamboo. Despite this seemingly unbalanced diet, pandas are actually quite healthy and have a low body fat percentage, thanks to their ability to digest bamboo efficiently.
#In addition to their unique eating habits, pandas are also known for their playful personalities. They are intelligent and curious creatures, often engaging in activities like playing with toys or climbing trees. However, they do not typically exhibit these behaviors in the wild, where they are solitary creatures who prefer to spend their time alone.
#One of the biggest threats to the panda's survival is habitat loss due to deforestation. As a result, many pandas now live in captivity, where they are cared for by dedicated staff and provided with enrichment opportunities to keep them engaged and stimulated. While it is important to protect these animals from extinction, it is also crucial to remember that they are still wild creatures and should be treated with respect and care.
#Overall, the panda is an amazing animal that has captured the hearts of people around the world. Whether you see them in the wild or in captivity, there is no denying the charm and allure of these gentle giants.

📄 許可證

代碼遵循 Apache 2.0 許可證，而模型權重完全開放用於學術研究，也允許免費商業使用。如需申請商業許可證，請填寫申請表（英文）/申請表（中文）。如有其他問題或合作需求，請聯繫 internlm@pjlab.org.cn。