Internlm Xcomposer2d5 7b Chat
InternLM-XComposer2.5-Chat is a dialogue model trained based on InternLM-XComposer2.5-7B, showing significant improvements in multimodal instruction following and open-ended dialogue capabilities.
Downloads 87
Release Time : 1/21/2025
Model Overview
This is a multimodal dialogue model that supports visual question answering and open-ended conversations, capable of understanding and analyzing images, video content, and engaging in natural language interactions.
Model Features
Multimodal Understanding Capability
Capable of processing and understanding images, videos, and text simultaneously
Video Content Analysis
Can analyze video frames, understanding actions and scenes within videos
High-Resolution Image Analysis
Capable of parsing detailed information in high-resolution images
Multi-Turn Dialogue Capability
Supports contextual understanding based on dialogue history
Model Capabilities
Video Content Understanding
Image Analysis
Multi-Turn Dialogue
Multimodal Instruction Following
Open-Ended Q&A
Use Cases
Content Analysis
Sports Video Analysis
Analyze sports match videos, identifying player actions and match outcomes
Can accurately identify player numbers, match results, and other key information
Vehicle Analysis
Compare the strengths and weaknesses of different vehicles
Can provide detailed analysis of various car models and their suitable scenarios
Information Extraction
Infographic Parsing
Extract structured data from complex infographics
Can accurately extract key data and facts from infographics
license: other pipeline_tag: visual-question-answering
InternLM-XComposer-2.5-Chat
InternLM-XComposer2.5-Chat is a chat model trained on internlm/internlm-xcomposer2d5-7b, offers improved multi-modal instruction following and open-ended dialogue capabilities.
Import from Transformers
To load the InternLM-XComposer2-2d5-Chat model using Transformers, use the following code:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
ckpt_path = "internlm/internlm-xcomposer2d5-7b-chat"
tokenizer = AutoTokenizer.from_pretrained(ckpt_path, trust_remote_code=True).cuda()
# Set `torch_dtype=torch.floatb16` to load model in bfloat16, otherwise it will be loaded as float32 and might cause OOM Error.
model = AutoModelForCausalLM.from_pretrained(ckpt_path, torch_dtype=torch.bfloat16, trust_remote_code=True).cuda()
model = model.eval()
Quickstart
We provide a simple example to show how to use InternLM-XComposer2.5 with 🤗 Transformers.
Video Understanding
import torch
from transformers import AutoModel, AutoTokenizer
torch.set_grad_enabled(False)
# init model and tokenizer
model = AutoModel.from_pretrained('internlm/internlm-xcomposer2d5-7b-chat', torch_dtype=torch.bfloat16, trust_remote_code=True).cuda().eval()
tokenizer = AutoTokenizer.from_pretrained('internlm/internlm-xcomposer2d5-7b-chat', trust_remote_code=True)
model.tokenizer = tokenizer
query = 'Here are some frames of a video. Describe this video in detail'
image = ['./examples/liuxiang.mp4',]
with torch.autocast(device_type='cuda', dtype=torch.float16):
response, his = model.chat(tokenizer, query, image, do_sample=False, num_beams=3, use_meta=True)
print(response)
# The video begins with a man in a red and yellow uniform standing on the starting line of a track, preparing to compete in the 110-meter hurdles at the Athens 2004 Olympic Games. He is identified as Liu Xiang, a Chinese athlete, and his bib number is 1363. The scene is set in a stadium filled with spectators, indicating the significance of the event.
# As the race begins, all the athletes start running, but Liu Xiang quickly takes the lead. However, he encounters a hurdle and knocks it over. Despite this setback, he quickly recovers and continues to run. The race is intense, with athletes from various countries competing fiercely. In the end, Liu Xiang emerges as the winner with a time of 12.91 seconds, securing the gold medal for China.
# The video then transitions to a slow-motion replay of the race, focusing on Liu Xiang's performance and the knockdown of the hurdle. This allows viewers to appreciate the skill and determination of the athlete.
# Following the race, Liu Xiang is seen lying on the track, possibly exhausted from the intense competition. He then stands up and begins to celebrate his victory, waving his arms in the air and running around the track. The crowd cheers and celebrates with him, creating a joyful atmosphere.
# The video concludes with a replay of Liu Xiang's gold medal-winning moment, emphasizing the significance of his achievement at the Athens 2004 Olympic Games.
# Throughout the video, the Olympic logo is prominently displayed, reminding viewers of the global significance of the event and the athletes' dedication and perseverance in their pursuit of victory.
query = 'tell me the athlete code of Liu Xiang'
image = ['./examples/liuxiang.mp4',]
with torch.autocast(device_type='cuda', dtype=torch.float16):
response, _ = model.chat(tokenizer, query, image, history=his, do_sample=False, num_beams=3, use_meta=True)
print(response)
# The athlete code of Liu Xiang is 1363.
Multi-Image Mutli-Tune Dialog
import torch
from transformers import AutoModel, AutoTokenizer
torch.set_grad_enabled(False)
# init model and tokenizer
model = AutoModel.from_pretrained('internlm/internlm-xcomposer2d5-7b-chat', torch_dtype=torch.bfloat16, trust_remote_code=True).cuda().eval()
tokenizer = AutoTokenizer.from_pretrained('internlm/internlm-xcomposer2d5-7b-chat', trust_remote_code=True)
model.tokenizer = tokenizer
query = 'Image1 <ImageHere>; Image2 <ImageHere>; Image3 <ImageHere>; I want to buy a car from the three given cars, analyze their advantages and weaknesses one by one'
image = ['./examples/cars1.jpg',
'./examples/cars2.jpg',
'./examples/cars3.jpg',]
with torch.autocast(device_type='cuda', dtype=torch.float16):
response, his = model.chat(tokenizer, query, image, do_sample=False, num_beams=3, use_meta=True)
print(response)
# Certainly! Let's analyze the advantages and disadvantages of each car:
#
# 1. **Mercedes-Benz G-Class (SUV)**:
# - **Advantages**:
# - **Off-Road Capability**: The G-Class is renowned for its off-road prowess. It can handle a variety of terrains, making it ideal for adventurous driving.
# - **Reliability**: Mercedes-Benz is known for producing reliable vehicles, and the G-Class is no exception. It has a reputation for durability and longevity.
# - **Luxury Features**: As a Mercedes-Benz, the G-Class comes with a host of luxury features, including high-quality materials and advanced technology.
# - **Disadvantages**:
# - **Fuel Efficiency**: The G-Class is not known for its fuel efficiency. It consumes a significant amount of gasoline, which can be a disadvantage for those concerned with fuel economy.
# - **Size and Weight**: The G-Class is large and heavy, which can affect its handling and maneuverability, especially in urban environments.
# - **Cost**: The G-Class is generally more expensive compared to other SUVs, which can be a deterrent for some buyers.
#
# 2. **Bugatti Chiron (Sports Car)**:
# - **Advantages**:
# - **Performance**: The Bugatti Chiron is one of the fastest production cars available. It boasts impressive acceleration and top speed, making it a thrilling driving experience.
# - **Design**: The Chiron has a sleek and futuristic design that is both aesthetically pleasing and aerodynamically efficient.
# - **Status Symbol**: Owning a Bugatti is often seen as a status symbol, attracting attention and admiration.
# - **Disadvantages**:
# - **Cost**: The Bugatti Chiron is one of the most expensive cars in the world, making it out of reach for many potential buyers.
# - **Maintenance**: Bugatti vehicles require specialized maintenance, which can be costly and difficult to find.
# - **Road Legality**: In some regions, the Chiron may not be road-legal due to its high top speed and powerful engine.
#
# 3. **BMW i Vision Dynamics (Electric Concept Car)**:
# - **Advantages**:
# - **Environmental Impact**: As an electric vehicle, the i Vision Dynamics has a lower environmental impact compared to traditional gasoline-powered cars.
# - **Design**: The i Vision Dynamics has a unique and futuristic design that sets it apart from conventional cars.
# - **Performance**: Despite being an electric vehicle, the i Vision Dynamics boasts impressive performance, including a top speed of over 300 km/h.
# - **Disadvantages**:
# - **Cost**: Like the Bugatti Chiron, the i Vision Dynamics is an expensive concept car, making it inaccessible to most buyers.
# - **Range**: Electric vehicles typically have a limited range compared to gasoline-powered cars, which can be a concern for long-distance travel.
# - **Infrastructure**: The availability of charging stations can vary widely by region, which can limit the practicality of owning an electric car.
# In summary, the Mercedes-Benz G-Class is a reliable and capable off-road vehicle with luxury features, but it may not be the most fuel-efficient or affordable option. The Bugatti Chiron offers unparalleled performance and status, but it is extremely expensive and requires specialized maintenance. The BMW i Vision Dynamics is an innovative and environmentally friendly concept car with impressive performance, but it is also costly and limited by range and charging infrastructure. Each car has its own unique set of advantages and disadvantages, and the choice ultimately depends on individual preferences and priorities.
query = 'Image4 <ImageHere>; How about the car in Image4'
image.append('./examples/cars4.jpg')
with torch.autocast(device_type='cuda', dtype=torch.float16):
response, _ = model.chat(tokenizer, query, image, do_sample=False, num_beams=3, history= his, use_meta=True)
print(response)
# The car in Image4 is a **Ferrari LaFerrari**. The LaFerrari is a hybrid supercar produced by Ferrari, combining a V12 engine with an electric motor. Here's a detailed analysis of its advantages and disadvantages:
#
# ### **Advantages:**
# 1. **Performance**: The LaFerrari is one of the fastest production cars available, with a top speed of over 217 mph (349 km/h). It boasts impressive acceleration and handling, making it a thrilling driving experience.
# 2. **Design**: The LaFerrari has a distinctive and aggressive design that sets it apart from other supercars. Its aerodynamic features and sleek lines contribute to its performance and visual appeal.
# 3. **Hybrid Technology**: The LaFerrari uses a hybrid powertrain, combining a 6.3-liter V12 engine with an electric motor. This hybrid system provides a balance of power and efficiency, reducing emissions compared to traditional gasoline engines.
# 4. **Status Symbol**: Owning a LaFerrari is often seen as a status symbol, attracting attention and admiration. It represents a pinnacle of automotive engineering and luxury.
# 5. **Reliability**: Ferrari is known for producing high-quality, reliable vehicles. The LaFerrari benefits from the brand's reputation for excellence in engineering and craftsmanship.
### **Disadvantages:**
# 1. **Cost**: The LaFerrari is one of the most expensive cars in the world, making it inaccessible to most potential buyers. Its high price can be a significant deterrent.
# 2. **Maintenance**: Ferrari vehicles require specialized maintenance, which can be costly and difficult to find. The hybrid system may also add to the complexity and expense of servicing the car.
# 3. **Road Legality**: In some regions, the LaFerrari may not be road-legal due to its high top speed and powerful engine. This can limit its usability and appeal.
# 4. **Fuel Efficiency**: Despite the hybrid system, the LaFerrari consumes a significant amount of fuel, which can be a disadvantage for those concerned with fuel economy.
# 5. **Size and Weight**: The LaFerrari is a large and heavy vehicle, which can affect its handling and maneuverability, especially in urban environments.
# In summary, the Ferrari LaFerrari is a high-performance hybrid supercar with a distinctive design and impressive capabilities. However, its high cost, specialized maintenance requirements, and limited road legality can be significant disadvantages for some buyers. The LaFerrari is best suited for those who prioritize performance, luxury, and status over practicality and affordability.
High Resolution Image Understanding
import torch
from transformers import AutoModel, AutoTokenizer
torch.set_grad_enabled(False)
# init model and tokenizer
model = AutoModel.from_pretrained('internlm/internlm-xcomposer2d5-7b-chat', torch_dtype=torch.bfloat16, trust_remote_code=True).cuda().eval()
tokenizer = AutoTokenizer.from_pretrained('internlm/internlm-xcomposer2d5-7b-chat', trust_remote_code=True)
model.tokenizer = tokenizer
query = 'Analyze the given image in a detail manner'
image = ['./examples/dubai.png']
with torch.autocast(device_type='cuda', dtype=torch.float16):
response, _ = model.chat(tokenizer, query, image, do_sample=False, num_beams=3, use_meta=True)
print(response)
# The image is an infographic titled "Amazing Facts About Dubai." Here's a detailed analysis of its content:
#
# 1. **Title and Introduction:**
# - The title is prominently displayed at the top of the image in bold, blue letters.
# - The image features a colorful skyline of Dubai, highlighting the city's modern architecture.
#
# 2. **Facts About Palm Jumeirah:**
# - Palm Jumeirah is the largest artificial island and is visible from space.
# - In 1968, there were only 1.5 million cars in Dubai.
#
# 3. **Dubai's Gold Chain:**
# - Dubai has the world's largest Gold Chain, which is 4.2 km long.
# - 7 out of the 10 tallest hotels in the world are located in Dubai.
#
# 4. **Crime Rate and Income Tax:**
# - The crime rate is near 0%.
# - The income tax rate is 0%.
#
# 5. **Dubai Mall:**
# - Dubai Mall is the largest shopping mall in the world with 1200 stores.
# - 17% of the population is Emirati, and 83% are immigrants.
#
# 6. **Dubai's Address System:**
# - Dubai has no standard address system, with no zip codes, area codes, or postal services.
#
# 7. **Dispense Gold:**
# - Dubai is building a climate-controlled City, 2.25 times as big as Monaco.
# - The Royal Suite at Burj Al Arab is $24,000 per night.
#
# 8. **License and Billionaires:**
# - You need a license to drink alcohol even at home.
# - The net worth of the four listed billionaires is roughly equal to the GDP of Honduras.
#
# 9. **Sources:**
# - The infographic cites sources from Wikipedia, Forbes, Gulf News, and The Guardian.
#
# 10. **Design and Compilation:**
# - The image is designed and compiled by FMEXtensions, a company based in the United Arab Emirates.
#
# The infographic uses a combination of text, icons, and images to convey interesting facts about Dubai, emphasizing its modernity, wealth, and unique features.
Open Source License
The code is licensed under Apache-2.0, while model weights are fully open for academic research and also allow free commercial usage. To apply for a commercial license, please fill in the application form (English)/申请表(中文). For other questions or collaborations, please contact internlm@pjlab.org.cn.
Clip Vit Large Patch14 336
A large-scale vision-language pretrained model based on the Vision Transformer architecture, supporting cross-modal understanding between images and text
Text-to-Image
Transformers

C
openai
5.9M
241
Fashion Clip
MIT
FashionCLIP is a vision-language model fine-tuned specifically for the fashion domain based on CLIP, capable of generating universal product representations.
Text-to-Image
Transformers English

F
patrickjohncyh
3.8M
222
Gemma 3 1b It
Gemma 3 is a lightweight advanced open model series launched by Google, built on the same research and technology as the Gemini models. This model is multimodal, capable of processing both text and image inputs to generate text outputs.
Text-to-Image
Transformers

G
google
2.1M
347
Blip Vqa Base
Bsd-3-clause
BLIP is a unified vision-language pretraining framework, excelling in visual question answering tasks through joint language-image training to achieve multimodal understanding and generation capabilities
Text-to-Image
Transformers

B
Salesforce
1.9M
154
CLIP ViT H 14 Laion2b S32b B79k
MIT
A vision-language model trained on the LAION-2B English dataset based on the OpenCLIP framework, supporting zero-shot image classification and cross-modal retrieval tasks
Text-to-Image
Safetensors
C
laion
1.8M
368
CLIP ViT B 32 Laion2b S34b B79k
MIT
A vision-language model trained on the English subset of LAION-2B using the OpenCLIP framework, supporting zero-shot image classification and cross-modal retrieval
Text-to-Image
Safetensors
C
laion
1.1M
112
Pickscore V1
PickScore v1 is a scoring function for text-to-image generation, used to predict human preferences, evaluate model performance, and rank images.
Text-to-Image
Transformers

P
yuvalkirstain
1.1M
44
Owlv2 Base Patch16 Ensemble
Apache-2.0
OWLv2 is a zero-shot text-conditioned object detection model that can localize objects in images through text queries.
Text-to-Image
Transformers

O
google
932.80k
99
Llama 3.2 11B Vision Instruct
Llama 3.2 is a multilingual, multimodal large language model released by Meta, supporting image-to-text and text-to-text conversion tasks with robust cross-modal understanding capabilities.
Text-to-Image
Transformers Supports Multiple Languages

L
meta-llama
784.19k
1,424
Owlvit Base Patch32
Apache-2.0
OWL-ViT is a zero-shot text-conditioned object detection model that can search for objects in images via text queries without requiring category-specific training data.
Text-to-Image
Transformers

O
google
764.95k
129
Featured Recommended AI Models