Co-Instruct Open-Source Vision-Language Model - Free Deployment to Boost Image-Text Generation and Question Answering

Co Instruct

Developed by q-future

Co-Instruct is a vision-language model focused on image-to-text generation tasks, capable of analyzing image content and generating relevant textual descriptions or answering questions about images.

Image-to-Text

Transformers

#Image Quality Assessment #Multi-Image Comparative Analysis #Visual Instruction Understanding

Downloads 470

Release Time : 1/10/2024

Model Overview

This model combines visual and language processing capabilities to understand image content and generate corresponding textual output, suitable for tasks such as image quality assessment and content description.

Model Features

Image Quality Assessment

Capable of analyzing image quality, identifying issues such as motion blur, overexposure, or underexposure.

Multi-Image Comparison

Supports simultaneous analysis and comparison of multiple images, providing quality assessment and cause analysis.

Vision-Language Understanding

Combines visual and language processing capabilities to understand image content and generate relevant textual descriptions.

Model Capabilities

Image Content Analysis

Image Quality Assessment

Multi-Image Comparison

Text Generation

Use Cases

Image Quality Assessment

Single Image Quality Analysis

Analyze quality issues in a single image, such as motion blur, overexposure, or underexposure.

Identified motion blur in the image.

Dual Image Quality Comparison

Compare the quality of two images and analyze which is better and why.

The second image has better quality due to accurate exposure, precise focus, and clear details.

Image Content Description

Image Content Q&A

Answer specific questions about image content.

🚀 Co-Instruct Model

This project provides a model that can handle image - text - to - text tasks, leveraging specific datasets.

🚀 Quick Start

Datasets

q-future/Q-Instruct-DB
q-future/Co-Instruct-DB

Pipeline Tag

image-text-to-text

📰 News

See its paper: https://huggingface.co/papers/2402.16641

📦 Installation

Load Model

import torch
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("q-future/co-instruct", 
                                             trust_remote_code=True, 
                                             torch_dtype=torch.float16,
                                             attn_implementation="eager", 
                                             device_map={"":"cuda:0"})

💻 Usage Examples

Basic Usage

import requests
from PIL import Image

### Single Image
prompt = "USER: The image: <|image|> Which happens in this image: motion-blur, over-exposure, or under-exposure? ASSISTANT:"
url = "https://raw.githubusercontent.com/Q-Future/Q-Align/main/fig/singapore_flyer.jpg"
image = Image.open(requests.get(url,stream=True).raw)
model.chat(prompt, [image], max_new_tokens=200)

## Motion blur

Advanced Usage

### Double Image Comparison
prompt_cmp = "USER: The first image: <|image|>\nThe second image: <|image|>Which image has better quality, and why? ASSISTANT:"
url = "https://raw.githubusercontent.com/Q-Future/Q-Align/main/fig/boy_colorful.jpg"
image_2 = Image.open(requests.get(url,stream=True).raw)
model.chat(prompt_cmp, [image, image_2], max_new_tokens=200)

## The second image has better quality. The description indicates that the image has accurate exposure, precise focus, clear details, rich colors, and sufficient lighting. Additionally, the texture details are clear, and the composition is centered. In comparison, the first image has good clarity and rich texture details, but the lighting is slightly weak, which can affect the overall quality of the image. Therefore, the second image is of higher quality due to its accurate exposure, precise focus, clear details, rich colors, sufficient lighting, and centered composition.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご