llava-1.5-13b-hf-i1-GGUF Open Source Model - Multiple quantization types adapt to the usage requirements of different scenarios

Llava 1.5 13b Hf I1 GGUF

Developed by mradermacher

This project provides weighted/matrix quantized versions of the llava-1.5-13b-hf model, including various quantization types to meet the usage requirements in different scenarios.

Text-to-Image

Transformers

English#Multimodal Vision-Language #Efficient Quantized Inference #Low-Resource Deployment

Downloads 332

Release Time : 4/25/2025

Model Overview

The quantized model of llava-1.5-13b-hf is an optimized version of the original model. It reduces the model size and improves the inference efficiency through quantization technology while maintaining good model performance.

Model Features

Multiple Quantized Versions

Provide multiple quantized versions ranging from 3.0GB to 10.8GB to meet the usage requirements under different hardware conditions

Efficient Inference

Optimize the model through quantization technology to improve the inference speed and reduce the hardware requirements

Flexible Quality Selection

Provide multiple quantization options from emergency use to high-quality inference, allowing users to balance quality and performance according to their needs

Model Capabilities

Multimodal Understanding

Image-Text Interaction

Visual Question Answering

Image Description Generation

Use Cases

Computer Vision Applications

Image Understanding and Analysis

Understand and analyze the content of the input image

Generate accurate image descriptions and relevant information

Visual Question Answering System

A question answering system based on image content

Provide accurate answers related to the image content

Edge Device Deployment

Mobile Vision Applications

Deploy vision understanding functions on mobile devices with limited resources

Implement efficient local image processing

🚀 LLava-1.5-13B-HF Quantized Model

This project provides weighted/imatrix quants of the llava-1.5-13b-hf model. It offers various quantized versions for different usage scenarios.

🚀 Quick Start

If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files.

✨ Features

Provides weighted/imatrix quants of the llava-1.5-13b-hf model.
Offers a variety of quantized versions with different sizes and qualities.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

No code examples are provided in the original document.

📚 Documentation

About

Weighted/imatrix quants of https://huggingface.co/llava-hf/llava-1.5-13b-hf. Static quants are available at https://huggingface.co/mradermacher/llava-1.5-13b-hf-GGUF.

Provided Quants

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Link	Type	Size/GB	Notes
GGUF	i1-IQ1_S	3.0	for the desperate
GGUF	i1-IQ1_M	3.2	mostly desperate
GGUF	i1-IQ2_XXS	3.6
GGUF	i1-IQ2_XS	4.0
GGUF	i1-IQ2_S	4.3
GGUF	i1-Q2_K_S	4.5	very low quality
GGUF	i1-IQ2_M	4.6
GGUF	i1-Q2_K	5.0	IQ3_XXS probably better
GGUF	i1-IQ3_XXS	5.1	lower quality
GGUF	i1-IQ3_XS	5.5
GGUF	i1-IQ3_S	5.8	beats Q3_K*
GGUF	i1-Q3_K_S	5.8	IQ3_XS probably better
GGUF	i1-IQ3_M	6.1
GGUF	i1-Q3_K_M	6.4	IQ3_S probably better
GGUF	i1-Q3_K_L	7.0	IQ3_M probably better
GGUF	i1-IQ4_XS	7.1
GGUF	i1-IQ4_NL	7.5	prefer IQ4_XS
GGUF	i1-Q4_0	7.5	fast, low quality
GGUF	i1-Q4_K_S	7.5	optimal size/speed/quality
GGUF	i1-Q4_K_M	8.0	fast, recommended
GGUF	i1-Q4_1	8.3
GGUF	i1-Q5_K_S	9.1
GGUF	i1-Q5_K_M	9.3
GGUF	i1-Q6_K	10.8	practically like static Q6_K

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.

Thanks

I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

🔧 Technical Details

No technical details are provided in the original document.

📄 License

The license of this project is llama2.

Property	Details
Model Type	Weighted/imatrix quants of `llava-1.5-13b-hf`
Training Data	Not provided
License	llama2
Quantized By	mradermacher
Tags	vision, image-text-to-text
Arxiv	2304.08485
Base Model	llava-hf/llava-1.5-13b-hf
Language	en
Library Name	transformers

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご