FLUX.1 Dev Qint4

🚀 black-forest-labs/FLUX.1-dev

本項目將 black-forest-labs/FLUX.1-dev 使用 Optimum Quanto 量化為 INT4，可用於文本到圖像的生成。

🚀 快速開始

本項目將 black-forest-labs/FLUX.1-dev 量化為 INT4，使用 Optimum Quanto 工具實現。

📦 安裝指南

在使用之前，需要安裝必要的依賴庫，可使用以下命令進行安裝：

pip install diffusers optimum-quanto

💻 使用示例

基礎用法

以下是一個完整的示例代碼，展示瞭如何加載量化後的模型並生成圖像：

import json
import torch
import diffusers
import transformers
from optimum.quanto import requantize
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download


def load_quanto_transformer(repo_path):
    with open(hf_hub_download(repo_path, "transformer/quantization_map.json"), "r") as f:
        quantization_map = json.load(f)
    with torch.device("meta"):
        transformer = diffusers.FluxTransformer2DModel.from_config(hf_hub_download(repo_path, "transformer/config.json")).to(torch.bfloat16)
    state_dict = load_file(hf_hub_download(repo_path, "transformer/diffusion_pytorch_model.safetensors"))
    requantize(transformer, state_dict, quantization_map, device=torch.device("cuda"))
    return transformer


def load_quanto_text_encoder_2(repo_path):
    with open(hf_hub_download(repo_path, "text_encoder_2/quantization_map.json"), "r") as f:
        quantization_map = json.load(f)
    with open(hf_hub_download(repo_path, "text_encoder_2/config.json")) as f:
        t5_config = transformers.T5Config(**json.load(f))
    with torch.device("meta"):
        text_encoder_2 = transformers.T5EncoderModel(t5_config).to(torch.bfloat16)
    state_dict = load_file(hf_hub_download(repo_path, "text_encoder_2/model.safetensors"))
    requantize(text_encoder_2, state_dict, quantization_map, device=torch.device("cuda"))
    return text_encoder_2


pipe = diffusers.AutoPipelineForText2Image.from_pretrained("Disty0/FLUX.1-dev-qint4", transformer=None, text_encoder_2=None, torch_dtype=torch.bfloat16)
pipe.transformer = load_quanto_transformer("Disty0/FLUX.1-dev-qint4")
pipe.text_encoder_2 = load_quanto_text_encoder_2("Disty0/FLUX.1-dev-qint4")
pipe = pipe.to("cuda", dtype=torch.bfloat16)


prompt = "A cat holding a sign that says hello world"
image = pipe(
    prompt,
    height=1024,
    width=1024,
    guidance_scale=3.5,
    num_inference_steps=50,
    max_sequence_length=512,
    generator=torch.Generator("cpu").manual_seed(0)
).images[0]
image.save("flux-dev.png")

高級用法

在上述基礎用法的基礎上，你可以根據自己的需求調整生成圖像的參數，如 height、width、guidance_scale 等，以獲得不同風格和質量的圖像。

📄 許可證

本項目使用 flux-1-dev-non-commercial-license 許可證，具體內容請參考 LICENSE.md。