基礎模型:

• black-forest-labs/FLUX.1-dev 庫名稱: diffusers 許可證: 其他 許可證名稱: flux-1-dev-non-commercial-license 許可證鏈接: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md 管道標籤: 圖像到圖像 推理: 否 標籤:
• ControlNet
• 超分辨率
• 放大工具

⚡ Flux.1-dev: 放大控制網絡 ⚡

這是由Jasper研究團隊開發的用於低分辨率圖像的Flux.1-dev ControlNet。

使用方法

該模型可以直接與diffusers庫一起使用

import torch
from diffusers.utils import load_image
from diffusers import FluxControlNetModel
from diffusers.pipelines import FluxControlNetPipeline

# 加載管道
controlnet = FluxControlNetModel.from_pretrained(
  "jasperai/Flux.1-dev-Controlnet-Upscaler",
  torch_dtype=torch.bfloat16
)
pipe = FluxControlNetPipeline.from_pretrained(
  "black-forest-labs/FLUX.1-dev",
  controlnet=controlnet,
  torch_dtype=torch.bfloat16
)
pipe.to("cuda")

# 加載控制圖像
control_image = load_image(
  "https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Upscaler/resolve/main/examples/input.jpg"
)

w, h = control_image.size

# 放大4倍
control_image = control_image.resize((w * 4, h * 4))

image = pipe(
    prompt="", 
    control_image=control_image,
    controlnet_conditioning_scale=0.6,
    num_inference_steps=28, 
    guidance_scale=3.5,
    height=control_image.size[1],
    width=control_image.size[0]
).images[0]
image

訓練

該模型採用了一種合成的複雜數據降級方案進行訓練，輸入為真實生活圖像，並通過結合多種降級方法（如高斯噪聲、泊松噪聲、圖像模糊和JPEG壓縮等）對其進行人工降級，類似於[1]中的方法。

[1] Wang, Xintao, 等. "Real-esrgan: Training real-world blind super-resolution with pure synthetic data." Proceedings of the IEEE/CVF international conference on computer vision. 2021.

許可證

該模型遵循Flux.1-dev模型許可證。