đ Flux.1-dev Quantized Model
This project offers Flux.1-dev in experimental custom formats, aiming to explore the potential of mixed tensor formats in image generation.
⨠Features
- Mixed Tensor Formats: Mixes tensors in Q8_0, fp16, and fp32 to potentially improve quality.
- Lossless Conversion: Converted from black - forest - labs' original bf16 weights, with fp32 conversion being lossless.
- Multiple Quantized Versions: Provides different quantized versions for users to choose from.
đĻ Installation
The model files can be used with the [ComfyUI - GGUF](https://github.com/city96/ComfyUI - GGUF) custom node. Place model files in ComfyUI/models/unet
. Refer to the GitHub readme for further install instructions.
đģ Usage Examples
Basic Usage
There is no specific code example provided in the original README. But you can use the model with the [ComfyUI - GGUF](https://github.com/city96/ComfyUI - GGUF) custom node as described in the installation section.
đ Documentation
Motivation
Flux's weights were published in bf16. Conversion to fp16 is slightly lossy, but fp32 is lossless. The author experimented with mixed tensor formats to see if it would improve quality.
Evaluation
The author tried comparing the outputs but couldn't say with certainty if these models are significantly better than pure Q8_0. Higher bits per weight (bpw) numbers result in slower computation:
20 s Q8_0
23 s 11.024bpw - txt16.gguf
30 s fp16
37 s 16.422bpw - txt32.gguf
310 s fp32
Update 2024 - 08 - 26
Two new files are added. This time the only tensors in Q8_0 are some or all of:
double_blocks.*.img_mlp.0.weight
double_blocks.*.img_mlp.2.weight
double_blocks.*.txt_mlp.0.weight
double_blocks.*.txt_mlp.2.weight
double_blocks.*.img_mod.lin.weight
double_blocks.*.txt_mod.lin.weight
single_blocks.*.linear1.weight
single_blocks.*.linear2.weight
single_blocks.*.modulation.lin.weight
flux1 - dev - Q8_0 - fp32 - 11.763bpw.gguf
This version has all the above layers in Q8_0.
flux1 - dev - Q8_0 - fp32 - 13.962bpw.gguf
This version preserves first 2 layers of all kinds, and first 4 MLP layers in fp32.
flux1 - dev - Q8_0 - fp32 - 16.161bpw.gguf
This one, first 4 layers of any kind and first 8 MLP layers in fp32.
In the txt16/32 files, the author quantized only these layers to Q8_0, unless they were one - dimensional:
img_mlp.0
img_mlp.2
img_mod.lin
linear1
linear2
modulation.lin
But left all these at fp16 or fp32, respectively:
txt_mlp.0
txt_mlp.2
txt_mod.lin
The resulting bpw number is just an approximation from file size.
General Information
This is a direct GGUF conversion of [black - forest - labs/FLUX.1 - dev](https://huggingface.co/black - forest - labs/FLUX.1 - dev/tree/main). As this is a quantized model not a finetune, all the same restrictions/original license terms still apply. Please refer to [this chart](https://github.com/ggerganov/llama.cpp/blob/master/examples/perplexity/README.md#llama - 3 - 8b - scoreboard) for a basic overview of quantization types.
(Model card mostly copied from [city96/FLUX.1 - dev - gguf](https://huggingface.co/city96/FLUX.1 - dev - gguf) - which contains conventional and useful GGUF files.)
đ License
- License Name: flux - 1 - dev - non - commercial - license
- License Link: LICENSE.md
đ Information Table
Property |
Details |
Base Model |
black - forest - labs/FLUX.1 - dev |
Library Name |
gguf |
Quantized By |
mo137 |
Tags |
text - to - image, image - generation, flux |
đĄ Usage Tip
You're probably better off using Q8_0, but these mixed tensor format models are shared in case someone finds them useful.