Ben-Brand-LoRA Open Source Model - Free Deployment for Text-to-Image Generation and Specific Artistic Style Conversion

Ben Brand LoRA

Developed by davidrd123

A PEFT LoRA model trained based on FLUX.1-dev, focused on text-to-image generation tasks, supporting specific artistic style conversion.

Image Generation Open Source License:Other #FLUX.1 style adaptation #High-resolution image generation #Artistic text-to-image

Downloads 253

Release Time : 2/19/2025

Model Overview

This is a standard PEFT LoRA model trained on the FLUX.1-dev base model, primarily used for text-to-image generation tasks, capable of generating images with specific artistic styles based on text descriptions.

Model Features

Artistic style conversion

Capable of generating images with specific artistic styles based on text descriptions, such as the b3nbr4nd style shown in the examples.

High-resolution output

Supports image generation at resolutions up to 1024x1024.

Efficient fine-tuning

Uses LoRA technology for parameter-efficient fine-tuning, achieving style transfer by training only a small number of parameters.

Model Capabilities

Text-to-image generation

Artistic style conversion

High-resolution image generation

Use Cases

Creative design

Concept art creation

Quickly generate concept art images based on text descriptions

Such as the example image of a giant green snake coiled around an obelisk

Stylized image generation

Convert ordinary descriptions into images with specific artistic styles

Such as generating images in the b3nbr4nd style

Game development

Game scene concept design

Quickly generate concept art for game scenes

Such as the example scene of a partially buried ancient ruin

🚀 Ben-Brand-LoRA

This is a standard PEFT LoRA derived from black-forest-labs/FLUX.1-dev. It offers a way to generate images based on specific settings and datasets, with clear training and inference processes.

🚀 Quick Start

This Ben-Brand-LoRA is a standard PEFT LoRA derived from black-forest-labs/FLUX.1-dev. No validation prompt was used during training.

✨ Features

It is derived from the black-forest-labs/FLUX.1-dev model.
The text encoder was not trained, and you can reuse the base model text encoder for inference.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

import torch
from diffusers import DiffusionPipeline

model_id = 'black-forest-labs/FLUX.1-dev'
adapter_id = 'davidrd123/Ben-Brand-LoRA'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)

prompt = "An astronaut is riding a horse through the jungles of Thailand."


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1024,
    height=1024,
    guidance_scale=3.0,
).images[0]
image.save("output.png", format="PNG")

📚 Documentation

Validation settings

CFG: 3.0
CFG Rescale: 0.0
Steps: 20
Sampler: FlowMatchEulerDiscreteScheduler
Seed: 42
Resolution: 1024x1024
Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

Training settings

Property	Details
Training epochs	2
Training steps	3750
Learning rate	0.00015 - Learning rate schedule: constant - Warmup steps: 100
Max grad norm	0.1
Effective batch size	6 - Micro-batch size: 2 - Gradient accumulation steps: 3 - Number of GPUs: 1
Gradient checkpointing	True
Prediction type	flow-matching (extra parameters=['shift=3', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0', 'flow_matching_loss=compatible', 'flux_lora_target=all'])
Optimizer	adamw_bf16
Trainable parameter precision	Pure BF16
Caption dropout probability	10.0%
LoRA Rank	64
LoRA Alpha	None
LoRA Dropout	0.1
LoRA initialisation style	default

Datasets

ben-brand-256

Repeats: 10
Total number of images: 98
Total number of aspect buckets: 3
Resolution: 0.065536 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

ben-brand-crop-256

Repeats: 10
Total number of images: 98
Total number of aspect buckets: 1
Resolution: 0.065536 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

ben-brand-512

Repeats: 10
Total number of images: 98
Total number of aspect buckets: 3
Resolution: 0.262144 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

ben-brand-crop-512

Repeats: 10
Total number of images: 98
Total number of aspect buckets: 1
Resolution: 0.262144 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

ben-brand-768

Repeats: 10
Total number of images: 98
Total number of aspect buckets: 3
Resolution: 0.589824 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

ben-brand-crop-768

Repeats: 10
Total number of images: 98
Total number of aspect buckets: 1
Resolution: 0.589824 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

ben-brand-1024

Repeats: 10
Total number of images: 98
Total number of aspect buckets: 4
Resolution: 1.048576 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

ben-brand-crop-1024

Repeats: 10
Total number of images: 98
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

ben-brand-1440

Repeats: 10
Total number of images: 98
Total number of aspect buckets: 2
Resolution: 2.0736 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

ben-brand-crop-1440

Repeats: 10
Total number of images: 98
Total number of aspect buckets: 1
Resolution: 2.0736 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

Inference

The provided Python code demonstrates how to perform inference using the Ben-Brand-LoRA model. It includes steps such as loading the model, setting up the pipeline, and generating an image based on a given prompt.

⚠️ Important Note

The validation settings are not necessarily the same as the training settings.

💡 Usage Tip

The model was quantised during training, and so it is recommended to do the same during inference time.

📄 License

The license is other.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご