Proteusv0.3 Lightning

🚀 ProteusV0.3-Lightning

This project uses the new Lightning method released by Bytedance for faster inference with minimal loss in terms of quality and prompt comprehension. It is an advanced version of OpenDalleV1.1, offering better responsiveness to prompts and enhanced creative capabilities.

🚀 Quick Start

✨ Features

• Advanced Enhancement: Proteus is a sophisticated upgrade over OpenDalleV1.1, leveraging its core functions to achieve better results. It has improved responsiveness to prompts and enhanced creative abilities.
• Fine - Tuning Data: Fine - tuned using about 220,000 GPTV captioned images from copyright - free stock images (including some anime), which were then normalized.
• DPO Optimization: Employed Direct Preference Optimization (DPO) with a collection of 10,000 carefully selected high - quality, AI - generated image pairs.
• LORA Models: Multiple Low - Rank Adaptation (LORA) models are trained independently and then selectively incorporated into the main model through dynamic application methods, resulting in better portrayal of facial features and skin textures, and good performance in various aesthetic domains.

📦 Installation

There is no specific installation information provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

import torch
from diffusers import (
    StableDiffusionXLPipeline, 
    EulerAncestralDiscreteScheduler,
    AutoencoderKL
)

# Load VAE component
vae = AutoencoderKL.from_pretrained(
    "madebyollin/sdxl-vae-fp16-fix", 
    torch_dtype=torch.float16
)

# Configure the pipeline
pipe = StableDiffusionXLPipeline.from_pretrained(
    "dataautogpt3/ProteusV0.3-Lightning", 
    vae=vae,
    torch_dtype=torch.float16
)
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
pipe.to('cuda')

# Define prompts and generate image
prompt = "black fluffy gorgeous dangerous cat animal creature, large orange eyes, big fluffy ears, piercing gaze, full moon, dark ambiance, best quality, extremely detailed"
negative_prompt = "nsfw, bad quality, bad anatomy, worst quality, low quality, low resolutions, extra fingers, blur, blurry, ugly, wrongs proportions, watermark, image artifacts, lowres, ugly, jpeg artifacts, deformed, noisy image"

image = pipe(
    prompt, 
    negative_prompt=negative_prompt, 
    width=1024,
    height=1024,
    guidance_scale=1,
    num_inference_steps=4
).images[0]

📚 Documentation

Settings for ProteusV0.3 - Lightning

Use these settings for the best results with ProteusV0.3 - Lightning:

• CFG Scale: Use a CFG scale of 1 to 2.
• Steps: 4 to 10 steps for more detail, 8 steps for faster results.
• Sampler: eular
• Scheduler: normal
• Resolution: 1280x1280 or 1024x1024

Please also consider using these keep words to improve your prompts: best quality, HD, ~*~aesthetic~*~.

If you are having trouble coming up with prompts, you can use this GPT I put together to help you refine the prompt. [Diffusion Master GPT](https://chat.openai.com/g/g - RziQNoydR - diffusion - master)

📄 License

This project is licensed under the GPL - 3.0 license.

Gallery

The following are some examples of input prompts and their corresponding output images:

Support

Please support the work I do through donating to me on: Buy Me a Coffee or following me on Twitter