Arcane-Diffusion Open-Source AI Model - Free Generation of "Arcane" Style Special Effect Images

Arcane Diffusion

Developed by nitrosocke

A Stable Diffusion model fine-tuned on scenes from the TV series 'Arcane', capable of triggering style effects with specific tags

Image Generation Open Source License:Openrail #Arcane style generation #TV series style transfer #Text-driven drawing

Downloads 2,366

Release Time : 10/2/2022

Model Overview

This is a text-to-image generation model based on the Stable Diffusion architecture, specifically fine-tuned for the 'Arcane' animation style. Users can generate images with the distinctive artistic style of the animation by adding the 'arcane style' tag to their prompts.

Model Features

Style trigger tag

Use the 'arcane style' prompt tag to activate the unique artistic style of the 'Arcane' animation

Multi-version optimization

Three iterative versions available, with V3 featuring a new training text encoder configuration that significantly improves generation quality and controllability

Format compatibility

Supports export to ONNX, MPS, or FLAX/JAX formats for different runtime environments

Model Capabilities

Text-to-image generation

Art style imitation

High-resolution image generation

Use Cases

Creative arts

Character concept design

Generate character concept art with 'Arcane' style for game or animation projects

Quick production of stylistically consistent character designs

Illustration creation

Automatically generate illustrations with specific artistic styles based on text descriptions

Significantly improves artistic creation efficiency

Entertainment applications

Fan art creation

Generate fan art for 'Arcane' enthusiasts

Maintains the original artistic style while enabling personalized creations

🚀 Arcane Diffusion

Arcane Diffusion is a fine - tuned Stable Diffusion model trained on images from the TV Show Arcane. Use the "arcane style" token in prompts to achieve the desired effect.

🚀 Quick Start

This model can be used just like any other Stable Diffusion model. For more information, please have a look at the Stable Diffusion. You can also export the model to ONNX, MPS and/or FLAX/JAX.

💻 Usage Examples

Basic Usage

#!pip install diffusers transformers scipy torch
from diffusers import StableDiffusionPipeline
import torch

model_id = "nitrosocke/Arcane-Diffusion"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "arcane style, a magical princess with golden hair"
image = pipe(prompt).images[0]

image.save("./magical_princess.png")

📚 Documentation

Gradio & Colab

We also support a Gradio Web UI and Colab with Diffusers to run fine - tuned Stable Diffusion models:

Sample Images

Sample images from v3:

output Samples v3

Sample images from the model:

output Samples

Sample images used for training:

Training Samples

Model Versions

Version 3 (arcane - diffusion - v3): This version uses the new train - text - encoder setting and improves the quality and edibility of the model immensely. Trained on 95 images from the show in 8000 steps.
Version 2 (arcane - diffusion - v2): This uses the diffusers based dreambooth training and prior - preservation loss is way more effective. The diffusers where then converted with a script to a ckpt file in order to work with automatics repo. Training was done with 5k steps for a direct comparison to v1 and results show that it needs more steps for a more prominent result. Version 3 will be tested with 11k steps.
Version 1 (arcane - diffusion - 5k): This model was trained using Unfrozen Model Textual Inversion utilizing the Training with prior - preservation loss methods. There is still a slight shift towards the style, while not using the arcane token.

📄 License

The model is licensed under the creativeml - openrail - m license.

💡 Usage Tip

If you enjoy my work, please consider supporting me.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご