Taiyi Stable Diffusion 1B Anime Chinese V0.1

🚀 Taiyi-Stable-Diffusion-1B-Chinese-v0.1

The first open-source Chinese Stable Diffusion anime model, trained on 1 million filtered Chinese anime image-text pairs, offering high-quality image generation capabilities.

🚀 Quick Start

This model is an open-source Chinese Stable Diffusion anime model. You can use it through a webui or write Python code to generate images. For more details, please refer to the following sections.

✨ Features

• Chinese Support: Specifically trained on Chinese anime image-text pairs, better adapted to Chinese prompts.
• High-Quality Output: Capable of generating high-resolution and high-quality anime images.
• Multiple Usage Modes: Supports text-to-image and image-to-image generation, as well as style transfer.

📦 Installation

This model can be used in a webui or by writing Python code. Here are the steps for using it in Python:

Prerequisites

Make sure you have installed the necessary libraries:

pip install git+https://github.com/huggingface/accelerate

Code Example

import torch
from diffusers import StableDiffusionPipeline
torch.backends.cudnn.benchmark = True
pipe = StableDiffusionPipeline.from_pretrained("IDEA-CCNL/Taiyi-Stable-Diffusion-1B-Anime-Chinese-v0.1", torch_dtype=torch.float16)
pipe.to('cuda')

prompt = '1个女孩,绿色头发,毛衣,看向阅图者,上半身,帽子,户外,下雪,高领毛衣'
image = pipe(prompt, guidance_scale=7.5).images[0]  
image.save("1个女孩.png")

💻 Usage Examples

Basic Usage

The following is a simple text-to-image generation example:

import torch
from diffusers import StableDiffusionPipeline
torch.backends.cudnn.benchmark = True
pipe = StableDiffusionPipeline.from_pretrained("IDEA-CCNL/Taiyi-Stable-Diffusion-1B-Anime-Chinese-v0.1", torch_dtype=torch.float16)
pipe.to('cuda')

prompt = '1个女孩,绿色头发,毛衣,看向阅图者,上半身,帽子,户外,下雪,高领毛衣'
image = pipe(prompt, guidance_scale=7.5).images[0]  
image.save("1个女孩.png")

Advanced Usage

You can use the super-resolution model to improve the image quality:

1个女孩,绿眼,棒球帽,金色头发,闭嘴,帽子,看向阅图者,短发,简单背景,单人,上半身,T恤
Negative prompt: 水彩,漫画,扫描件,简朴的画作,动画截图,3D,像素风,原画,草图,手绘,铅笔
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 3900970600, Size: 512x512, Model hash: 7ab6852a

Generate a 512 * 512 image:

Select the R-ESRGAN 4x+ Anime6B model in the extra section of the webui to perform super-resolution on the image:

You can get a 2048 * 2048 ultra-high-definition large image:

📚 Documentation

Model Taxonomy

Model Information

We used two anime datasets (1 million low-quality data and 10k high-quality data) for two-stage fine-tuning training of the Chinese anime model based on our pre-trained model IDEA-CCNL/Taiyi-Stable-Diffusion-1B-Chinese-v0.1. It took 100 hours to train this model using 4 x A100. This model is a preliminary version, and we will continuously update and open-source it. Welcome to exchange ideas!

Result Examples

• Style Transfer Examples:
• Text-to-Image Examples: | prompt1 | prompt2 | | ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | 1个男生,帅气,微笑,看着阅图者,简单背景,白皙皮肤,
  上半身,衬衫,短发,单人 | 1个女孩,绿色头发,毛衣,看向阅图者,上半身,帽子,户外,下雪,高领毛衣 | | | | | 户外,天空,云,蓝天,无人,多云的天空,风景,日出,草原 | 室内,杯子,书,无人,窗,床,椅子,桌子,瓶子,窗帘,阳光,
  风景,盘子,木地板,书架,蜡烛,架子,书堆,绿植,梯子,地毯,小地毯 | | | | | 户外,天空,水,树,无人,夜晚,建筑,风景,反射,灯笼,船舶,
  建筑学,灯笼,船,反射水,东亚建筑 | 建筑,科幻,城市,城市风景,摩天大楼,赛博朋克,人群 | | | | | 无人,动物,(猫:1.5),高清,棕眼 | 无人,动物,(兔子:1.5),高清,棕眼 | | | |

Usage Guides

• Taiyi Handbook
• Taiyi Painting Handbook v1.1
• Taiyi Anime Painting Handbook v1.0

Fine-Tuning

• Finetune Code

DreamBooth

• DreamBooth Code

🔧 Technical Details

We used two anime datasets for two-stage fine-tuning training. The calculation cost was 4 x A100 for about 100 hours.

📄 License

This model is licensed under the CreativeML OpenRAIL-M license. Please read the full license here: https://huggingface.co/spaces/CompVis/stable-diffusion-license

📖 Citation

If you use our model in your work, you can cite our paper:

@article{fengshenbang,
  author    = {Jiaxing Zhang and Ruyi Gan and Junjie Wang and Yuxiang Zhang and Lin Zhang and Ping Yang and Xinyu Gao and Ziwei Wu and Xiaoqun Dong and Junqing He and Jianheng Zhuo and Qi Yang and Yongfeng Huang and Xiayu Li and Yanghan Wu and Junyu Lu and Xinyu Zhu and Weifeng Chen and Ting Han and Kunhao Pan and Rui Wang and Hao Wang and Xiaojun Wu and Zhongshen Zeng and Chongpei Chen},
  title     = {Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence},
  journal   = {CoRR},
  volume    = {abs/2209.02970},
  year      = {2022}
}

You can also cite our website:

@misc{Fengshenbang-LM,
  title={Fengshenbang-LM},
  author={IDEA-CCNL},
  year={2021},
  howpublished={\url{https://github.com/IDEA-CCNL/Fengshenbang-LM}},
}