Cool Japan Diffusion 2 1 2

🚀 Cool Japan Diffusion 2.1.2 Model Card

Cool Japan Diffusion is a model fine - tuned from Stable Diffusion, specializing in expressing Cool Japan elements such as anime, manga, and games.

🚀 Quick Start

If you want to have a quick try, please use this Space. Detailed instructions on how to handle this model can be found here. You can download the model here.

✨ Features

Cool Japan Diffusion is a text - to - image generation model based on diffusion models. It can generate appropriate images according to prompts. The algorithms used are Latent Diffusion Model and OpenCLIP - ViT/H.

📦 Installation

Diffusers

Use 🤗's Diffusers library. First, run the following script to install the library:

pip install --upgrade git+https://github.com/huggingface/diffusers.git transformers accelerate scipy

💻 Usage Examples

Basic Usage

from diffusers import StableDiffusionPipeline, EulerAncestralDiscreteScheduler
import torch

model_id = "aipicasso/cool-japan-diffusion-2-1-2"

scheduler = EulerAncestralDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler")
pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, torch_dtype=torch.float32)
pipe = pipe.to("cuda")

prompt = "anime, masterpiece, a portrait of a girl, good pupil, 4k, detailed"
negative_prompt="deformed, blurry, bad anatomy, bad pupil, disfigured, poorly drawn face, mutation, mutated, extra limb, ugly, poorly drawn hands, bad hands, fused fingers, messy drawing, broken legs censor, low quality, mutated hands and fingers, long body, mutation, poorly drawn, bad eyes, ui, error, missing fingers, fused fingers, one hand with more than 5 fingers, one hand with less than 5 fingers, one hand with more than 5 digit, one hand with less than 5 digit, extra digit, fewer digits, fused digit, missing digit, bad digit, liquid digit, long body, uncoordinated body, unnatural body, lowres, jpeg artifacts, 3d, cg, text, japanese kanji"
images = pipe(prompt,negative_prompt=negative_prompt, num_inference_steps=20).images
images[0].save("girl.png")

Advanced Usage

• Using xformers: It is said that using xformers can speed up the process.
• For users with limited GPU memory: When using a GPU, if you have limited GPU memory, please use pipe.enable_attention_slicing().

📚 Documentation

Expected Use Cases

• Reporting on image - generation AI: It is applicable not only to public broadcasters but also to for - profit enterprises. The reason is that the "right to know" information about image - synthesis AI is judged not to have a negative impact on the creative industry. Also, freedom of the press is respected.
• Introduction of Cool Japan: Explain what Cool Japan is to people from other countries. Many international students are attracted to Japan because of Cool Japan. Alfred Increment believes that they are often disappointed to find that what is considered "Cool Japan" is not "cool" in Japan. People from other countries should be more proud of their own cultures that others admire.
• Research and development:
  • Model usage on Discord: Prompt engineering, fine - tuning (also known as additional learning) such as DreamBooth, and merging with other models.
  • Compatibility between Latent Diffusion Model and Cool Japan: Investigate the relationship between them.
  • Performance evaluation: Check the performance of this model using metrics like FID.
  • Independence check: Use checksums or hash functions to verify that this model is independent of models other than Stable Diffusion.
• Education:
  • Graduation projects for art college students and vocational school students: They can use this model for their graduation works.
  • Graduation theses and assignment projects for university students: It can be applied in academic works.
  • Teachers' dissemination of the current situation of image - generation AI: Teachers can use it to teach students about the current state of image - generation AI.
• Self - expression: Express one's emotions and thoughts on SNS.
• Use cases described in Hugging Face's Community: Please ask questions in Japanese or English.

Unexpected Use Cases

• Presenting things as facts: Avoid presenting non - factual information as facts.
• Use in monetized content: Do not use it in monetized content such as YouTube videos.
• Direct commercial service provision: Do not directly provide it as a commercial service.
• Causing trouble to teachers: Do not do things that may trouble teachers.
• Negative impact on the creative industry: Avoid any actions that may have a negative impact on the creative industry.

Prohibited or Malicious Use Cases

• Digital Forgery: Do not publish digital forgeries (Digital Forgery) as it may violate copyright laws.
• Unauthorized Image - to - Image: Do not perform Image - to - Image operations on others' works without permission, which may violate copyright laws.
• Pornography distribution: Do not distribute pornographic materials, which may violate Article 175 of the criminal code.
  • Violating industry etiquette: Do not do things that violate the general etiquette in the industry.
• Spreading false information: Do not spread non - factual information as facts, which may lead to the application of the crime of interfering with business by force.
  • Fake news: Avoid creating and spreading fake news.

🔧 Technical Details

Model Limitations

The limitations are not well - understood yet.

Biases

This model has the same biases as Stable Diffusion. Please be cautious.

Training

Training Data

• VAE: About 650,000 types of data that comply with Japanese domestic laws, excluding Danbooru and Danbooru dataset (an infinite number of samples can be created through data augmentation).
• U - Net: 2 million pairs of data that comply with Japanese domestic laws, excluding Danbooru and Danbooru dataset, and 3 merged models.

Training Process The VAE and U - Net of Stable Diffusion were fine - tuned.

• Hardware: A6000
• Optimizer: AdamW
• Gradient Accumulations: 1
• Batch Size: 1

Evaluation Results

No evaluation results are provided in the original document.

Environmental Impact

The environmental impact is minimal.

• Hardware Type: A6000
• Usage Time (in hours): 200
• Cloud Provider: None
• Training Location: Japan
• Carbon Emissions: Not significant

📄 License

The license of this model is based on the original CreativeML Open RAIL++ - M License, with the addition of a prohibition on commercial use except for certain exceptions. The reason for adding this prohibition is the concern that it may have a negative impact on the creative industry. If this concern is dispelled, the next version will revert to the original license, allowing commercial use. The Japanese translation of the original license can be found here. People in for - profit enterprises should consult their legal departments. For those using it for personal hobbies, as long as they follow common sense, they should be fine. As stated in the license, if you modify this model, you need to inherit this license.

Legal and Ethical Considerations

This model was created in Japan, so Japanese laws apply. The creator claims that the training of this model is legal based on Article 30 - 4 of the Copyright Law. Also, regarding the distribution of this model, the creator claims that it does not constitute the principal offender or an accessory offender in light of the Copyright Law and Article 175 of the Criminal Code. For more details, please refer to the opinion of lawyer Kakinuma. However, as stated in the license, the products generated by this model should be handled in accordance with various laws and regulations.

The creator believes that the act of distributing this model is not ethically good because the permission of the copyright holders of the training materials was not obtained. However, legally, obtaining the permission of the copyright holders is not necessary for training, just like search engines, and there is no legal problem. Therefore, please consider that this distribution also serves the purpose of investigating the ethical aspects rather than just the legal ones.

Model Details

⚠️ Important Note

• When using this model, please comply with relevant laws and regulations and ethical standards. Do not use it for prohibited or malicious purposes.
• For commercial use, please note that it is generally prohibited, except for certain exceptions.

💡 Usage Tip

• It is recommended to install xformers when using the Web UI.
• If you have limited GPU memory, use pipe.enable_attention_slicing() when using the Diffusers method.

This model card was written by Alfred Increment based on Stable Diffusion v2.