Pai Bloom 1b1 Text2prompt Sd V2

🚀 BeautifulPrompt-v2

We open-sourced an automatic Prompt generation model. You can directly input a very simple Prompt and get a Prompt optimized by the language model, which helps you generate high-quality images more easily. Compared with v1, we have improved the performance in complex scenarios and added the ability to generate weights (used with sd-webui).

🚀 Quick Start

We release an automatic Prompt generation model, you can directly enter an extremely simple Prompt and get a Prompt optimized by the language model to help you generate more beautiful images simply. Compared with v1, we have improved the performance in complex scenarios and increased the ability to generate weights (use with sd-webui).

• Github: EasyNLP

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained('alibaba-pai/pai-bloom-1b1-text2prompt-sd-v2')
model = AutoModelForCausalLM.from_pretrained('alibaba-pai/pai-bloom-1b1-text2prompt-sd-v2').eval().cuda()
raw_prompt = '1 girl'

TEMPLATE_V2 = 'Converts a simple image description into a prompt. \
Prompts are formatted as multiple related tags separated by commas, plus you can use () to increase the weight, [] to decrease the weight, \
or use a number to specify the weight. You should add appropriate words to make the images described in the prompt more aesthetically pleasing, \
but make sure there is a correlation between the input and output.\n\
### Input: {raw_prompt}\n### Output:'

input = TEMPLATE_V2.format(raw_prompt=raw_prompt)
input_ids = tokenizer.encode(input, return_tensors='pt').cuda()
outputs = model.generate(
    input_ids,
    max_new_tokens=384,
    do_sample=True,
    temperature=0.9,
    top_k=50,
    top_p=0.95,
    repetition_penalty=1.1,
    num_return_sequences=5)

prompts = tokenizer.batch_decode(outputs[:, input_ids.size(1):], skip_special_tokens=True)
prompts = [p.strip() for p in prompts]
print(prompts)

📚 Documentation

Gallery

Before	After
prompt: a beautiful girl	prompt: (8k, RAW photo, best quality, masterpiece:1.2), (realistic, photo-realistic:1.37), octane render, ultra high res, photon mapping, radiosity, physically-based rendering, ue5, ((white dress)), ((long hair)), ((beautiful face)), ((light brown eyes)), ((smile))) extremely detailed CG unity 8k wallpaper, makeup, (glowing lips), (fantasy lining), (intricate details), light bokeh, (sharp focus) centered at the center of the face (wide angle:0.6), full body

Before	After
prompt: Astronaut rides horse	prompt: (masterpiece), (best quality), astronaut on horseback, (rides horse), ( helmet ), (standing on horseback), panorama, looking ahead, detailed background, solo

generated by sd-xl-1.0

Notice for Use

If you want to use this model, please read this document carefully and abide by the terms.

Paper Citation

If you find the model useful, please consider cite the paper:

@inproceedings{emnlp2023a,
  author    = {Tingfeng Cao and
            Chengyu Wang and
            Bingyan Liu and
            Ziheng Wu and
            Jinhui Zhu and
            Jun Huang},
  title     = {BeautifulPrompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesis},
  booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track},
  pages     = {1--11},
  year      = {2023}
}

📄 License

This project is licensed under the Apache-2.0 license.