Controlnet Mlsd 202305011046

🚀 controlnet- tsungtao/controlnet-mlsd-202305011046

These ControlNet weights are trained on runwayml/stable-diffusion-v1-5 with a new type of conditioning, enabling specific image generation tasks.

🚀 Quick Start

These are ControlNet weights trained on runwayml/stable-diffusion-v1-5 with a new type of conditioning. You can find some example images below.

Example Images

• Prompt: a living room with a dining table and chairs
• Prompt: a living room with couches and a tv

📚 Documentation

Model Information

Training Data Details

The training data has two columns: raw training data, conditioning training data, and their prompts.

• Raw Training Data: Crawled from the internet for living rooms with a special training purpose. They were resized using a local tool as a standardization process before training.
• Conditioning Data: Created based on the raw training data. We deployed the local Stable Diffusion and ControlNet plugin in our local development environment to set up the target conditioning data. After that, the conditioning data was also resized through the standardization procedure.
• Prompts: Added manually due to the small testing dataset. However, it can also be done using some online tools.

We have an upload script to combine all the raw data, conditioning data, and prompts into a dataset and then automatically upload them to Hugging Face.

Model Functionality

From the conditioning data, you can see the framework of the raw data. Training based on this will result in a model that can extract the line framework of the input. At the same time, the raw data all have a special style. So, based on this model, you can get the line framework of your raw input living - room data and then transform the input living room into a living room with a special style.

Functionally, it is similar to the "MLSD" model, but it performs better on living - room data with a special style.

📄 License

This project is licensed under creativeml-openrail-m.