SD14_pathology_controlnet Open Source Model - Freely Generate High-Quality Pathology Images

SD14 Pathology Controlnet

Developed by RiddleHe

A ControlNet model trained based on CompVis/stable-diffusion-v1-4, specifically designed for generating pathological images

Image Generation Open Source License:Openrail #Pathological Image Generation #Medical Imaging Control #Breast Cancer Cell Structure

Downloads 19

Release Time : 2/2/2025

Model Overview

This model is a Stable Diffusion-based ControlNet that generates high-quality pathological images through novel conditional control methods, particularly suitable for medical image generation such as breast cancer tissues.

Model Features

Pathological Image Generation

Optimized specifically for pathological images, capable of generating high-quality medical tissue images

ControlNet Architecture

Utilizes ControlNet conditional control for more precise image generation

Stable Diffusion Foundation

Based on the mature Stable Diffusion v1.4 model, ensuring generation quality

Model Capabilities

Text-to-Image Generation

Medical Image Generation

Conditional Control Image Generation

Use Cases

Medical Education

Pathological Teaching Material Generation

Generates various pathological images as teaching materials for medical students

Can generate detailed pathological images such as breast cancer tissues

Medical Research

Data Augmentation

Generates supplementary training data for medical imaging research

Can generate synthetic images with specific pathological features

🚀 controlnet-RiddleHe/SD14_pathology_controlnet

These are controlnet weights trained on CompVis/stable - diffusion - v1 - 4 with a new type of conditioning, enabling the generation of histopathology images of breast cancer.

🚀 Quick Start

These are controlnet weights trained on CompVis/stable-diffusion-v1-4 with new type of conditioning. You can find some example images below.

Example Images

Prompt: A histopathology image of breast cancer tissue.
Prompt: An image of breast cancer histopathology with detailed cellular structures.

✨ Features

Base Models: CompVis/stable-diffusion-v1-4, RiddleHe/SD14_pathology_lora
Library: diffusers
License: creativeml-openrail-m
Inference: Supported
Tags: stable-diffusion, stable-diffusion-diffusers, text-to-image, diffusers, controlnet, diffusers-training

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

controlnet = ControlNetModel.from_pretrained("RiddleHe/SD14_pathology_controlnet", torch_dtype=torch.float16)
pipe = StableDiffusionControlNetPipeline.from_pretrained(
    "RiddleHe/SD14_pathology_base", controlnet=controlnet, torch_dtype=torch.float16
)
pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)
pipe.to('cuda')

prompt = "A histopathology image of breast cancer tissue."
mask = mask.convert("RGB")  # Provide a mask

generator = torch.Generator(device='cuda').manual_seed(42)

with torch.no_grad():
  out = pipe(prompt, image=mask, num_inference_steps=70, num_images_per_prompt=3, generator=generator).images

Limitations and bias

[TODO: provide examples of latent issues and potential remediations]

🔧 Technical Details

The model is trained on 28216 image - mask pairs from the BRCA breast cancer dataset. Input is mask and output is image.

Mask is a single channel image with integer values from 0 to 21 representing 22 classes, eg. 1 representing tumor, 2 representing stroma.

📄 License

The model is released under the creativeml - openrail - m license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご