Janus-Pro-1B Open-source Model - Unifying Multimodal Understanding and Generation, Super Practical for Handling Multiple Tasks!

Janus Pro 1B

Developed by deepseek-ai

Janus-Pro is a novel autoregressive framework that unifies multimodal understanding and generation capabilities. By decoupling visual encoding paths, it uses a single Transformer architecture to handle multimodal tasks.

Text-to-Image

Transformers

Open Source License:MIT #Multimodal Unified Model #Autoregressive Framework #Decoupled Visual Encoding

Downloads 34.02k

Release Time : 1/26/2025

Model Overview

Janus-Pro is a unified model for multimodal understanding and generation. Through its decoupled visual encoding design, it resolves conflicts between understanding and generation roles, offering high flexibility and efficiency.

Model Features

Decoupled Visual Encoding

Decouples visual encoding into independent paths, alleviating conflicts between understanding and generation roles, enhancing model flexibility.

Unified Architecture

Uses a single Transformer architecture to handle multimodal tasks, simplifying model design.

High Performance

Surpasses previous unified models, achieving or exceeding the performance of specialized task models.

Model Capabilities

Multimodal Understanding

Text-to-Image Generation

Visual Question Answering

Image Caption Generation

Use Cases

Content Generation

Image Generation

Generates high-quality images based on text descriptions.

Supports 16x downsampling rate, producing images with rich details.

Visual Understanding

Image Analysis

Understands image content and answers related questions.

Supports 384 x 384 resolution image input.

🚀 Janus-Pro

Janus-Pro is a novel autoregressive framework that unifies multimodal understanding and generation, offering high flexibility and effectiveness.

🚀 Quick Start

Please refer to Github Repository

✨ Features

Janus-Pro is a novel autoregressive framework that unifies multimodal understanding and generation.
It decouples visual encoding into separate pathways while using a single, unified transformer architecture, alleviating the conflict between the visual encoder’s roles in understanding and generation and enhancing flexibility.
It surpasses previous unified models and matches or exceeds the performance of task - specific models.

Github Repository

📚 Documentation

Model Summary

Janus-Pro is a unified understanding and generation MLLM, which decouples visual encoding for multimodal understanding and generation. Janus-Pro is constructed based on the DeepSeek-LLM-1.5b-base/DeepSeek-LLM-7b-base.

For multimodal understanding, it uses the SigLIP-L as the vision encoder, which supports 384 x 384 image input. For image generation, Janus-Pro uses the tokenizer from here with a downsample rate of 16.

Property	Details
Pipeline Tag	any - to - any
Library Name	transformers
Tags	multimodal, text - to - image, unified - model
Model Type	Unified understanding and generation MLLM
Based on	DeepSeek-LLM-1.5b-base/DeepSeek-LLM-7b-base
Vision Encoder for Understanding	SigLIP-L (supports 384 x 384 image input)
Tokenizer for Image Generation	Tokenizer from here (downsample rate of 16)

📄 License

This code repository is licensed under the MIT License. The use of Janus-Pro models is subject to DeepSeek Model License.

📚 Citation

@article{chen2025janus,
  title={Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling},
  author={Chen, Xiaokang and Wu, Zhiyu and Liu, Xingchao and Pan, Zizheng and Liu, Wen and Xie, Zhenda and Yu, Xingkai and Ruan, Chong},
  journal={arXiv preprint arXiv:2501.17811},
  year={2025}
}

📞 Contact

If you have any questions, please raise an issue or contact us at service@deepseek.com.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご