AA Chameleon 7b Plus

🚀 Align-Anything Chameleon 7B Plus

A powerful model for text-image interleaved input and output, further aligned by the Align-Anything algorithm.

🚀 Quick Start

This repository is for Align-Anything Chameleon 7B Plus, a potent model designed for text-image interleaved input and output. It undergoes further alignment through the Align-Anything algorithm. This model is built upon the Chameleon model and is trained and aligned within the Align-Anything framework. This process further unlocks its image generation capabilities and enhances its alignment with human preferences.

💻 Usage Examples

Basic Usage

To use this model, you can refer to the Align-Anything repository for comprehensive details regarding training, inference, and evaluation:

git clone https://github.com/PKU-Alignment/align-anything.git
cd align-anything/projects/text_image_to_text_image

Then, follow the instructions in the README.md file to set up the environment and execute the scripts.

Currently, the official Transformer repository does not support the Chameleon model with image output (for more details, see this PR). Therefore, we rely on a specific fork of the repository.

After installing Align-Anything and correctly configuring the environment, you can install the forked stable version of the repository by running:

pip install git+https://github.com/htlou/transformers.git@hantao_stable_cham

Advanced Usage

If you wish to generate images (pure text generation can be directly accomplished using Transformers), you can follow the instructions in the mmsg_chameleon repository to perform inference.

git clone https://github.com/htlou/mmsg_chameleon.git
cd mmsg_chameleon

Then, set up the environment using:

pip install -e . 

After setting up the environment, configure the correct paths in scripts/interleaved_gen.sh and then run:

bash scripts/interleaved_gen.sh

📄 License

This project is licensed under the CC BY 4.0 license.