Mmalaya

🚀 MMAlaya

MMAlaya is a multimodal model based on the large language model Alaya. It can effectively process and integrate multiple types of information, providing strong support for various multimodal tasks.

🚀 Quick Start

MMAlaya is a multimodal model built upon the large language model Alaya. The model weight files can be found at DataCanvas/MMAlaya.

MMAlaya consists of the following three modules:

1. The large language model Alaya-7B-Chat.
2. The image-text feature encoder is EVA - G from blip2-opt-2.7b.
3. The connector from image-text features to the large language model uses the Qformer and linear projector from blip2-opt-2.7b.

The model is mainly trained based on the LLaVA architecture.

In the OpenCompass evaluation leaderboard, it achieved an average score of 41.1 and ranked 25th. In the MMBench evaluation leaderboard, among the open - source models on the Chinese test set, it achieved an average score of 58.6 and ranked 25th.

For inference, you can refer to inference.py.

📄 License

MMAlaya uses the Apache 2.0 License. The model weights are open - sourced and commercial use is permitted. If your project references our MMAlaya, please indicate the source:

@misc{datacanvas2024mmalaya,
    author = {DataCanvas Ltd.},
    title = {mmalaya},
    year = {2024},
    howpublished = {\url{https://github.com/DataCanvasIO/MMAlaya}},
}