Mol Llama 3.1 8B Instruct

🚀 Mol-Llama-3.1-8B-Instruct

This project provides the weights of Mol-LLaMA, including LoRA weights and projectors, based on meta-llama/Llama-3.1-8B-Instruct, aiming to offer capabilities in biology, chemistry, and medical fields.

[Project Page] [Paper] [GitHub]

✨ Features

This repo contains the weights of Mol-LLaMA including the LoRA weights and projectors, based on meta-llama/Llama-3.1-8B-Instruct.

🔧 Technical Details

Architecture

1. Molecular encoders: Pretrained 2D encoder (MoleculeSTM) and 3D encoder (Uni-Mol)
2. Blending Module: Combining complementary information from 2D and 3D encoders via cross-attention
3. Q-Former: Embed molecular representations into query tokens based on SciBERT
4. LoRA: Adapters for fine-tuning LLMs

Training Dataset

Mol-LLaMA is trained on Mol-LLaMA-Instruct, to learn the fundamental characteristics of molecules with the reasoning ability and explanbility.

💻 Usage Examples

Please check out the exemplar code for inference in the Github repo.

📚 Documentation

Citation

If you find our model useful, please consider citing our work.

@misc{kim2025molllama,
    title={Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Model},
    author={Dongki Kim and Wonbin Lee and Sung Ju Hwang},
    year={2025},
    eprint={2502.13449},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

Acknowledgements

We appreciate LLaMA, 3D-MoLM, MoleculeSTM, Uni-Mol and SciBERT for their open-source contributions.

📄 License

The project is under the apache-2.0 license.

Information Table