Sciglm 6B

🚀 SciGLM: Training Scientific Language Models with Self-Reflective Instruction Annotation and Tuning

SciGLM is a suite of scientific language models capable of performing college-level scientific reasoning. The core of our approach is a novel self-reflective instruction annotation framework, designed to tackle the data scarcity issue in the scientific domain. This framework uses existing large language models (LLMs) to generate step-by-step reasoning for unlabeled scientific questions, followed by a self-reflective critic-and-revise process. Using this framework, we created SciInstruct, a diverse and high - quality dataset covering physics, chemistry, math, and formal proofs.

✨ Features

SciInstruct

We construct the SciInstruct as follows:

We release our data and model for public use. If you wish to use SciInstruct or SciGLM, you can download them from the following links:

• Download data:
  • Google Drive
  • Tsinghua Cloud
• Download model:
  • Hugging Face

📦 Installation

Fine - tuning

You can use the SciGLM model through Huggingface's Transformers library.

git clone https://github.com/THUDM/SciGLM.git
cd SciGLM
pip install -r requirements.txt

To train the 6B model, run:

bash /path/training/finetune.sh

Inference

cd /path/to/inference
python cli_demo.py

📄 License

The project is licensed under the Apache - 2.0 license.

📚 Documentation

Citation

If you find our work helpful, please kindly cite our paper:

@article{zhang2024sciglm,
  title={Sciglm: Training scientific language models with self - reflective instruction annotation and tuning},
  author={Zhang, Dan and Hu, Ziniu and Zhoubian, Sining and Du, Zhengxiao and Yang, Kaiyu and Wang, Zihan and Yue, Yisong and Dong, Yuxiao and Tang, Jie},
  journal={arXiv preprint arXiv:2401.07950},
  year={2024}
}