Llama 3.1 Centaur 70B

🚀 Llama-3.1-Centaur-70B

Llama-3.1-Centaur-70B is a foundation model of cognition that can predict and simulate human behavior in any behavioral experiment expressed in natural language.

🚀 Quick Start

Llama-3.1-Centaur-70B is a powerful foundation model for simulating and predicting human behavior in natural - language - described behavioral experiments.

• Paper: A foundation model to predict and capture human cognition
• More Information: Documentation
• Point of Contact: Marcel Binz

✨ Features

• Cognitive Prediction: Capable of predicting and simulating human behavior in behavioral experiments described in natural language.
• Data - Encapsulated Training: Trained on a dataset where human choices are encapsulated by "<<" and ">>" tokens.

📦 Installation

You can use the model using HuggingFace Transformers library with 2 or more 80GB GPUs (NVIDIA Ampere or newer) with at least 150GB of free disk space to accommodate the download.

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "marcelbinz/Llama-3.1-Centaur-70B"
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

More details are provided in this example script. You can alternatively run the model with unsloth on a single 80GB GPU using the low - rank adapter.

⚠️ Important Note

Note that Centaur is trained on a data set in which human choices are encapsulated by "<<" and ">>" tokens. For optimal performance, it is recommended to adjust prompts accordingly.

📄 License

Llama 3.1 Community License Agreement

📚 Documentation

Citation Information

@misc{binz2024centaurfoundationmodelhuman,
      title={Centaur: a foundation model of human cognition}, 
      author={Marcel Binz and Elif Akata and Matthias Bethge and Franziska Brändle and Fred Callaway and Julian Coda-Forno and Peter Dayan and Can Demircan and Maria K. Eckstein and Noémi Éltető and Thomas L. Griffiths and Susanne Haridi and Akshay K. Jagadish and Li Ji-An and Alexander Kipnis and Sreejan Kumar and Tobias Ludwig and Marvin Mathony and Marcelo Mattar and Alireza Modirshanechi and Surabhi S. Nath and Joshua C. Peterson and Milena Rmus and Evan M. Russek and Tankred Saanum and Natalia Scharfenberg and Johannes A. Schubert and Luca M. Schulze Buschoff and Nishad Singhi and Xin Sui and Mirko Thalmann and Fabian Theis and Vuong Truong and Vishaal Udandarao and Konstantinos Voudouris and Robert Wilson and Kristin Witte and Shuchen Wu and Dirk Wulff and Huadong Xiong and Eric Schulz},
      year={2024},
      eprint={2410.20268},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2410.20268}, 
}