Llemma 34B Open-Source Mathematical Language Model - Freely Assist in Solving Various Mathematical Puzzles

Llemma 34b

Developed by EleutherAI

Llemma 34B is a language model specialized in the field of mathematics, initialized based on the weights of Code Llama 34B and trained on 50 billion tokens from the Proof-Pile-2 dataset.

Large Language Model

Transformers

English#Mathematical Reasoning #Theorem Proving #Chain-of-Thought Reasoning

Downloads 60

Release Time : 9/27/2023

Model Overview

Llemma is an open mathematical language model focused on mathematical reasoning and computational tasks, excelling in chain-of-thought mathematical reasoning and the use of mathematical computation tools such as Python and formal theorem provers.

Model Features

Mathematical Expertise

Optimized specifically for the field of mathematics, excelling in mathematical reasoning and computational tasks.

Chain-of-Thought Reasoning

Supports complex chain-of-thought reasoning processes, enabling step-by-step solutions to mathematical problems.

Tool Integration

Capable of using mathematical computation tools such as Python and formal theorem provers.

Open Model

Released under an open license, available for both research and commercial use.

Model Capabilities

Mathematical problem-solving

Theorem proving

Mathematical reasoning

Chain-of-thought reasoning

Python code generation

Formal proof

Use Cases

Education

Mathematical Problem Solving

Helps students understand and solve various mathematical problems.

Performs excellently on mathematical test sets like GSM8k.

Math Tutoring

Provides step-by-step guidance for solving mathematical problems.

Demonstrates problem-solving processes through chain-of-thought reasoning.

Research

Mathematical Theorem Proving

Assists mathematical researchers in proving theorems.

Capable of using formal theorem provers.

Mathematical Computation

Performs complex mathematical computation tasks.

Supports computation tools like Python.

🚀 Llemma 34B

Llemma 34B is a language model designed for mathematics. It addresses the need for advanced mathematical reasoning and computational support, offering high - performance solutions in mathematical tasks.

Llemma

Authors: Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q. Jiang, Jia Deng, Stella Biderman, Sean Welleck

This model was initialized with Code Llama 34B weights and trained on the Proof - Pile - 2 for 50B tokens. There is also a 7B parameter version available: Llemma 7B.

✨ Features

Chain - of - thought Mathematical Reasoning: Llemma models excel at chain - of - thought mathematical reasoning, outperforming many other models in relevant tasks.
Computational Tool Use: It has strong capabilities in using computational tools for mathematics, such as Python and formal theorem provers.

📚 Documentation

Evaluations

Chain - of - thought Math

On chain - of - thought mathematics tasks, Llemma models show remarkable performance. The following table compares the performance of different models:

Model	Size	GSM8k	OCW	MMLU - STEM	SAT	MATH
Llama 2	7B	11.8%	3.7%	29.9%	25%	3.2%
Code Llama	7B	10.5%	4.4%	25.1%	9.4%	4.5%
LLEMMA	7B	36.4%	7.7%	37.7%	53.1%	18.0%
Minerva	8B	16.2%	7.7%	35.6%	-	14.1%
------------	------	--------	-------	-----------	-------	-------
Code Llama	34B	29.6%	7.0%	40.5%	40.6%	12.2%
LLEMMA	34B	51.5%	11.8%	49.0%	71.9%	25.0%
------------	------	--------	-------	-----------	-------	-------
Minerva	62B	52.4%	12.0%	53.9%	-	27.6%
Minerva	540B	58.8%	17.6%	63.9%	-	33.6%

Further performance can be achieved by using majority voting:

Model	Size	GSM8k maj@100	OCW maj@100	MMLU - STEM maj@16	SAT maj@16	MATH maj@256
LLEMMA	7B	54.0%	14.3%	49.9%	78.1%	33.5
Minerva	8B	28.4%	12.5%	43.4%	-	25.4%
---------	------	-------------	-----------	-----------------	-----------	------------
LLEMMA	34B	69.3%	18.4%	59.7%	81.3%	43.1%
---------	------	-------------	-----------	-----------------	-----------	------------
Minerva	62B	68.5%	23.5%	63.5%	-	43.4%
Minerva	540B	78.5%	30.8%	75.0%	-	50.3%

Tool Use and Theorem Proving

In addition to chain - of - thought reasoning, Llemma has strong capabilities in computational mathematics tasks. For tool use and formal theorem proving evaluations, refer to our paper.

Citation

@misc{azerbayev2023llemma,
      title={Llemma: An Open Language Model For Mathematics}, 
      author={Zhangir Azerbayev and Hailey Schoelkopf and Keiran Paster and Marco Dos Santos and Stephen McAleer and Albert Q. Jiang and Jia Deng and Stella Biderman and Sean Welleck},
      year={2023},
      eprint={2310.10631},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

📄 License

The model uses the Llama 2 license.

Property	Details
Model Type	Language model for mathematics
Training Data	EleutherAI/proof - pile - 2, open - web - math/open - web - math

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご