P

Pythia 160m

Developed by EleutherAI
Pythia-160M is a language model dedicated to interpretability research developed by EleutherAI. It belongs to the 160M parameter scale version in the Pythia suite and is based on the Transformer architecture, trained on the Pile dataset.
Downloads 163.75k
Release Time : 2/8/2023

Model Overview

Pythia-160M is a causal language model designed specifically for interpretability research of large language models, providing 154 training checkpoints to facilitate the analysis of the model's evolution process.

Model Features

Support for interpretability research
Provides 154 training checkpoints (including initial and intermediate states) to support the analysis of the model's behavioral evolution
Strict training control
All Pythia models are trained with the exact same data order to ensure experimental comparability
Performance optimization
Uses Flash Attention technology to improve training efficiency

Model Capabilities

English text generation
Analysis of language model behavior
Text continuation

Use Cases

Academic research
Interpretability research of language models
Utilize the multi-checkpoint feature to analyze the behavioral changes during the model's learning process
Can track the formation process of specific capabilities or biases
Research on model scaling laws
Explore the impact of the number of parameters by comparing models of different scales in the Pythia suite
Featured Recommended AI Models
AIbase
Empowering the Future, Your AI Solution Knowledge Base
© 2025AIbase