C2S-Scale-Pythia-1b-pt Open-source Model - Empowering Single-cell and Multi-cell Analysis Tasks!

C2S Scale Pythia 1b Pt

Developed by vandijklab

A model pre-trained on the Pythia-1b architecture and fine-tuned on single-cell RNA sequencing data via the Cell2Sentence framework, suitable for various single-cell and multicellular analysis tasks.

Molecular Model

Transformers

English#Single-cell RNA analysis #Gene sequence generation #Multicellular modeling

Downloads 156

Release Time : 4/16/2025

Model Overview

This model employs the Cell2Sentence method to transform scRNA-seq data into ordered sequences of gene names, enabling large language models to adapt to single-cell biology research and perform a wide range of single-cell and multicellular tasks.

Model Features

Cell2Sentence Framework

Transforms scRNA-seq data into ordered sequences of gene names based on expression levels, enabling LLMs to adapt to single-cell biology research.

Large-scale Training Data

Training data covers over 800 single-cell RNA sequencing datasets, encompassing more than 57 million human and mouse cells.

Multitasking Capability

Capable of performing various tasks, including single-cell and multicellular analysis, gene set analysis, and more.

Extended Context Length

Extends the default Pythia model's context length to 8192 tokens via rotary position embedding technology.

Model Capabilities

Unconditional single-cell generation

Cell type prediction

Cell type conditional generation

Unconditional multicellular generation

Tissue origin prediction

Tissue conditional multicellular generation

Cell type conditional multicellular generation

Multicellular to summary generation

Summary to multicellular generation

Gene set name to gene generation

Gene to gene set name generation

Use Cases

Biomedical Research

Single-cell RNA Sequencing Data Analysis

Analyze single-cell RNA sequencing data to predict cell types or states.

Cell Generation

Generate simulated single-cell or multicellular data based on specific conditions.

Gene Set Analysis

Generate gene lists from gene set names or vice versa.

🚀 C2S-Scale-1B Pretrained Model

A versatile model based on Pythia-1b, fine - tuned for single - and multi - cell analyses in single - cell biology.

🚀 Quick Start

This README provides an overview of the C2S - Scale - 1B pretrained model, its training data, tasks it can perform, and relevant links.

✨ Features

Based on the Pythia - 1b architecture developed by EleutherAI.
Fine - tuned using the Cell2Sentence (C2S) framework on scRNA - seq datasets.
Capable of performing a wide range of single - and multi - cell tasks.

📦 Installation

The original document does not provide installation steps, so this section is skipped.

💻 Usage Examples

The original document does not provide code examples, so this section is skipped.

📚 Documentation

Overview

This is the C2S - Scale - 1B pretrained model, based on the Pythia - 1b architecture developed by EleutherAI, fine - tuned using the Cell2Sentence (C2S) framework on a wide array of single - cell RNA sequencing (scRNA - seq) datasets from CellxGene and the Human Cell Atlas. Cell2Sentence is a cutting - edge method that adapts large language models (LLMs) to single - cell biology by converting scRNA - seq data into "cell sentences" — ordered sequences of gene names based on expression levels. This model has been trained to perform a broad range of single - and multi - cell tasks, making it a versatile tool for various single - cell and multi - cell analyses.

Training Data

This model was trained on over 57 million human and mouse cells gathered from over 800 single - cell RNA sequencing datasets from CellxGene and the Human Cell Atlas. This dataset covers a broad range of cell types and conditions from multiple tissues in both human and mouse.

This model was trained with a variable number of genes per cell sentence, with a maximum context length of 8192 tokens. The context length of the default Pythia model was extended using rotary positional embeddings prior to C2S training.

Cells: For multi - cell samples, each training sample contained between 5 and 20 cells, with the same number of genes for each of the cells in the same sample.
Genes: For single - cell samples, each cell sentence contained between 100 and 2048 genes. For multi - cell samples, each cell sentence per cell contained between 100 and 400 genes.

Tasks

This model is designed for the following tasks:

Single - Cell Tasks

Unconditional single - cell generation: Generate single cell sentences unconditionally.
Cell type prediction: Predict the cell type of a given single cell.
Cell type - conditioned generation: Generate a single cell sentence conditioned on a specific cell type.

Multi - Cell Tasks

Unconditional multi - cell generation: Generate multiple cell sentences unconditionally.
Tissue prediction: Predict the tissue of origin for a group of cells.
Cell type prediction: Predict the cell type for each cell in a group of multiple cells.
Tissue - conditioned multi - cell generation: Generate multiple cell sentences conditioned on a specific tissue.
Cell type - conditioned multi - cell generation: Generate multiple cell sentences conditioned on the cell type of each individual cell.
Multi - cells to abstract: Generate a research paper abstract based on the provided multi - cell sentences.
Abstract to multi - cells: Generate multiple cell sentences based on a given research paper abstract.

Gene Set Tasks

Gene set name to genes: Generate an alphabetical list of genes given a gene set name.
Genes to gene set name: Generate the name of a gene set given an alphabetical list of genes.

🔧 Technical Details

The model is based on the Pythia - 1b architecture. The context length of the default Pythia model was extended using rotary positional embeddings before fine - tuning with the C2S framework. The model was trained on a large dataset of over 57 million human and mouse cells from scRNA - seq datasets, with specific settings for the number of genes per cell sentence and context length.

📄 License

The model is licensed under CC - BY - NC - ND - 4.0.

Information Table

Property	Details
Model Type	C2S - Scale - 1B pretrained model
Base Model	EleutherAI/pythia - 1b
Library Name	transformers
Tags	biology, scRNAseq
License	CC - BY - NC - ND - 4.0
Training Data	Over 57 million human and mouse cells from over 800 single - cell RNA sequencing datasets from CellxGene and the Human Cell Atlas

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご