# Large-scale pre-training

Qwen3-8B-Base is the latest generation of Tongyi's large model series, with 8.2 billion parameters and support for 119 languages. It is suitable for a variety of natural language processing tasks.

LHM is a feedforward model that can reconstruct an animatable 3D human body from a single image within seconds. Trained with image reconstruction loss on a large-scale video dataset, our model demonstrates strong generalization capabilities across diverse real-world scenarios.

Japanese wav2vec2.0 Large model pre-trained on large-scale Japanese TV broadcast audio data

A Japanese HuBERT Large model pre-trained on 62,215 hours of Japanese TV broadcast audio data for speech feature extraction

Japanese speech feature extraction model pre-trained on 62,215 hours of Japanese TV broadcast audio data

VideoMAEv2-Base is a self-supervised video feature extraction model that employs a dual masking mechanism pre-trained on the UnlabeldHybrid-1M dataset.

SAM2 model based on HieraDet image encoder, focusing on efficient image feature extraction

EVA-CLIP is a vision-language model based on the CLIP architecture, supporting zero-shot image classification tasks.

A dual-purpose vision-language model trained on the MetaCLIP-2.5B dataset, supporting zero-shot image classification tasks

A dual-framework compatible vision model trained on MetaCLIP-2.5B dataset, supporting zero-shot image classification tasks

Vision Transformer model trained on MetaCLIP-400M dataset, supporting zero-shot image classification tasks

A dual-framework compatible vision model trained on the MetaCLIP-2.5B dataset, supporting both OpenCLIP and timm frameworks

A vision Transformer model trained on the MetaCLIP-2.5B dataset, compatible with both open_clip and timm frameworks

A multimodal table recognition model based on InternVL2-1B, supporting conversion of table images to LaTeX/HTML/Markdown formats

EuroLLM-1.7B is the first pre-trained model in the EuroLLM series, with multilingual processing capabilities, capable of understanding and generating text in multiple European and other related languages.

A text generation model trained on the SlimPajama-627B dataset, utilizing the retina network architecture.

InternViT-6B-224px is a foundational vision model focused on image feature extraction, with 5903 million parameters, supporting image inputs of 224x224 pixels.

CLIPA-v2 model, focusing on zero-shot image classification tasks, achieving efficient visual representation learning through contrastive image-text training

CLIPA-v2 model, an efficient contrastive image-text model, focused on zero-shot image classification tasks

CLIPA-v2 model, an efficient contrastive vision-language model designed for zero-shot image classification tasks.

MetaCLIP is a vision-language model trained on CommonCrawl data for constructing shared image-text embedding spaces.

MetaCLIP is a vision-language model trained on CommonCrawl data for constructing shared image-text embedding spaces

MetaCLIP is a vision-language model trained on 2.5 billion data points from CommonCrawl (CC) to construct a shared image-text embedding space.

This is an unsupervised learning-based Japanese sentence embedding model, specifically designed to generate high-quality Japanese sentence embeddings.

The Nucleotide Transformer is a set of foundational language models pre-trained on whole-genome DNA sequences, integrating genomic data from over 3,200 human genomes and 850 diverse species.

This is a large automatic speech recognition (ASR) model based on the FastConformer architecture, specifically designed for transcribing English speech into text.

SAM is a visual model capable of generating high-quality object masks from input points or bounding boxes, with zero-shot transfer capability.

SAM is a vision model capable of generating high-quality object masks from input prompts (such as points or boxes), supporting zero-shot segmentation tasks

mGPT 13B is a multi-language language model that supports 61 languages, covering 25 language families. It is trained on 600GB of text data and has powerful multi-language processing capabilities.

SAM is a vision model capable of generating high-quality object masks based on input prompts, supporting zero-shot transfer to new tasks

A DNA sequence analysis model pre-trained on genomes from 850 species, supporting tasks such as molecular phenotype prediction

A Russian pre-trained language model based on the T5 architecture, employing a mixed training strategy with 7 denoisers similar to UL2, supporting various text generation tasks.

DeBERTa-base is a pre-trained bidirectional encoder for Russian, mainly used for processing Russian text tasks.

Large-scale ConvNeXt-Large CLIP model trained on LAION-2B dataset, supporting zero-shot image classification and image-text retrieval tasks

A T5 model trained on the large-scale cleaned German Common Crawl corpus (GC4), specializing in German natural language processing tasks.

Russian pre-trained language model based on T5 architecture, employing a UL2-like mixed training strategy with 7 denoising tasks, 1.7 billion parameters

A Russian BERT model jointly trained by Sber AI team and MLSA Lab at Moscow State University's AI Institute, specializing in scientific text processing

BiT is a simple method for scaling up pre-training of ResNet-like architectures, bringing significant improvements in transfer learning.

GPT-2 is an autoregressive language model based on the Transformer architecture. It is pre-trained on a large-scale English corpus through self-supervised learning and excels at text generation tasks.

Named entity recognition model fine-tuned on the English CoNLL-2003 dataset based on the XLM-RoBERTa-large model