Fairseq-dense-2.7B Open-source Language Model - Free to Use for Diverse Applications like Text Processing

Fairseq Dense 2.7B

Developed by KoboldAI

A converted version of the 2.7 billion parameter dense model based on the paper 'Efficient Large-scale Language Modeling with Mixture of Experts' by Artetxe et al.

Large Language Model

Transformers

English#Large-scale Language Model #Mixture of Experts Architecture #Multi-task Evaluation

Downloads 18

Release Time : 3/2/2022

Model Overview

This model is a Hugging Face transformers-compatible version of the original dense 2.7 billion parameter model, focusing on large-scale language modeling tasks.

Model Features

Large-scale Parameters

Dense model architecture with 2.7 billion parameters, providing powerful language modeling capabilities.

Hugging Face Compatibility

Converted to Hugging Face transformers-compatible format for easy integration and use.

Efficient Modeling

Implements efficient large-scale language modeling using the Mixture of Experts approach.

Model Capabilities

Text Generation

Language Understanding

Contextual Learning

Use Cases

Natural Language Processing

Open-domain Dialogue

Can be used to build open-domain dialogue systems.

Text Completion

Generates coherent text completions based on context.

Property	Details
Avg.	33.67
ARC (25-shot)	33.79
HellaSwag (10-shot)	65.74
MMLU (5-shot)	26.44
TruthfulQA (0-shot)	34.57
Winogrande (5-shot)	63.93
GSM8K (5-shot)	0.0
DROP (3-shot)	11.24

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご

Fairseq Dense 2.7B

Model Overview

Model Features

Model Capabilities

Use Cases

🚀 Hugging Face-compatible Dense 2.7B-parameter Model

🚀 Quick Start

📚 Documentation

Open LLM Leaderboard Evaluation Results