M

M2 BERT 8k Retrieval Encoder V1

Developed by hazyresearch
M2-BERT-8K is an 80-million-parameter long-context retrieval model based on the architecture proposed in the paper 'Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT'.
Downloads 52
Release Time : 5/22/2024

Model Overview

M2-BERT-8K is a BERT variant specifically designed for long-context retrieval tasks, supporting sequences up to 8192 tokens in length and capable of generating 768-dimensional embedding vectors for retrieval tasks.

Model Features

Long Context Support
Supports sequences up to 8192 tokens, making it suitable for long-document retrieval tasks.
Efficient Retrieval
Generates 768-dimensional embedding vectors optimized for retrieval efficiency.
Custom Architecture
A BERT variant improved with the Monarch Mixer architecture.

Model Capabilities

Text Embedding Generation
Long Document Retrieval
Masked Language Modeling

Use Cases

Information Retrieval
Document Retrieval System
Building a retrieval system that supports long documents.
Capable of effectively processing documents up to 8192 tokens in length.
Featured Recommended AI Models
AIbase
Empowering the Future, Your AI Solution Knowledge Base
© 2025AIbase