B

Bigbird Base Japanese

Developed by nlp-waseda
A Japanese BigBird model pre-trained on Japanese Wikipedia, CC-100, and OSCAR datasets, suitable for long sequence processing tasks.
Downloads 38
Release Time : 6/3/2023

Model Overview

This is a Japanese BigBird base model pre-trained on Japanese Wikipedia, the Japanese portion of CC-100, and the Japanese portion of OSCAR, primarily used for masked language modeling and downstream task fine-tuning.

Model Features

Long Sequence Processing
Supports sequences up to 4096 tokens in length, ideal for long-text tasks.
Efficient Pre-training
Utilizes 16 NVIDIA A100 GPUs with DeepSpeed optimization for high training efficiency.
Professional Tokenization
Input text must be tokenized using Juman++ to ensure linguistic processing accuracy.

Model Capabilities

Japanese Text Understanding
Masked Language Modeling
Long Sequence Processing
Downstream Task Fine-tuning

Use Cases

Text Classification
Sentiment Analysis
Classify sentiment polarity in Japanese text
Achieves 0.959 accuracy on the MARC-ja task
Semantic Similarity
Sentence Similarity Calculation
Compute semantic similarity between two Japanese sentences
Achieves a Pearson coefficient of 0.888 on the JSTS task
Question Answering
Japanese Reading Comprehension
Answer Japanese questions based on given text
Achieves an F1 score of 0.933 on the JSQuAD task
Featured Recommended AI Models
AIbase
Empowering the Future, Your AI Solution Knowledge Base
© 2025AIbase