Llm Jp 3.1 1.8b

🚀 llm-jp-3.1-1.8b

LLM-jp-3.1 is a series of large language models developed by the Research and Development Center for Large Language Models at the National Institute of Informatics. It builds upon the LLM-jp-3 series, incorporating mid-training (instruction pre-training), which significantly enhances instruction-following capabilities compared to the original LLM-jp-3 models. This repository provides the llm-jp-3.1-1.8b model.

🚀 Quick Start

This section provides a high - level overview of getting started with the llm - jp - 3.1 - 1.8b model. You can find detailed usage examples and requirements below.

✨ Features

• Enhanced Instruction - Following: Incorporating mid - training (instruction pre - training), the LLM - jp - 3.1 models have significantly improved instruction - following capabilities compared to the original LLM - jp - 3 models.
• Multiple Language Support: Trained on a diverse set of datasets including Japanese, English, Code, Chinese, and Korean.
• Fine - Tuning Options: Fine - tuned with supervised fine - tuning and further aligned with Direct Preference Optimization.

📦 Installation

Required Libraries and Their Versions

• torch>=2.3.0
• transformers>=4.40.1
• tokenizers>=0.19.1
• accelerate>=0.29.3
• flash - attn>=2.5.8

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("llm-jp/llm-jp-3.1-1.8b")
model = AutoModelForCausalLM.from_pretrained("llm-jp/llm-jp-3.1-1.8b", device_map="auto", torch_dtype=torch.bfloat16)
text = "自然言語処理とは何か"
tokenized_input = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(model.device)
with torch.no_grad():
    output = model.generate(
        tokenized_input,
        max_new_tokens=100,
        do_sample=True,
        top_p=0.95,
        temperature=0.7,
        repetition_penalty=1.05,
    )[0]
print(tokenizer.decode(output))

📚 Documentation

Model Details

Architectures

Dense model:

MoE model:

Tokenizer

The tokenizer of this model is based on huggingface/tokenizers Unigram byte - fallback model. The vocabulary entries were converted from [llm - jp - tokenizer v3.0](https://github.com/llm - jp/llm - jp - tokenizer/releases/tag/v3.0b2). Please refer to [README.md](https://github.com/llm - jp/llm - jp - tokenizer) of llm - jp - tokenizer for details on the vocabulary construction procedure.

Datasets

Pre - training

The models have been pre - trained using a blend of the following datasets:

Mid - training

In the LLM - jp - 3.1 series, continuous pre - training was performed based on [Instruction Pre - Training](https://aclanthology.org/2024.emnlp - main.148/). Approximately 90B tokens of instruction - response data were prepared and mixed with pre - training datasets, conducting continuous pre - training on a total of 400B tokens. Each model was initialized from existing checkpoints and underwent continuous instruction pre - training. Since the LLM - jp - 3 series was originally pre - trained on 2.1T tokens, the total pre - training token count amounts to 2.5T tokens.

Post - training

The pre - trained checkpoint was fine - tuned with supervised fine - tuning and further aligned with Direct Preference Optimization.

Supervised Fine - Tuning

Direct Preference Optimization

For Direct Preference Optimization (DPO), rejection sampling was adopted. Prompts were sampled from the dataset used in SFT, and multiple responses were generated for each prompt. These responses were then scored (by [Qwen/Qwen2.5 - 32B - Instruct](https://huggingface.co/Qwen/Qwen2.5 - 32B - Instruct)), and DPO was performed by treating high - scoring responses as positive examples and low - scoring responses as negative examples. DPO was conducted in two stages, and in the second stage, [ac - self - inst](https://huggingface.co/datasets/llm - jp/ac - self - inst), a Japanese preference dataset focused on safety, was additionally used.

Evaluation

MT Bench (Japanese and English)

The models were evaluated using gpt - 4o - 2024 - 08 - 06. The scores represent the average values obtained from three rounds of inference and evaluation.

AnswerCarefully - Eval

[AnswerCarefully - Eval](https://www.anlp.jp/proceedings/annual_meeting/2025/pdf_dir/Q4 - 19.pdf) assesses the safety of Japanese language model outputs using the LLM - as - a - Judge approach, based on the test set from [llm - jp/AnswerCarefully](https://huggingface.co/datasets/llm - jp/AnswerCarefully). The models were evaluated using gpt - 4o - 2024 - 08 - 06. The scores represent the average values obtained from three rounds of inference and evaluation.

🔧 Technical Details

The models are based on a Transformer - based architecture. Checkpoints are in the Hugging Face Transformers format. The training process involves pre - training, mid - training (instruction pre - training), and post - training (supervised fine - tuning and Direct Preference Optimization).

📄 License

[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE - 2.0)

🚫 Risks and Limitations

The models released here are in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations.

📧 Send Questions to

llm - jp(at)nii.ac.jp

👥 Model Card Authors

The names are listed in alphabetical order.

Hirokazu Kiyomaru and Takashi Kodama.