nllb-200-distilled-350M_en-ko Open-source Translation Model - Lightweight Free English-to-Korean Translation, Runs on CPU

Nllb 200 Distilled 350M En Ko

Developed by dhtocks

This is a lightweight English-to-Korean translation model optimized from the NLLB-200 600M model, with only 350M parameters and can run on CPU.

Machine Translation

Transformers

Supports Multiple Languages#Lightweight translation #English-Korean mutual translation #CPU-friendly

Downloads 103

Release Time : 4/25/2024

Model Overview

This model focuses on the English-to-Korean translation task. By reducing the number of layers, it reduces the computational resource requirements and is suitable for users with limited resources.

Model Features

Lightweight model

With only 350M parameters, it is lighter than the original NLLB-200 600M model, reducing the computational resource requirements.

Runnable on CPU

It can run on CPU without mixed precision and quantization, suitable for users with limited resources.

Efficient inference

The inference time is 1.43 seconds on CPU and 0.24 seconds on GPU, suitable for real-time translation needs.

Model Capabilities

English-to-Korean translation

Running in low-resource environments

Use Cases

Translation applications

Real-time text translation

Translate English text into Korean in real-time, suitable for scenarios such as chatting and emails.

The translation quality reaches chrF(++) 24.6

Translation in low-resource environments

Run translation tasks on devices with limited computational resources, such as mobile devices or edge computing devices.

It can run efficiently on CPU

🚀 NLLB-200 Distilled-350M_en2ko

The NLLB-200 model demonstrated remarkable performance in translation tasks, especially in addressing challenges related to low-resource languages. However, running models of 600M or over 1B parameters remains a challenge for those with limited computing resources. This project presents a much smaller model specialized in English-to-Korean translation, which can even run on a CPU without mixed-precision or quantization.

🚀 Quick Start

The NLLB-200 model has shown excellent results in translation tasks and has helped to solve problems related to low-resource languages. However, it's still difficult for those with insufficient computing resources to run models of 600M or over 1B parameters. Therefore, a smaller model specialized in English-to-Korean translation has been developed, which can be run on a CPU (without mixed-precision and quantization).

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model = AutoModelForSeq2SeqLM.from_pretrained('dhtocks/nllb-200-distilled-350M_en-ko', forced_bos_token_id=256098)
tokenizer = AutoTokenizer.from_pretrained('dhtocks/nllb-200-distilled-350M_en-ko', src_lang='eng_Latn', tgt_lang='kor_Hang')

inputs = tokenizer('[YOUR_INPUT]', return_tensors="pt")
output = model.generate(**inputs)
print(tokenizer.decode(output[0]))

✨ Features

Specialized in English-to-Korean translation.
Can be run on a CPU without mixed-precision or quantization.

🔧 Technical Details

Model

Model: Based on NLLB-200 600M
- Parameters: 350,537,728 (350M)
- Encoder layers: Reduced from 12 to 3
- Decoder layers: Reduced from 12 to 3
- FFN dimension: 4096 (unchanged)
- Embed dimension: 1024 (unchanged)
- Vocab size: 256206 (unchanged)
License: CC-BY-NC

Data

Training Data: NLLB dataset
Evaluation Data: Flores-200 dataset

Metric

CPU: Intel (R) Xeon(R) CPU @ 2.20GHz (16 cores)
GPU: NVIDIA L4 24GB

Property	Details
Model Type	Based on NLLB-200 600M
Training Data	NLLB dataset
Evaluation Data	Flores-200 dataset
CPU	Intel (R) Xeon(R) CPU @ 2.20GHz (16 cores)
GPU	NVIDIA L4 24GB

	#Params	chrF(++)	GPU Inference time (s)	CPU Inference time (s)
NLLB-200 3.3B	3.3B	34.3	0.98 s	4.65 s
NLLB-200 1.3B	1.3B	32.1	0.89 s	2.46 s
NLLB-200 600M	600M	32	0.43 s	1.52 s
NLLB-200 350M (ours)	350M	24.6	0.24 s	1.43 s

📄 License

This project is licensed under CC-BY-NC-4.0.

📚 Documentation

Citation

@misc{,
  title={NLLB-200 distilled_350M_en-ko},
  author={Saechan Oh},
  year={2024}
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご