Nekomata-14B-Instruction-GGUF Open-Source Model - Achieve Lightweight Inference with llama.cpp

Nekomata 14b Instruction Gguf

Developed by rinna

This model is the GGUF version of rinna/nekomata-14b-instruction, compatible with llama.cpp for lightweight inference.

Large Language Model Supports Multiple LanguagesOpen Source License:Other #Japanese-English Translation #Lightweight Inference #Instruction Fine-tuning

Downloads 89

Release Time : 12/19/2023

Model Overview

A quantized version based on rinna/nekomata-14b-instruction, suitable for Japanese and English text generation tasks.

Model Features

Lightweight Inference

Efficient lightweight inference achieved through GGUF format and llama.cpp.

Multilingual Support

Supports text generation tasks in both Japanese and English.

Quantization Optimization

Recommended to use GGUF q4_K_M for 4-bit quantization, balancing performance and resource consumption.

Model Capabilities

Japanese Text Generation

English Text Generation

Instruction Following

Translation Tasks

Use Cases

Text Generation

Japanese to English Translation

Translate Japanese text into English.

Generates fluent English translation results.

Instruction Following

Task Response Generation

Generate appropriate responses based on given instructions and inputs.

Generates text responses that meet the instruction requirements.

🚀 `rinna/nekomata-14b-instruction-gguf`

This is the GGUF version of rinna/nekomata-14b-instruction, enabling lightweight inference with llama.cpp.

🚀 Quick Start

The model is the GGUF version of rinna/nekomata-14b-instruction. It can be used with llama.cpp for lightweight inference.

Quantization of this model may cause stability issue in GPTQ, AWQ and GGUF q4_0. We recommend GGUF q4_K_M for 4-bit quantization.

See rinna/nekomata-14b-instruction for details about model architecture and data.

✨ Features

Contributors
Release date December 22, 2023

📦 Installation

See llama.cpp for more usage details.

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make

MODEL_PATH=/path/to/nekomata-14b-instruction-gguf/nekomata-14b-instruction.Q4_K_M.gguf
MAX_N_TOKENS=512
PROMPT_INSTRUCTION="次の日本語を英語に翻訳してください。"
PROMPT_INPUT="大規模言語モデル（だいきぼげんごモデル、英: large language model、LLM）は、多数のパラメータ（数千万から数十億）を持つ人工ニューラルネットワークで構成されるコンピュータ言語モデルで、膨大なラベルなしテキストを使用して自己教師あり学習または半教師あり学習によって訓練が行われる。"
PROMPT="以下は、タスクを説明する指示と、文脈のある入力の組み合わせです。要求を適切に満たす応答を書きなさい。\n\n### 指示:\n${PROMPT_INSTRUCTION}\n\n### 入力:\n${PROMPT_INPUT}\n\n### 応答:\n"

./main -m ${MODEL_PATH} -n ${MAX_N_TOKENS} -p "${PROMPT}"

📚 Documentation

Tokenization

Please refer to rinna/nekomata-14b for tokenization details.

How to cite

@misc{rinna-nekomata-14b-instruction-gguf,
    title = {rinna/nekomata-14b-instruction-gguf},
    author = {Wakatsuki, Toshiaki and Zhao, Tianyu and Sawada, Kei},
    url = {https://huggingface.co/rinna/nekomata-14b-instruction-gguf}
}

@inproceedings{sawada2024release,
    title = {Release of Pre-Trained Models for the {J}apanese Language},
    author = {Sawada, Kei and Zhao, Tianyu and Shing, Makoto and Mitsui, Kentaro and Kaga, Akio and Hono, Yukiya and Wakatsuki, Toshiaki and Mitsuda, Koh},
    booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
    month = {5},
    year = {2024},
    pages = {13898--13905},
    url = {https://aclanthology.org/2024.lrec-main.1213},
    note = {\url{https://arxiv.org/abs/2404.01657}}
}