Vntl Llama3 8b V2 Imatrix Gguf

🚀 imatrix quants of lmg-anon/vntl-llama3-8b-v2-hf

This project uses a multilingual fork of Bartowski's imatrix dataset to perform imatrix quants on lmg-anon/vntl-llama3-8b-v2-hf, aiming to improve the performance of LLMs in translating Japanese visual novels to English.

🚀 Quick Start

This is a LLaMA 3 Youko qlora fine - tune, created using a new version of the VNTL dataset. The goal is to enhance the performance of large language models (LLMs) in translating Japanese visual novels to English.

✨ Features

• Improved Performance: Rebuilt and expanded the VNTL dataset from scratch. It shows better accuracy and stability compared to the previous version, making far fewer mistakes even at high temperatures.
• Prompt Format Update: Switched to the default LLaMA3 prompt format, addressing the issues users had with the custom one.
• Multi - line Translation Support: Added proper support for multi - line translations, while the old version only handled single lines.
• Higher Translation Accuracy: Overall, it provides better translation accuracy, though the translations are more literal compared to the previous version.

🔧 Technical Details

Notes

For this new version of VNTL 8B, the VNTL's dataset has been rebuilt and expanded. It performs well, outperforming the previous version in accuracy and stability. It makes fewer mistakes even at high temperatures, but it's still recommended to use temperature 0 for the best accuracy.

Sampling Recommendations

For optimal results, it's highly recommended to use neutral sampling parameters (temperature 0 with no repetition penalty) when using this model.

Training Details

This fine - tune was done using similar hyperparameters as the previous version. The only difference is the dataset, which is brand - new.

Translation Prompt

This fine - tune uses the LLaMA 3 prompt format. Here is an example prompt for translation:

<|begin_of_text|><|start_header_id|>Metadata<|end_header_id|>

[character] Name: Uryuu Shingo (瓜生 新吾) | Gender: Male | Aliases: Onii-chan (お兄ちゃん)
[character] Name: Uryuu Sakuno (瓜生 桜乃) | Gender: Female<|eot_id|><|start_header_id|>Japanese<|end_header_id|>

[桜乃]: 『……ごめん』<|eot_id|><|start_header_id|>English<|end_header_id|>

[Sakuno]: 『... Sorry.』<|eot_id|><|start_header_id|>Japanese<|end_header_id|>

[新吾]: 「ううん、こう言っちゃなんだけど、迷子でよかったよ。桜乃は可愛いから、いろいろ心配しちゃってたんだぞ俺」<|eot_id|><|start_header_id|>English<|end_header_id|>

[Shingo]: "Nah, I know it’s weird to say this, but I’m glad you got lost. You’re so cute, Sakuno, so I was really worried about you."<|eot_id|>

The generated translation for that prompt, with temperature 0, is:

[Shingo]: "Nah, I know it’s weird to say this, but I’m glad you got lost. You’re so cute, Sakuno, so I was really worried about you."

Trivia

The Metadata section isn't limited to character information. You can also add trivia and teach the model the correct way to pronounce words it struggles with.

Here's an example:

<|begin_of_text|><|start_header_id|>Metadata<|end_header_id|>

[character] Name: Uryuu Shingo (瓜生 新吾) | Gender: Male | Aliases: Onii-chan (お兄ちゃん)
[character] Name: Uryuu Sakuno (瓜生 桜乃) | Gender: Female
[element] Name: Murasamemaru (叢雨丸) | Type: Quality<|eot_id|><|start_header_id|>Japanese<|end_header_id|>

[桜乃]: 『……ごめん』<|eot_id|><|start_header_id|>English<|end_header_id|>

[Sakuno]: 『... Sorry.』<|eot_id|><|start_header_id|>Japanese<|end_header_id|>

[新吾]: 「ううん、こう言っちゃなんだけど、迷子でよかったよ。桜乃は叢雨丸いから、いろいろ心配しちゃってたんだぞ俺」<|eot_id|><|start_header_id|>English<|end_header_id|>

The generated translation for that prompt, with temperature 0, is:

[Shingo]: "Nah, I know it’s not the best thing to say, but I’m glad you got lost. Sakuno’s Murasamemaru, so I was really worried about you, you know?"