Irbis-7b-v0.1 Open-source Language Model - Optimized for Kazakh to Enhance Language Processing Quality

Irbis 7b V0.1

Developed by Gen2B

Irbis-7B is a Transformer-based language model specifically optimized for Kazakh language, significantly improving Kazakh language processing quality

Large Language Model

Transformers

Supports Multiple LanguagesOpen Source License:MIT #Kazakh language optimization #Extended tokenizer #Multilingual generation

Downloads 164

Release Time : 3/31/2024

Model Overview

Irbis-7B is a large language model focused on Kazakh language processing, excelling in Kazakh text generation and processing tasks through enhanced tokenizer and pre-training optimization

Model Features

Enhanced tokenizer

Tokenizer vocabulary expanded from 32K to 60K, including more Kazakh words, significantly improving Kazakh tokenization efficiency

Kazakh language optimization

Pre-trained with 20GB of predominantly Kazakh text data, significantly outperforming other open-source models in Kazakh language processing quality

Performance optimization

New tokenizer increases text generation speed by 3-4 times while optimizing context window filling efficiency

Model Capabilities

Kazakh text generation

Multilingual text processing

Context understanding

Use Cases

Education

Kazakh language learning assistance

Helps students generate Kazakh language learning materials and exercises

Improves learning efficiency

Content creation

Kazakh content generation

Automatically generates Kazakh articles, stories, and other creative content

Accelerates content creation process

🚀 Irbis-7B

Irbis-7B is a language model based on the Transformer architecture, adapted for the Kazakh language. It offers enhanced tokenization and better performance on Kazakh text compared to other open - source models.

🚀 Quick Start

✨ Features

Enhanced Tokenizer: The tokenizer's vocabulary has been expanded from 32k to 60k tokens, including more Kazakh words. This improvement enhances the tokenization efficiency for the Kazakh language.
Pre - training: The model was pre - trained on 20 GB of mainly Kazakh and a small amount of Russian texts to adapt to the new tokenizer.

As a result, the model demonstrates significantly better performance on the Kazakh language compared to other open - source models. Thanks to the new tokenizer, the text generation speed has increased by 3 - 4 times, and the filling of the context window has also been optimized.

You can read more details in this article.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
import torch

model_name = "Gen2B/Irbis-7b-v0.1"
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    return_dict=True,
    torch_dtype=torch.float16,
    device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Сұрақ: Шөп неге жасыл?\nЖауап: "

input_ids = tokenizer([prompt], return_tensors = "pt")["input_ids"].to("cuda")

generation_config = GenerationConfig(
    temperature=0.6,
    repetition_penalty=1.15
)
print("Generating...")
generation_output = model.generate(
    input_ids=input_ids,
    generation_config=generation_config,
    return_dict_in_generate=True,
    output_scores=True,
    max_new_tokens=2048,
    pad_token_id=tokenizer.eos_token_id,
)
for s in generation_output.sequences:
    print(tokenizer.decode(s)) # Өсімдіктер ауасыз өмір сүре алмайды, сондықтан олар жасыл түсті болады.

📚 Documentation

No detailed documentation content other than the above information is provided in the original document, so this section is skipped.

🔧 Technical Details

No in - depth technical details are provided in the original document, so this section is skipped.

📄 License

The model is released under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご