Turkcell-LLM-7b-v1 Open Source Turkish Large Language Model - Free Deployment for Practical Conversations and Text Generation

Turkcell LLM 7b V1

Developed by TURKCELL

A Turkish large language model based on the Mistral 7B architecture, trained on 5 billion Turkish tokens and fine-tuned with instructions

Large Language Model

Transformers

OtherOpen Source License:Apache-2.0 #Turkish Large Language Model #DORA-LORA Training #Mistral Architecture Extension

Downloads 3,771

Release Time : 4/4/2024

Model Overview

This is a 7B-parameter large language model specifically optimized for Turkish, based on the Mistral architecture. It is pre-trained using the DORA method and fine-tuned with the LORA method, suitable for Turkish text understanding and generation tasks

Model Features

Turkish Optimization

Specifically optimized for Turkish with tokenizer extension and training data optimization

Two-Stage Training

Pre-trained using the DORA method followed by instruction fine-tuning with the LORA method

Efficient Fine-Tuning

Parameter-efficient fine-tuning using the LORA method, reducing computational resource requirements

Model Capabilities

Turkish text understanding

Turkish text generation

Turkish question answering

Turkish instruction following

Use Cases

Customer Service

Turkish Customer Service Chatbot

Used to handle Turkish customer inquiries

Provides a smooth and natural Turkish interaction experience

Content Generation

Turkish Content Creation

Generates Turkish articles, reports, etc.

Produces high-quality text that conforms to Turkish language conventions

🚀 Turkcell-LLM-7b-v1

This model is an extended Turkish Large Language Model (LLM) based on Mistral. It offers high - quality language processing capabilities for Turkish, trained on a large - scale Turkish dataset.

🚀 Quick Start

This model is an extended version of a Mistral - based Large Language Model (LLM) for Turkish. It was trained on a cleaned Turkish raw dataset containing 5 billion tokens. The training process involved using the DORA method initially. Following this, we utilized Turkish instruction sets created from various open - source and internal resources for fine - tuning with the LORA method.

✨ Features

Base Model: Mistral 7B based LLM
Tokenizer Extension: Specifically extended for Turkish
Training Dataset: Cleaned Turkish raw data with 5 billion tokens, custom Turkish instruction sets
Training Method: Initially with DORA, followed by fine - tuning with LORA

🔧 Technical Details

DORA Configuration

Property	Details
`lora_alpha`	128
`lora_dropout`	0.05
`r`	64
`target_modules`	"all - linear"

LORA Fine - Tuning Configuration

Property	Details
`lora_alpha`	128
`lora_dropout`	0.05
`r`	256
`target_modules`	"all - linear"

💻 Usage Examples

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained("TURKCELL/Turkcell-LLM-7b-v1")
tokenizer = AutoTokenizer.from_pretrained("TURKCELL/Turkcell-LLM-7b-v1")

messages = [
    {"role": "user", "content": "Türkiye'nin başkenti neresidir?"},
]

encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")

eos_token = tokenizer("<|im_end|>",add_special_tokens=False)["input_ids"][0]

model_inputs = encodeds.to(device)
model.to(device)

generated_ids = model.generate(model_inputs, 
                               max_new_tokens=1024, 
                               do_sample=True, 
                               eos_token_id=eos_token)
                               
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])

📄 License

This project is licensed under the Apache 2.0 License.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご