Hebrew-Gemma-11B-Instruct Open-Source Text Model - Free Deployment to Boost Hebrew Multi-Round Dialogue Interaction

Hebrew Gemma 11B Instruct

Developed by yam-peleg

A version fine-tuned with multi-turn dialogue datasets based on the Hebrew-Gemma-11B generative text model

Supports Multiple LanguagesOpen Source License:Other #Hebrew instruction fine-tuning #Multi-turn dialogue optimization #Bilingual generation model

Downloads 2,670

Release Time : 3/6/2024

Model Overview

This model is further pre-trained on the gemma-7b foundation, scaled to a larger size, and additionally trained on 3 billion tokens of English and Hebrew text data, specifically optimized for Hebrew and English instruction response tasks

Model Features

Bilingual support

Supports both English and Hebrew instruction responses and text generation

Instruction optimization

Specially fine-tuned with multi-turn dialogue datasets to optimize instruction response capabilities

Large-scale pre-training

Extended from gemma-7b with additional training on 3 billion bilingual text tokens

Model Capabilities

Text generation

Instruction response

Code generation

Multi-turn dialogue

Use Cases

Programming assistance

Code generation

Generate programming code based on natural language instructions

Can generate code snippets in multiple languages including Python

Language services

Hebrew Q&A

Answer various questions in Hebrew

🚀 Hebrew-Gemma-11B-Instruct

The Hebrew-Gemma-11B-Instruct is an instruct fine - tuned Large Language Model (LLM). It is based on the Hebrew-Gemma-11B generative text model, fine - tuned using various conversation datasets. It is a continued pretrain of gemma - 7b, extended to a larger scale and trained on an additional 3B tokens of both English and Hebrew text data.

🚀 Quick Start

Base Models

07.03.2024: Hebrew-Gemma-11B
16.03.2024: Hebrew-Gemma-11B-V2

Instruct Models

07.03.2024: Hebrew-Gemma-11B-Instruct

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Hebrew-Gemma-11B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="cuda")

chat = [
    { "role": "user", "content": "כתוב קוד פשוט בפייתון שמדפיס למסך את התאריך של היום" },
]
prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)

Instruction format

This format must be strictly respected, otherwise the model will generate sub - optimal outputs.

<bos><start_of_turn>user
Write a hello world program<end_of_turn>
<start_of_turn>model
Here is a simple hellow world program<end_of_turn><eos>

The conversation starts with <bos>.
Each turn is preceded by a <start_of_turn> delimiter and then the role of the entity (user or model).
Turns finish with the <end_of_turn> token.
Conversation finish with the <eos> token.

You can follow this format to build the prompt manually, if you need to do it without the tokenizer's chat template.

📚 Documentation

Terms of Use

As an extention of Gemma - 7B, this model is subject to the original license and terms of use by Google.

Notice

Hebrew-Gemma-11B is a pretrained base model and therefore does not have any moderation mechanisms.

👥 Authors

Trained by Yam Peleg.
In collaboration with Jonathan Rouach and Arjeo, inc.

📄 License

This model is under the gemma-terms-of-use.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご