Zion_Alpha_Instruction_Tuned_SLERP Open-source Language Model - Skilled in Hebrew Sentiment Analysis and Reasoning

Zion Alpha Instruction Tuned SLERP

Developed by SicariusSicariiStuff

Zion_Alpha_Instruction_Tuned_SLERP is an innovative language model trained on Hebrew, which performs excellently in tasks such as sentiment analysis and natural language inference.

Large Language Model

Transformers

Supports Multiple LanguagesOpen Source License:Apache-2.0 #Hebrew NLP #Leading in sentiment analysis #Natural language inference

Downloads 3,180

Release Time : 6/9/2024

Model Overview

The world's first true Hebrew model, based on the Mistral 7B architecture, fine-tuned for specific tasks, and has set multiple world records in Hebrew NLP tasks.

Model Features

Pioneering Hebrew model

The world's first true Hebrew model, opening up a new path for Hebrew natural language processing.

Excellent sentiment analysis ability

Achieved the world's highest score of 70.3 in the sentiment analysis task of Hebrew large language models, surpassing the 35B model that is five times its size.

Powerful natural language inference ability

Set a world - high score record of 84.05 in the SNLI task of Hebrew open large language models on Hugging Face.

Optimized 7B architecture

Based on the Mistral 7B architecture, it shows great optimization potential in Hebrew tasks.

Model Capabilities

Hebrew text generation

Sentiment analysis

Natural language inference

English - Hebrew translation

Use Cases

Language processing

Hebrew sentiment analysis

Analyze the sentiment tendency of Hebrew text

Achieved the world's highest score of 70.3 in the evaluation on June 10, 2024

Natural language inference

Perform natural language inference tasks in Hebrew

Set a world - high score record of 84.05 in the SNLI task

Bilingual translation

Translation between English and Hebrew

Performs better than the Mistral model in long - text translation

Creative writing

Hebrew story generation

Generate creative Hebrew texts and stories

Demonstrates excellent creative ability in a zero - shot setting

🚀 Zion_Alpha_Instruction_Tuned_SLERP

Zion_Alpha_Instruction_Tuned_SLERP is a remarkable Hebrew model. It has achieved outstanding results in sentiment analysis and SNLI scores, demonstrating significant potential in the Hebrew language processing field.

🚀 Quick Start

This section provides a quick overview of how to start using the model. However, specific installation and usage steps are not provided in the original document.

✨ Features

World's First Hebrew Model: Zion_Alpha is the world's first REAL Hebrew model.
High Performance: On June 10th, 2024, it achieved the highest sentiment analysis score in the world for Hebrew LLMs, reaching an impressive 70.3, surpassing even a 35B model five times its size.
Optimization Potential: Although the current SLERP merge model has a lower SNLI score, it still shows significant untapped potential in optimizing the training process, indicating that 7B models can deliver far more performance in Hebrew than previously thought.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

English to Hebrew Example

Zion_Alpha English to Hebrew example

Hebrew to English Example

Zion_Alpha Hebrew to English example

📚 Documentation

Model Details

Zion_Alpha is the first REAL Hebrew model in the world. This version was fine - tuned for tasks using SOTA techniques and the author's insights from years of experience.

Future Plans

The previous LLM, Zion_Alpha, set a world record on Hugging Face with a SNLI score of 84.05 for Hebrew open LLMs. The current SLERP merge model achieved a lower SNLI score but surprised everyone with the highest sentiment analysis score of 70.3. This shows significant potential in optimizing the training process. This will be the author's last Hebrew model for a while as they have other adventures to pursue.

Looking for Sponsors

Since all work is done on - premises and constrained by current hardware, the author would greatly appreciate support in acquiring an A6000 to train larger models faster.

Contact Details

If you'd like to reach out to the author, you can email at sicariussicariistuff@gmail.com. Note that this email might receive more messages than the author can handle.

Versions and QUANTS

Base model: FP16
Instruction tuned: FP16 | GGUF

Model architecture

Based on Mistral 7B. The tokenizer was not altered.

Recommended Prompt Setting

temperature: 1
top_p: 1
top_k: 1
typical_p: 1
min_p: 1
repetition_penalty: 1

Recommended Instruction Template

{%- for message in messages %}
    {%- if message['role'] == 'system' -%}
        {{- message['content'] -}}
    {%- else -%}
        {%- if message['role'] == 'user' -%}
            {{-'[INST] ' + message['content'].rstrip() + ' [/INST]'-}}
        {%- else -%}
            {{-'' + message['content'] + '</s>' -}}
        {%- endif -%}
    {%- endif -%}
{%- endfor -%}
{%- if add_generation_prompt -%}
    {{-''-}}
{%- endif -%}

🔧 Technical Details

The model is based on Mistral 7B, and the tokenizer remains unchanged. The author used SOTA techniques and personal insights for fine - tuning.

📄 License

The model is licensed under the apache - 2.0 license.

Citation Information

@llm{Zion_Alpha_Instruction_Tuned_SLERP,
  author = {SicariusSicariiStuff},
  title = {Zion_Alpha_Instruction_Tuned_SLERP},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP}
}

Support

My Ko-fi page ALL donations will go for research resources and compute, every bit counts.
My Patreon ALL donations will go for research resources and compute, every bit counts.

Videos

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご