Llama_3.1_8b_Smarteaz_V1.01 Open-source Text Generation Model - Achieve Efficient Text Creation for Free

Llama 3.1 8b Smarteaz V1.01

Developed by Nexesenex

An 8B-parameter model based on the Llama 3.1 architecture, created by merging multiple Smarteaz series models, specializing in text generation tasks

Large Language Model

Transformers

#Strict instruction adherence #Generalization of complex tasks #Multi-benchmark fusion

Downloads 85

Release Time : 2/27/2025

Model Overview

This is a text generation model that merges multiple Smarteaz series 8B-parameter models using the Model Stock method, serving as an intelligent module for more complex 8B model merging

Model Features

Multi-model merging

Utilizes the Model Stock method to merge multiple Smarteaz series 8B models, integrating the strengths of different models

High-performance text generation

Demonstrates excellent performance in benchmarks like IFEval, achieving a strict accuracy rate of 81.51 in zero-shot settings

Modular design

Can serve as an intelligent module for more complex 8B model merging, offering good scalability

Model Capabilities

Text generation

Instruction following

Few-shot learning

Use Cases

Evaluation testing

IFEval assessment

Used to evaluate the model's performance in instruction-following tasks

Achieved 81.51 strict accuracy rate in zero-shot settings

BBH assessment

Evaluates the model's performance in complex reasoning tasks

Achieved 32.28 normalized accuracy rate in 3-shot settings

Academic research

Mathematical problem solving

Used to solve competition-level mathematical problems

Achieved 23.41 exact match rate in 4-shot settings

🚀 Llama_3.1_8b_Smarteaz_V1.01

This is a successfully merged model in the Smarteaz V1 70b line, which can serve as a smart block for more complex 8b merges.

📚 Documentation

General Information

Property	Details
License	llama3.1
Library Name	transformers
Tags	mergekit, merge
Base Model	Nexesenex/Llama_3.1_8b_Smarteaz_0.21_R1, Nexesenex/Llama_3.1_8b_Smarteaz_0.11a, Nexesenex/Llama_3.1_8b_Smarteaz_0.21_SN

Model Index

The model named Llama_3.1_8b_Smarteaz_V1.01 has the following evaluation results in text generation tasks:

Dataset: IFEval (0 - Shot):
- Task: Text Generation
- Metric: strict accuracy (inst_level_strict_acc and prompt_level_strict_acc), value = 81.51
- Source: Open LLM Leaderboard
Dataset: BBH (3 - Shot):
- Task: Text Generation
- Metric: normalized accuracy (acc_norm), value = 32.28
- Source: Open LLM Leaderboard
Dataset: MATH Lvl 5 (4 - Shot):
- Task: Text Generation
- Metric: exact match, value = 23.41
- Source: Open LLM Leaderboard
Dataset: GPQA (0 - shot):
- Task: Text Generation
- Metric: acc_norm, value = 7.94
- Source: Open LLM Leaderboard
Dataset: MuSR (0 - shot):
- Task: Text Generation
- Metric: acc_norm, value = 8.2
- Source: Open LLM Leaderboard
Dataset: MMLU - PRO (5 - shot):
- Task: Text Generation
- Metric: accuracy (acc), value = 30.4
- Source: Open LLM Leaderboard

Merge Details

This is a merge of pre - trained language models created using mergekit.

Merge Method

This model was merged using the Model Stock merge method with Nexesenex/Llama_3.1_8b_Smarteaz_0.11a as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

merge_method: model_stock
models:
  - model: Nexesenex/Llama_3.1_8b_Smarteaz_0.21_R1
    parameters:
      weight: 1.0
  - model: Nexesenex/Llama_3.1_8b_Smarteaz_0.21_SN
    parameters:
      weight: 1.0
base_model: Nexesenex/Llama_3.1_8b_Smarteaz_0.11a
dtype: bfloat16
normalize: true
chat_template: auto
tokenizer:
  source: union

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	30.62
IFEval (0 - Shot)	81.51
BBH (3 - Shot)	32.28
MATH Lvl 5 (4 - Shot)	23.41
GPQA (0 - shot)	7.94
MuSR (0 - shot)	8.20
MMLU - PRO (5 - shot)	30.40

📄 License

The model is under the llama3.1 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご