Qwen2.5-Dyanka-7B-Preview Open-Source Language Model: Blend Multiple Models for Super-Practical Chatting and Q&A

Qwen2.5 Dyanka 7B Preview

Developed by Xiaojian9992024

A 7B-parameter language model based on the Qwen2.5 architecture, created by fusing multiple pre-trained models using the TIES method

Large Language Model

Transformers

Open Source License:Apache-2.0 #Multitask Text Generation #Zero-shot Learning #High-precision Instruction Following

Downloads 1,497

Release Time : 2/25/2025

Model Overview

This is a 7B-parameter model obtained by merging multiple pre-trained language models using the mergekit tool, employing the TIES fusion method, with capabilities in text generation and reasoning

Model Features

Multi-model Fusion

Utilizes the TIES method to merge 6 different pre-trained models, combining their strengths

Balanced Performance

Demonstrates balanced performance across various benchmarks, achieving 76.4% accuracy on IFEval

Open-source License

Released under Apache-2.0 license, permitting both commercial and research use

Model Capabilities

Text Generation

Instruction Following

Logical Reasoning

Mathematical Problem Solving

Knowledge Question Answering

Use Cases

Education

Mathematical Problem Solving

Solving middle to high school level math problems

Achieved 48.79% accuracy on MATH Lvl 5 test

Research

Knowledge QA System

Building domain-specific question answering systems

Achieved 37.51% accuracy on MMLU-PRO test

🚀 Qwen2.5-Dyanka-7B-Preview

Qwen2.5-Dyanka-7B-Preview is a merged pre - trained language model, offering enhanced text - generation capabilities through a specific merging method.

Qwen2.5-Dyanka-7B-Preview

🚀 Quick Start

This is a merge of pre - trained language models created using mergekit.

✨ Features

📦 Merge Details

Merge Method

This model was merged using the TIES merge method using gz987/qwen2.5-7b-cabs-v0.3 as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: gz987/qwen2.5-7b-cabs-v0.3
    #no parameters necessary for base model
  - model: suayptalha/Clarus-7B-v0.1
    parameters:
      density: 0.2
      weight: 0.2
  - model: Xiaojian9992024/Qwen2.5-THREADRIPPER-Small
    parameters:
      density: 0.2
      weight: 0.2
  - model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
    parameters:
      density: 0.2
      weight: 0.2
  - model: prithivMLmods/WebMind-7B-v0.1
    parameters:
      density: 0.2
      weight: 0.2
  - model: fblgit/cybertron-v4-qw7B-MGS
    parameters:
      density: 0.2
      weight: 0.2
      
merge_method: ties
base_model: gz987/qwen2.5-7b-cabs-v0.3
parameters:
  normalize: false
  int8_mask: true
dtype: bfloat16

📚 Open LLM Leaderboard Evaluation Results

Detailed results can be found here! Summarized results can be found here!

Property	Details
Base Models	rombodawg/Rombos-LLM-V2.5-Qwen-7b, suayptalha/Clarus-7B-v0.1, gz987/qwen2.5-7b-cabs-v0.3, prithivMLmods/WebMind-7B-v0.1, fblgit/cybertron-v4-qw7B-MGS, Xiaojian9992024/Qwen2.5-THREADRIPPER-Small
Library Name	transformers
Tags	mergekit, merge
Model Name	Qwen2.5-Dyanka-7B-Preview

Metric	Value (%)
Average	37.30
IFEval (0-Shot)	76.40
BBH (3-Shot)	36.62
MATH Lvl 5 (4-Shot)	48.79
GPQA (0-shot)	8.95
MuSR (0-shot)	15.51
MMLU-PRO (5-shot)	37.51

📄 License

This project is licensed under the apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご