Qwen2.5-14B-YOYO-V5 Open-source AI Model - Integrating the Advantages of Multiple Models and Supporting 1000,000-token Context

Qwen2.5 14B YOYO V5

Developed by YOYO-AI

The fifth-generation Qwen2.5-YOYO model integrates features from multiple advanced models, optimizes the model merging formula, and supports a context length of 1 million tokens.

Large Language Model

Safetensors

Supports Multiple LanguagesOpen Source License:Apache-2.0 #Multi-stage Fusion Model #Long Text Processing #Instruction Optimization

Downloads 33

Release Time : 3/22/2025

Model Overview

Qwen2.5-14B-YOYO-V5 is a multi-stage merged model based on Qwen2.5-14B, integrating models like Light-R1-14B-DS, optimizing the merging formula, and supporting long text processing.

Model Features

Multi-stage Merge Optimization

Integrates features from multiple advanced models through multi-stage merging to optimize model performance.

Long Text Support

Supports a context length of 1 million tokens, suitable for long text processing tasks.

Multi-language Support

Supports text generation tasks in English and Chinese.

Model Capabilities

Text Generation

Long Text Processing

Multi-language Support

Use Cases

Text Generation

Long Text Summarization

Generates summaries for long texts, supporting a context length of 1 million tokens.

Multi-language Text Generation

Generates text content in English or Chinese.

🚀 Qwen2.5-14B-YOYO-V5

The fifth - generation Qwen2.5 - YOYO model is officially released! It integrates advanced techniques and model merging strategies to offer enhanced performance.

image/jpeg

✨ Features

Upgrade Points

Integrate Light - R1 - 14B - DS
Optimize the model merging formula

🚀 Quick Start

First stage

models:  
  - model: tanliboy/lambda-qwen2.5-14b-dpo-test  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: Qwen/Qwen2.5-14B-Instruct  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-dpo-it

Second stage

Step 1

Create three different instruction models and one code model

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: arcee-ai/Virtuoso-Small-v2  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-v2-dpo

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: Azure99/Blossom-V6-14B  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-V6-dpo

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: arcee-ai/SuperNova-Medius  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-Nova-dpo

models:  
  - model: Qwen/Qwen2.5-Coder-14B-Instruct  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: Qwen/Qwen2.5-Coder-14B  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-code

Step 2

Create two different reasoning models.

merge_method: model_stock
base_model: arcee-ai/Virtuoso-Small-v2
models:
  - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
  - model: qihoo360/Light-R1-14B-DS
dtype: float16
tokenizer_source: base
int8_mask: true
normalize: true
name: Qwen2.5-14B-YOYO-DS-v2

merge_method: model_stock
base_model: Azure99/Blossom-V6-14B
models:
  - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
  - model: qihoo360/Light-R1-14B-DS
dtype: float16
tokenizer_source: base
int8_mask: true
normalize: true
name: Qwen2.5-14B-YOYO-DS-V6

Third stage

Create a base model with a context of 1 million tokens.

merge_method: sce  
models:
  # Pivot model
  - model: Qwen/Qwen2.5-14B-Instruct-1M
  # Target models  
  - model: Qwen/Qwen2.5-14B  
base_model: Qwen/Qwen2.5-14B-Instruct-1M  
parameters:  
  select_topk: 1  
dtype: float16  
tokenizer_source: base  
normalize: true  
int8_mask: true  
name: Qwen2.5-14B-1M

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: mergekit-community/Qwen2.5-14B-1M  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-1M-dpo

Final stage

merge_method: model_stock
base_model: mergekit-community/Qwen2.5-14B-della-1M-dpo
models:
  - model: mergekit-community/Qwen2.5-14B-della-v2-dpo
  - model: mergekit-community/Qwen2.5-14B-della-V6-dpo
  - model: mergekit-community/Qwen2.5-14B-della-Nova-dpo
  - model: mergekit-community/Qwen2.5-14B-della-1M-dpo
  - model: mergekit-community/Qwen2.5-14B-YOYO-DS-v2
  - model: mergekit-community/Qwen2.5-14B-YOYO-DS-V6
  - model: mergekit-community/Qwen2.5-14B-della-code
dtype: float16
tokenizer_source: base
int8_mask: true
normalize: true
name: Qwen2.5-14B-YOYO-V5

📄 License

This project is licensed under the Apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご