Qwen2.5 14B YOYO V5

🚀 Qwen2.5-14B-YOYO-V5

Qwen2.5-YOYO第五代模型正式发布！该模型集成了多种优秀的基础模型，通过优化模型合并公式，在性能和功能上有显著提升，适用于文本生成等多种自然语言处理任务。

✨ 主要特性

• 集成Light - R1 - 14B - DS：将Light - R1 - 14B - DS模型的优势融入其中，提升了模型的综合性能。
• 优化模型合并公式：通过优化合并公式，使得模型在不同任务上的表现更加出色。

📚 详细文档

第一阶段

models:  
  - model: tanliboy/lambda-qwen2.5-14b-dpo-test  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: Qwen/Qwen2.5-14B-Instruct  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-dpo-it

第二阶段

步骤1：创建三个不同的指令模型和一个代码模型

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: arcee-ai/Virtuoso-Small-v2  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-v2-dpo

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: Azure99/Blossom-V6-14B  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-V6-dpo

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: arcee-ai/SuperNova-Medius  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-Nova-dpo

models:  
  - model: Qwen/Qwen2.5-Coder-14B-Instruct  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: Qwen/Qwen2.5-Coder-14B  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-code

步骤2：创建两个不同的推理模型

merge_method: model_stock
base_model: arcee-ai/Virtuoso-Small-v2
models:
  - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
  - model: qihoo360/Light-R1-14B-DS
dtype: float16
tokenizer_source: base
int8_mask: true
normalize: true
name: Qwen2.5-14B-YOYO-DS-v2

merge_method: model_stock
base_model: Azure99/Blossom-V6-14B
models:
  - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
  - model: qihoo360/Light-R1-14B-DS
dtype: float16
tokenizer_source: base
int8_mask: true
normalize: true
name: Qwen2.5-14B-YOYO-DS-V6

第三阶段

创建一个上下文为100万个标记的基础模型。

merge_method: sce  
models:
  # 枢轴模型
  - model: Qwen/Qwen2.5-14B-Instruct-1M
  # 目标模型  
  - model: Qwen/Qwen2.5-14B  
base_model: Qwen/Qwen2.5-14B-Instruct-1M  
parameters:  
  select_topk: 1  
dtype: float16  
tokenizer_source: base  
normalize: true  
int8_mask: true  
name: Qwen2.5-14B-1M

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: mergekit-community/Qwen2.5-14B-1M  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-1M-dpo

最终阶段

merge_method: model_stock
base_model: mergekit-community/Qwen2.5-14B-della-1M-dpo
models:
  - model: mergekit-community/Qwen2.5-14B-della-v2-dpo
  - model: mergekit-community/Qwen2.5-14B-della-V6-dpo
  - model: mergekit-community/Qwen2.5-14B-della-Nova-dpo
  - model: mergekit-community/Qwen2.5-14B-della-1M-dpo
  - model: mergekit-community/Qwen2.5-14B-YOYO-DS-v2
  - model: mergekit-community/Qwen2.5-14B-YOYO-DS-V6
  - model: mergekit-community/Qwen2.5-14B-della-code
dtype: float16
tokenizer_source: base
int8_mask: true
normalize: true
name: Qwen2.5-14B-YOYO-V5

📄 许可证

本项目采用Apache - 2.0许可证。

📋 信息表格