Qwen2.5 14B YOYO V5

🚀 Qwen2.5-14B-YOYO-V5

Qwen2.5-YOYO第五代模型正式發佈！該模型集成了多種優秀的基礎模型，通過優化模型合併公式，在性能和功能上有顯著提升，適用於文本生成等多種自然語言處理任務。

✨ 主要特性

• 集成Light - R1 - 14B - DS：將Light - R1 - 14B - DS模型的優勢融入其中，提升了模型的綜合性能。
• 優化模型合併公式：通過優化合並公式，使得模型在不同任務上的表現更加出色。

📚 詳細文檔

第一階段

models:  
  - model: tanliboy/lambda-qwen2.5-14b-dpo-test  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: Qwen/Qwen2.5-14B-Instruct  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-dpo-it

第二階段

步驟1：創建三個不同的指令模型和一個代碼模型

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: arcee-ai/Virtuoso-Small-v2  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-v2-dpo

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: Azure99/Blossom-V6-14B  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-V6-dpo

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: arcee-ai/SuperNova-Medius  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-Nova-dpo

models:  
  - model: Qwen/Qwen2.5-Coder-14B-Instruct  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: Qwen/Qwen2.5-Coder-14B  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-code

步驟2：創建兩個不同的推理模型

merge_method: model_stock
base_model: arcee-ai/Virtuoso-Small-v2
models:
  - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
  - model: qihoo360/Light-R1-14B-DS
dtype: float16
tokenizer_source: base
int8_mask: true
normalize: true
name: Qwen2.5-14B-YOYO-DS-v2

merge_method: model_stock
base_model: Azure99/Blossom-V6-14B
models:
  - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
  - model: qihoo360/Light-R1-14B-DS
dtype: float16
tokenizer_source: base
int8_mask: true
normalize: true
name: Qwen2.5-14B-YOYO-DS-V6

第三階段

創建一個上下文為100萬個標記的基礎模型。

merge_method: sce  
models:
  # 樞軸模型
  - model: Qwen/Qwen2.5-14B-Instruct-1M
  # 目標模型  
  - model: Qwen/Qwen2.5-14B  
base_model: Qwen/Qwen2.5-14B-Instruct-1M  
parameters:  
  select_topk: 1  
dtype: float16  
tokenizer_source: base  
normalize: true  
int8_mask: true  
name: Qwen2.5-14B-1M

models:  
  - model: mergekit-community/Qwen2.5-14B-dpo-it  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: mergekit-community/Qwen2.5-14B-1M  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: float16  
tokenizer_source: base  
name: Qwen2.5-14B-della-1M-dpo

最終階段

merge_method: model_stock
base_model: mergekit-community/Qwen2.5-14B-della-1M-dpo
models:
  - model: mergekit-community/Qwen2.5-14B-della-v2-dpo
  - model: mergekit-community/Qwen2.5-14B-della-V6-dpo
  - model: mergekit-community/Qwen2.5-14B-della-Nova-dpo
  - model: mergekit-community/Qwen2.5-14B-della-1M-dpo
  - model: mergekit-community/Qwen2.5-14B-YOYO-DS-v2
  - model: mergekit-community/Qwen2.5-14B-YOYO-DS-V6
  - model: mergekit-community/Qwen2.5-14B-della-code
dtype: float16
tokenizer_source: base
int8_mask: true
normalize: true
name: Qwen2.5-14B-YOYO-V5

📄 許可證

本項目採用Apache - 2.0許可證。

📋 信息表格