Reasoning-SCE-Coder-v1.0 Open-source Large Language Model - Multi-model Fusion Empowers Efficient Reasoning Applications

Reasoning SCE Coder V1.0

Developed by BenevolenceMessiah

A 32B-parameter large language model constructed based on the SCE fusion method, integrating multiple high-performance pre-trained models

Large Language Model

Transformers

#32B large model fusion #SCE adaptive weighting #Multi-task optimization

Downloads 235

Release Time : 3/8/2025

Model Overview

This is a large language model that merges multiple 32B-parameter models using the mergekit tool, employing the adaptive weighting algorithm proposed in the SCE paper, specifically optimized for generation quality and inference efficiency

Model Features

Multi-model fusion

Uses the SCE adaptive weighting algorithm to fuse multiple high-performance 32B models, balancing the strengths of each model

Efficient inference

Optimized with Flash Attention mechanism to improve long-text processing efficiency

Code generation capability

Incorporates specialized code-generation models to enhance performance in programming-related tasks

Model Capabilities

Text generation

Code generation and completion

Q&A systems

Text summarization

Multi-turn dialogue

Instruction following

Use Cases

Development assistance

Code auto-completion

Helps developers quickly generate code snippets

Improves programming efficiency and reduces repetitive work

Document generation

Automatically generates technical documentation from code

Ensures documentation stays synchronized with code

Content creation

Article writing

Assists creators in generating article drafts

Provides creative inspiration and content frameworks

🚀 Language Model Merge Project

This project focuses on merging pre - trained language models, leveraging the power of multiple models to create a more robust and high - performance language model.

🚀 Quick Start

This is a merge of pre - trained language models created using mergekit.

📚 Documentation

✨ Features

Utilize the SCE merge method to combine multiple pre - trained language models.
Select Rombo-Org/Rombo-LLM-V3.1-QWQ-32b as the base model.
Combine multiple high - quality pre - trained models to enhance the overall performance of the model.

📦 Installation

The document does not provide installation steps, so this section is skipped.

💻 Usage Examples

The document does not provide code examples, so this section is skipped.

🔧 Technical Details

Merge Details

Merge Method

This model was merged using the SCE merge method using Rombo-Org/Rombo-LLM-V3.1-QWQ-32b as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview
    parameters:
      weight: 1.2      # Slightly favor
      density: 0.9     # Sparsified a bit to reduce noise
  - model: FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview
    parameters:
      weight: 1
      density: 0.9
  - model: FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview
    parameters:
      weight: 1
      density: 0.9
merge_method: sce         # SCE for adaptive weighting
base_model: Rombo-Org/Rombo-LLM-V3.1-QWQ-32b
parameters:
  normalize: true
  int8_mask: true
  select_topk: 0.1        # Retain the top 10% high-variance elements
tokenizer_source: union    # Union to combine vocabularies
dtype: bfloat16

📄 License

The document does not provide license information, so this section is skipped.

Property	Details
Base Model	FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-32B-Preview, Rombo-Org/Rombo-LLM-V3.1-QWQ-32b, FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview, FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview
Library Name	transformers
Tags	mergekit, merge

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご