Blockchainlabs_7B Open-Source Large Model - Optimized and Pruned for Flexible Use on Device Ends

Home

Blockchainlabs 7B Merged Test2 4 Prune Sft 4bit DPO Orca

Developed by alnrg2arg

This is a small 7B-parameter LLM optimized for device-side use, pruned and trained with DPO

Large Language Model

Transformers

English#Device-side optimization #Pruned model #DPO fine-tuning

Downloads 18

Release Time : 1/23/2024

Model Overview

This model is a 7B-parameter language model based on the Mistral architecture, optimized through merging, pruning (50% sparsity), and DPO training to reduce model size while maintaining performance, suitable for deployment on resource-constrained devices

Model Features

Device-side optimization

Significantly reduces model size through 50% sparsity pruning, suitable for deployment on resource-limited devices

DPO training

Trained using Direct Preference Optimization (DPO) method to improve output quality

Efficient inference

Utilizes 8-bit optimization and AdamW optimizer to enhance inference efficiency

Model Capabilities

English text generation

Instruction following

Dialogue generation

Use Cases

Mobile applications

Device-side chat assistant

Deploy lightweight chatbots on mobile devices like smartphones

Reduces resource usage while maintaining response quality

Edge computing

Localized text processing

Perform text generation and processing on edge devices without cloud dependency

Enhances privacy protection and response speed

🚀 Blockchainlab Test 2.4 Model

This project aims to create a small LLM for on - device use by merging, pruning, and applying DPO on a base model.

🚀 Quick Start

This model is derived from blockchainlab test 2.4, specifically the merged model alnrg2arg/blockchainlabs_7B_merged_test2_4_prune. The project's goal is to develop a small - scale large language model (LLM) for on - device applications.

The overall pipeline for this iteration consists of the following steps:

Merge models to create a 7B base model.
Prune the model to reduce parameters, achieving 50% sparsity.
For the recovery phase of pruning, choose Direct Preference Optimization (DPO).

✨ Features

Pruned Model: This model has been pruned to reduce its parameter count.
DPO Application: DPO is used in the recovery phase of pruning.

📦 Installation

No installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import TrainingArguments, AutoModelForCausalLM
from trl import DPOTrainer

dpo_trainer = DPOTrainer(
    model = model,
   
    ref_model = None,
    args = TrainingArguments(
        per_device_train_batch_size = 8,
        gradient_accumulation_steps = 8,
        warmup_ratio = 0.1,
        num_train_epochs = 3,
        learning_rate = 5e-6,
        fp16 = not torch.cuda.is_bf16_supported(),
        bf16 = torch.cuda.is_bf16_supported(),
        logging_steps = 1,
        optim = "adamw_8bit",
        weight_decay = 0.0,
        lr_scheduler_type = "linear",
        seed = 42,
        output_dir = "output_DPO",
    ),
    beta = 0.1,
    train_dataset = dataset,
    # eval_dataset = raw_datasets["test"],
    tokenizer = tokenizer,
    max_length = 1024,
    max_prompt_length = 512,
)

The code and parameters are borrowed from this Colab notebook.

📚 Documentation

Base Model: alnrg2arg/blockchainlabs_7B_merged_test2_4_prune
Datasets: Intel/orca_dpo_pairs

📄 License

This project is licensed under the CC - BY - NC 4.0 license.

🔖 Tags

text - generation - inference
transformers
unsloth
mistral
trl

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご