Sikong-llama-7b-chinese Open-source Model - Fine-tuned Based on Specific Datasets to Meet Diverse Needs

Sikong Llama 7b Chinese

Developed by SikongSphere

Model fine-tuned on custom datasets based on Linly-Chinese-LLaMA-7b-hf

Large Language Model

Transformers

#Chinese LLaMA Fine-tuning #Multi-GPU Training #Mini-batch Learning

Downloads 55

Release Time : 6/17/2023

Model Overview

This model is a fine-tuned version based on Linly-Chinese-LLaMA-7b-hf on custom datasets. Specific functionalities and applications require further details.

Model Features

Chinese Optimization

Fine-tuned on Chinese datasets, optimized for Chinese text generation capabilities

Multi-GPU Training

Distributed training using 4 GPUs, improving training efficiency

Model Capabilities

Chinese Text Generation

Language Understanding

Use Cases

Text Generation

Chinese Content Creation

Can be used to generate Chinese articles, stories, and other content

🚀 finetune

This model is a fine - tuned version of [/root/autodl - tmp/sikong/repo/LMFlow/output_models/Linly - Chinese - LLaMA - 7b - hf](https://huggingface.co//root/autodl - tmp/sikong/repo/LMFlow/output_models/Linly - Chinese - LLaMA - 7b - hf) on the customized dataset, offering enhanced performance for specific tasks.

🚀 Quick Start

This section will be updated with quick - start instructions once they are available.

📚 Documentation

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e - 05
train_batch_size: 2
eval_batch_size: 8
seed: 42
distributed_type: multi - GPU
num_devices: 4
total_train_batch_size: 8
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
lr_scheduler_type: linear
num_epochs: 50.0

Training results

More information needed

Framework versions

Transformers 4.28.0.dev0
Pytorch 2.0.0+cu117
Datasets 2.10.1
Tokenizers 0.13.3

Property	Details
Model Type	Fine - tuned model based on [/root/autodl - tmp/sikong/repo/LMFlow/output_models/Linly - Chinese - LLaMA - 7b - hf](https://huggingface.co//root/autodl - tmp/sikong/repo/LMFlow/output_models/Linly - Chinese - LLaMA - 7b - hf)
Training Data	Customized dataset