AIbase
EN
Home
Llama3.1 1B Neo BAAI 1000k
L

Llama3.1 1B Neo BAAI 1000k

Developed by yang31210999
Llama3.1-Neo-1B-100w is an efficient language model pruned to 1.4B parameters from Meta-Llama-3.1-8B-Instruct and fine-tuned using the LLM-Neo method (combining LoRA and knowledge distillation). The training data consists of 1 million samples from BAAI/Infinity-Instruct.
Large Language Model
Transformers
Open Source License:Apache-2.0#Knowledge Distillation Optimization#Instruction Fine-tuning#Lightweight LLM
Downloads 39
Release Time : 9/10/2024

Model Overview

This model is an efficient large language model after parameter knowledge distillation, focusing on text generation tasks and suitable for various natural language processing scenarios.

Model Features

Efficient Parameter Knowledge Distillation
Uses the LLM-Neo method combining LoRA and knowledge distillation techniques to significantly reduce model parameters while maintaining performance.
Lightweight Design
Pruned from 8B to 1.4B parameters, greatly reducing computational resource requirements.
High-Quality Fine-tuning Data
Fine-tuned with 1 million carefully selected samples from the BAAI/Infinity-Instruct dataset.

Model Capabilities

Text Generation
Question Answering System
Instruction Following
Knowledge Reasoning

Use Cases

Education
Academic Q&A System
Used to answer various academic questions from students
Achieved 31.58% accuracy on the CEVAL advanced mathematics subset
Business
Accounting Knowledge Q&A
Handles basic accounting-related questions
Achieved 24.49% accuracy on the CEVAL accounting subset
General AI Assistant
Daily Problem Solving
Answers various questions in daily life
Achieved 58.43% accuracy on the PIQA benchmark
Featured Recommended AI Models
AIbase
Empowering the Future, Your AI Solution Knowledge Base
English简体中文繁體中文にほんご
© 2025AIbase