%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Tinyllama 1.1B Python V0.1
TinyLlama is a lightweight Llama model with 1.1 billion parameters, pre - trained on 3 trillion tokens, suitable for application scenarios with limited computing resources.
Downloads 1,274
Release Time : 10/3/2023
Model Overview
TinyLlama is a lightweight language model based on the Llama 2 architecture, optimized to complete pre - training within 90 days. It is compatible with the Llama ecosystem and is suitable as an auxiliary model or for resource - constrained environments.
Model Features
Efficient pre - training
Using 16 A100 - 40G GPUs, pre - training on 3 trillion tokens can be completed within 90 days.
Lightweight design
Only 1.1 billion parameters, suitable for environments with limited computing and memory resources.
Fully compatible with the Llama ecosystem
Adopts the same architecture and tokenizer as Llama 2, and can be used out - of - the - box.
Multi - language retention ability
Although mainly fine - tuned on Python data, it still retains the ability for other languages such as C/Java.
Auxiliary model function
Can serve as a draft model to provide speculative decoding support for larger models (such as CodeLlama).
Model Capabilities
Text generation
Code generation
Speculative decoding assistance
Multi - language processing
Use Cases
Programming assistance
Python code generation
Generate Python code snippets based on the context.
14% accuracy on the HumanEval benchmark test
Multi - language code completion
Supports code completion for languages such as C/Java (the ability is weaker than that for Python).
Model acceleration
Speculative decoding assistance
Accelerate inference as a draft model for large models such as CodeLlama.
Featured Recommended AI Models
Š 2025AIbase