đ Llama 3.2 400M Amharic
This is a smaller version of Meta's Llama-3.2-1B decoder transformer model. It's pretrained from scratch for 23 hours using a single A100 40GB GPU and 274 million tokens of Amharic text, offering a cost - effective solution for Amharic text processing.
đ Quick Start
This is a smaller version of the Meta's Llama-3.2-1B decoder transformer model. It was pretrained from scratch for 23 hours using a single A100 40GB GPU and 274 million tokens of Amharic text.
- It has 400 Million parameters.
- The context size of this model is 1024 tokens.
- It uses the same tokenizer as Llama - 3.2 - 1B, which was trained from scratch using the same Amharic dataset as the model, with a vocabulary size of 32k.
- Validation Perplexity: 41.3
- This is a base model and hasn't undergone any supervised finetuning yet.
đĻ Installation
First, you need to install the latest version of transformers
pip install -Uq transformers
đģ Usage Examples
Basic Usage
You can use this model directly with a pipeline for text generation:
from transformers import pipeline
llama_am = pipeline(
"text-generation",
model="rasyosef/Llama-3.2-400M-Amharic",
device_map="auto"
)
prompt = "á á˛áĩ á á áŖ"
llama_am(
prompt,
max_new_tokens=128,
temperature=0.5,
do_sample=True,
top_k=8,
top_p=0.8,
repetition_penalty=1.2
)
Output:
[{'generated_text': 'á á˛áĩ á á áŖáŖ áŗá
áŗáĩ 8 áŖ2012 (á¤á áĸ á˛) á¨áĸáá´áĒ á¨áá ááŗá áááĩáĩá á áļ ááą á ááŗááá¸á á¨á ááĒáĢ á
áĨá¨áĩ á¨áĩáĢ á áĩááģáááŊ ááá á¤áĩ áá°á á áĩáĨá°áŖ áá ááá á¨á´áááĢá˛á áĒááĨáá áŽáá á áģá¸á ááĒ áąááŖ ááá áĨá á¨áááŊ á¨á ááĒáĢ á ááŖáŗá°áŽáŊ áá á°ááĢáá°áááĸá ááááŗá¸áá á á ááĒáĢ á¨áŽáŽá áĢáá¨áĩá ááá¨áá¨á áĨá¨á°á¨ááá áŖá áĩáĢááŊ ááĒáĢ ááá¨áĢá¸áá á áĩáá°á ááģá¸á á áĩáá¨áááĸá¨áááąá áááĢáĩ áááááĩ á á°ááá¨á°ááĨ áĸáĩáŽáĩáĢ á á
áĨá¨áą áá á ááŖáŗá°áááĩ áŖááĩ áááááĩ áášááĩ áá
á¨áĨá á¨áá°áá
ááááá á ááĩá°áááĸáĸáĩáŽáĩáĢ á¨áŽáĒáĩ19 áá¨ááŊáá ááááŗáĩ áĨáĢá°á¨ááŊ áŖáá áĨá¨áĩ áá°áĄáĨ á ááĒáĢ ááĩáá á á
ááŖá áá áᤠáĢáá¨áąá ááááŖá á á¨áĸáĩáŽáĩáĢ áá áĨáá°ááĩáá á áááá ááá¨áĨááá¸á áĨáĢáá áá°ááĩ á áá°áŠ á¨áĩáĨáĨá ááĩáŽáŊ áá á°áááá¨á ááĩáĢáĩ áĨáááááá áĨááááĸá ááŖáá áááą'}]
| Property |
Details |
| Model Type |
Decoder Transformer |
| Training Data |
274 million tokens of Amharic text |
| Parameters |
400 Million |
| Context Size |
1024 tokens |
| Tokenizer Vocabulary Size |
32k |
| Validation Perplexity |
41.3 |
| Fine - tuning Status |
Not yet supervised finetuned |
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
