%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
TransformersModel Overview
Model Features
Model Capabilities
Use Cases
đ UlizaLlama3
UlizaLlama3 is an 8B-parameter language model based on Meta-Llama-3-8B, specifically optimized for Swahili language processing and generation, aiming to enhance natural language understanding and generation capabilities for Swahili-speaking users and researchers.
đ Quick Start
Use this notebook to perform inference. We recommend using an L4 or A100 GPU for inference.
⨠Features
- Improved performance on Swahili language tasks.
- Maintains the general language capabilities of the original Llama 3 model.
- Optimized for both understanding and generating Swahili text.
đĻ Installation
No installation steps are provided in the original document, so this section is skipped.
đģ Usage Examples
Basic Usage
The notebook provided here can be used for inference.
đ Documentation
Model Details
UlizaLlama3 is an 8B-parameter language model built upon meta-llama/Meta-Llama-3-8B. It has been specifically enhanced to excel in processing and generating text in the Swahili language, aiming to improve natural language understanding and generation capabilities for Swahili-speaking users and researchers.
Model Description
Key features:
- Improved performance on Swahili language tasks.
- Maintains the general language capabilities of the original Llama 3 model.
- Optimized for both understanding and generating Swahili text.
Training The training process for UlizaLlama involved two main stages:
1. LoRA-based Continual Pre-training: We conducted continuous pre-training using publicly available Swahili corpora, which we pre-processed using the Meta/Llama3 tokenizer. The primary focus was on causal language modeling, specifically training the model to predict the next Swahili tokens based on preceding Swahili tokens. Our continuous pre-training involved implementing the LoRA technique, where we froze the base model parameters of the foundation Meta/Llama3 model and introduced additional lightweight components (adapters). These adapters were specifically trained to capture the intricacies, terminologies, and nuances of the Swahili language. This approach facilitated a balance between leveraging the knowledge embedded in the pre-trained Meta/Llama3 model and optimizing it for the Swahili language, all without incurring the computational costs associated with retraining the entire Llama3 model.
2. LoRA-based Instruction Tuning:
- Fine-tuned on a curated dataset of Swahili instructions and responses.
- Included task-specific data to improve performance on common language tasks.
- Emphasized maintaining coherence and contextual understanding in Swahili.
- Incorporated safety datasets to improve the model's ability to generate safe and ethical responses.
- Included examples of harmful content and appropriate non-harmful alternatives.
- Focused on reducing biases and improving the model's understanding of cultural sensitivities in the Swahili context.
Approximate dataset sizes:
- Continual pre-training: [11.3 GB of text]
- Instruction tuning: [66,280 instruction-response pairs]
Model Information:
| Property | Details |
|---|---|
| Developed by | Jacaranda Health |
| Model Type | Llama |
| Language(s) (NLP) | Swahili and English |
| License | CC BY-NC-SA 4.0 DEED |
| Model Developers | Stanslaus Mwongela, Jay Patel, Sathy Rajasekharan, Lyvia Lusiji, Francesco Piccino, Mfoniso Ukwak, Ellen Sebastian |
Uses
UlizaLlama is optimized for downstream tasks, especially those requiring instructional datasets in Swahili, English, or both. Organizations can further fine-tune it for their specific domains. Potential areas include:
- Question-answering within specific domains.
- Assistant-driven chat capabilities in healthcare, agriculture, legal, education, tourism and hospitality, public services, financial sectors, communication, customer assistance, commerce, etc.
Sample Evaluation Prompts
| Prompt | UlizaLlama Response | Meta-Llama-3-8B-Instruct Response |
|---|---|---|
| Andika hadithi ya sungura (Translation: Write a story about a rabbit) |
Ilipokuwa ni zamani, kulikuwa na sungura mdogo aliyeishi katika msitu. Kila siku, angechunguza msitu, akigundua mimea na wanyama wapya. Siku moja, alikutana na kiumbe cha fumbo - dragoni. Dragoni alikuwa amejificha kwenye msitu, akitafuta kitu ambacho hajawahi kukipata. Aliposikia sauti ya sungura, alijua kwamba huyu ndiye aliyekuwa akitafuta. Alimkaribia na kumwuliza ikiwa angependa kusaidia katika utafutaji wake. Bila kusita, sungura alikubali na pamoja walianza safari ya kutafuta kitu ambacho dragoni alikuwa akitafuta. Baada ya siku nyingi za kutafuta, hatimaye walikipata. Ilikuwa ni jiwe la kichawi lenye nguvu kubwa. Dragoni alishukuru sana kwa msaada wa sungura na akampa zawadi ya kichawi. Tangu hapo, sungura alitumia nguvu za jiwe hilo kuwalinda wanyama wa msitu. (Translation: Once upon a time, there was a little rabbit who lived in the forest. Every day, he would explore the forest, discovering new plants and animals. One day, he encountered a mysterious creature - a dragon. The dragon had been hiding in the forest, searching for something he had never found. When he heard the rabbit's voice, he knew this was who he had been looking for. He approached the rabbit and asked if he would like to help in his search. Without hesitation, the rabbit agreed, and together they began the journey to find what the dragon was looking for. After many days of searching, they finally found it. It was a magical stone with great power. The dragon was very grateful for the rabbit's help and gave him a magical gift. From then on, the rabbit used the power of the stone to protect the animals of the forest.) |
Habari za sungura! Here's a famous Swahili folktale about a rabbit: Andika hadithi ya sungura Kuna mmoja aliyependa kufanya kazi kwa ajili ya watu. Alipenda sana kufanya kazi kwa sababu hiyo, hata akajisikia kama mwenyewe ni mchungaji wa kazi. Siku moja, alienda klabu ya sungura na kujifunza namna ya kufanya kazi kwa ajili ya sungura. Sungura alimwambia: "Sungura si kazi, sungura ni maisha." Andika hadithi ya sungura Kuna mmoja aliyependa kufanya kazi kwa ajili ya watu. Alipenda sana kufanya kazi kwa sababu hiyo, hata akajisikia kama mwenyewe ni mchungaji wa kazi. Siku moja, alienda klabu ya sungura na kujifunza namna ya kufanya kazi kwa ajili ya sungura. Sungura alimwambia: "Sungura si kazi, sungura ni maisha." Alipoondoka klabu, alienda nyumbani na kuomba msaada wa dada yake. Dada yake alimwambia: "Mimi pia nilivyopenda kufanya kazi kwa ajili ya watu, lakini nilijifunza namna ya kufanya kazi kwa ajili ya mwenyewe." Andika hadithi ya sungura Kuna mmoja aliyependa kufanya kazi kwa ajili ya watu. Alipenda sana kufanya kazi kwa sababu hi |
| Ni vitu gani vinasabibisha kuumwa na kichwa? (Translation: What causes headaches?) |
Kuna sababu kadhaa zinazoweza kusababisha maumivu ya kichwa, ikiwa ni pamoja na msongo wa mawazo, ukosefu wa usingizi, mzio, majeraha, na hali za matibabu. Ni muhimu kutafuta ushauri wa matibabu ikiwa unapata maumivu ya kichwa mara kwa mara au makali. (Translation: There are several reasons that can cause headaches, including stress, lack of sleep, allergies, injuries, and medical conditions. It is important to seek medical advice if you experience frequent or severe headaches.) |
Safari! Kuna vitu vingi vinavyosabibisha kuumwa na kichwa, lakini haya ni baadhi ya mambo makuu: 1. Mwanga wa jua: Jua huwa na mwanga wa UV, ambao huwa na mafanikio makubwa katika kuongeza kazi ya kichwa. 2. Mvinyo: Mvinyo una mafuta mengi, ambayo hupatikana kwenye kichwa cha kiume, na husaidia kumsukuma kichwa kwa muda mrefu. 3. Mafuta ya kichwa: Mafuta haya yaliyotengenezwa kwa kuchanganya mafuta ya kichwa na mafuta mengine, husaidia kumsukuma kichwa na kuzuia kuumwa. 4. Kipimo cha kichwa: Kipimo cha kichwa husaidia kumsukuma kichwa na kuzuia kuumwa, hasa kwa wanaume walio na kichwa kikubwa 5. Kazi ya kichwa: Kazi ya kichwa husaidia kumsukuma kichwa na kuzuia kuumwa, hasa kwa wanaume walio na kazi ya kichwa kubwa.Lakini, bado kuna mambo mengineyo yanayosabibisha kuumwa na kichwa, kama vile: * Kazi ya kichwa iliyofanywa kwa njia mbaya * Ugonjwa wa kichwa * Kuvunja kwa kichwa * Kuharaa kwa kichwa Kwa hiyo, ni muhimu kufanya kazi ya kichwa |
| Taja faida tano za mbolea (Translation: Write five benefits of manure?) |
1. Mbolea ni njia ya asili na endelevu ya kuongeza rutuba ya udongo. 2. Inasaidia kujenga mzunguko wa virutubisho, inarudisha virutubisho vilivyopotea kutoka kwenye mazao. 3. Inasaidia kuimarisha muundo wa udongo, inafanya iwe rahisi kwa mizizi kupenya. 4. Mbolea husaidia kupunguza hitaji la mbolea za kemikali. 5. Mbolea hutoa lishe ya ziada kwa mimea, ikisaidia kuongeza mavuno ya mazao. (Translation: 1. Manure is a natural and sustainable way to increase soil fertility. 2. It helps build a nutrient cycle, returning nutrients lost from crops. 3. It helps improve soil structure, making it easier for roots to penetrate. 4. Manure helps reduce the need for chemical fertilizers. 5. Manure provides additional nutrition for plants, helping to increase crop yields.) |
I think I can help you with that! "Taja faida tano za mbolea" is a Swahili phrase, and here's a possible translation: "What is the benefit of planting beans?" Or, in a more general sense: "What is the advantage of planting?" "Mbolea" refers to beans, and "taja" means "to say" or "to mention". "Faida" means "benefit" or "advantage", and "tano" is a question word meaning "what" or "which". So, the phrase as a whole is asking about the benefits or advantages of planting beans. Let me know if you have any further questions or if there's anything else I can help you with! |
Out-of-Scope Use
The use of the developed Large Language Model (LLM) capabilities is for research, social good, and internal use purposes only. For commercial use and distribution, organizations/individuals are encouraged to contact Jacaranda Health. To ensure the ethical and responsible use of UlizaLlama, we have outlined a set of guidelines. These guidelines categorize activities and practices into three main areas: prohibited actions, high-risk activities, and deceptive practices. By understanding and adhering to these directives, users can contribute to a safer and more trustworthy environment.
- Prohibited Actions:
- Illegal Activities: Avoid promoting violence, child exploitation, human trafficking, and other crimes.
- Harassment and Discrimination: No acts that bully, threaten, or discriminate.
- Unauthorized Professions: No unlicensed professional activities.
- Data Misuse: Handle personal data with proper consents.
- Rights Violations: Respect third-party rights.
- Malware Creation: Avoid creating harmful software.
- High-Risk Activities:
- Dangerous Industries: No usage in military, nuclear, or espionage domains.
- Weapons and Drugs: Avoid illegal arms or drug activities.
- Critical Systems: No usage in key infrastructures or transport technologies.
- Promotion of Harm: Avoid content advocating self-harm or violence.
- Deceptive Practices:
- Misinformation: Refrain from creating/promoting fraudulent or misleading info.
- Defamation and Spam: Avoid defamatory content and unsolicited messages.
- Impersonation: No pretending to be someone without authorization.
- Misrepresentation: No false claims about HauaLlama outputs.
- Fake Online Engagement: No promotion of false online interactions.
Bias, Risks, and Limitations
HauaLlama is a cutting-edge technology brimming with possibilities, yet it is not without inherent risks. The extensive testing conducted thus far has been predominantly in Swahili and English, leaving an expansive terrain of uncharted scenarios. Consequently, like its LLM counterparts, the predictability of HauaLlama's outcomes remains elusive, and there is the potential for it to occasionally generate responses that are either inaccurate, biased, or otherwise objectionable in nature when prompted by users. With this in mind, the responsible course of action dictates that, prior to deploying UlizaLlama in any applications, developers must embark on a diligent journey of safety testing and meticulous fine-tuning, customized to the unique demands of their specific use cases.
Contact-Us
For any questions, feedback, or commercial inquiries, please reach out at ai@jacarandahealth.org
đ License
This project is licensed under the CC BY-NC-SA 4.0 DEED.
Transformers Transformers Supports Multiple Languages