Llama 68M Chat V1 GGUF

🚀 Felladrin/Llama-68M-Chat-v1-GGUF

This repository provides quantized GGUF model files for Llama-68M-Chat-v1 developed by Felladrin. These files offer different quantization options to balance between model size and performance.

📦 Model Information

📊 Quantized Model Files

📚 Original Model Card

A Llama Chat Model of 68M Parameters

• Base model: JackFram/llama-68m
• Datasets:
  • THUDM/webglm-qa
  • databricks/databricks-dolly-15k
  • cognitivecomputations/wizard_vicuna_70k_unfiltered
  • totally-not-an-llm/EverythingLM-data-V3
  • Amod/mental_health_counseling_conversations
  • sablo/oasst2_curated
  • starfishmedical/webGPT_x_dolly
  • Open-Orca/OpenOrca
  • mlabonne/chatml_dpo_pairs
• Availability in other ML formats:
  • ONNX: Felladrin/onnx-Llama-68M-Chat-v1

💻 Usage Examples

Recommended Prompt Format

<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant

Recommended Inference Parameters

penalty_alpha: 0.5
top_k: 4

Widget Examples

• General Knowledge Assistant

<|im_start|>system
You are a knowledgeable assistant. Help the user as much as you can.<|im_end|>
<|im_start|>user
How to become healthier?<|im_end|>
<|im_start|>assistant

• Career Counselor

<|im_start|>system
You are a career counselor. The user will provide you with an individual looking for guidance in their professional life, and your task is to assist them in determining what careers they are most suited for based on their skills, interests, and experience. You should also conduct research into the various options available, explain the job market trends in different industries, and advice on which qualifications would be beneficial for pursuing particular fields.<|im_end|>
<|im_start|>user
Heya!<|im_end|>
<|im_start|>assistant
Hi! How may I help you?<|im_end|>
<|im_start|>user
I am interested in developing a career in software engineering. What would you recommend me to do?<|im_end|>
<|im_start|>assistant

• Concise Assistant

<|im_start|>system
You are a helpful assistant who provides concise responses.<|im_end|>
<|im_start|>user
Hi!<|im_end|>
<|im_start|>assistant
Hello there! How may I help you?<|im_end|>
<|im_start|>user
I need to build a simple website. Where should I start learning about web development?<|im_end|>
<|im_start|>assistant

• Creative Assistant

<|im_start|>system
You are a very creative assistant. User will give you a task, which you should complete with all your knowledge.<|im_end|>
<|im_start|>user
Write the background story of an RPG game about wizards and dragons in a sci-fi world.<|im_end|>
<|im_start|>assistant