đ matrixportal/gemma-3-1b-it-abliterated-GGUF
This project focuses on converting the model to the GGUF format, facilitating local deployment and utilization of the model. It provides various quantization options to meet different user needs, and also lists a series of applications and tools suitable for locally quantized LLMs.
đ Quick Start
This model was converted to GGUF format from mlabonne/gemma-3-1b-it-abliterated using llama.cpp via the ggml.ai's all-gguf-same-where space.
Refer to the original model card for more details on the model.
⨠Features
- Multiple Quantization Options: Offers a variety of quantization options to balance speed and quality according to different user needs.
- Rich Application List: Lists a series of desktop, mobile, and image generation applications and tools suitable for locally quantized LLMs.
đĻ Model Information
| Property |
Details |
| Base Model |
mlabonne/gemma-3-1b-it-abliterated |
| Library Name |
transformers |
| License |
gemma |
| Pipeline Tag |
image-text-to-text |
| Tags |
llama-cpp, matrixportal |
đĻ Quantized Models Download List
⨠Recommended Quantizations
- đģ General CPU Use:
Q4_K_M (Best balance of speed/quality)
- đą ARM Devices:
Q4_0 (Optimized for ARM CPUs)
- đ Maximum Quality:
Q8_0 (Near-original quality)
đ Full Quantization Options
đĄ Usage Tip
Use F16 for maximum precision when quality is critical
đ Applications and Tools for Locally Quantized LLMs
đģ Desktop Applications
| Application |
Description |
Download Link |
| Llama.cpp |
A fast and efficient inference engine for GGUF models. |
GitHub Repository |
| Ollama |
A streamlined solution for running LLMs locally. |
Website |
| AnythingLLM |
An AI-powered knowledge management tool. |
GitHub Repository |
| Open WebUI |
A user-friendly web interface for running local LLMs. |
GitHub Repository |
| GPT4All |
A user-friendly desktop application supporting various LLMs, compatible with GGUF models. |
GitHub Repository |
| LM Studio |
A desktop application designed to run and manage local LLMs, supporting GGUF format. |
Website |
| GPT4All Chat |
A chat application compatible with GGUF models for local, offline interactions. |
GitHub Repository |
đą Mobile Applications
| Application |
Description |
Download Link |
| ChatterUI |
A simple and lightweight LLM app for mobile devices. |
GitHub Repository |
| Maid |
Mobile Artificial Intelligence Distribution for running AI models on mobile devices. |
GitHub Repository |
| PocketPal AI |
A mobile AI assistant powered by local models. |
GitHub Repository |
| Layla |
A flexible platform for running various AI models on mobile devices. |
Website |
đ¨ Image Generation Applications
| Application |
Description |
Download Link |
| Stable Diffusion |
An open-source AI model for generating images from text. |
GitHub Repository |
| Stable Diffusion WebUI |
A web application providing access to Stable Diffusion models via a browser interface. |
GitHub Repository |
| Local Dream |
Android Stable Diffusion with Snapdragon NPU acceleration. Also supports CPU inference. |
GitHub Repository |
| Stable-Diffusion-Android (SDAI) |
An open-source AI art application for Android devices, enabling digital art creation. |
GitHub Repository |
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
