NeuroBERT-Mini-GGUF Open-Source Miniature AI Model - Optimized for Edge Devices, Highly Efficient and Practical

Home

Neurobert Mini GGUF

Developed by mradermacher

Statically quantized version based on boltuix/NeuroBERT-Mini, optimized for edge devices

Large Language Model

Transformers

Open Source License:MIT #Miniature BERT Quantization #Edge AI Inference #Low-resource NLP

Downloads 219

Release Time : 5/20/2025

Model Overview

Lightweight English natural language processing model suitable for intent recognition, command classification, and other tasks in low-resource environments

Model Features

Edge Device Optimization

Offers multiple quantized versions, smallest being only 0.1GB, suitable for IoT and embedded system deployment

Diverse Quantization Options

Provides 12 GGUF quantized versions with different precisions, from Q2_K to f16 to meet various needs

Real-time Processing Capability

Designed for low-latency scenarios, suitable for real-time applications like smart home devices and toy robots

Model Capabilities

English text understanding

Intent recognition

Command classification

Named entity recognition

Offline inference

Real-time processing

Use Cases

Smart Home

Voice Command Understanding

Interprets users' voice commands for smart devices

Low-latency response, accuracy unspecified

Embedded Systems

Toy Robot Control

Understands and executes simple commands for toy robots from children

Suitable for resource-constrained environments

Wearable Devices

Offline Voice Assistant

Processes voice commands without internet connection

Privacy protection, fast response

🚀 NeuroBERT-Mini GGUF Quantized Model

This project provides static quantizations of the NeuroBERT-Mini model, offering various quantized versions for different use cases in NLP.

🚀 Quick Start

If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files.

✨ Features

Multiple Quantization Types: Offers a variety of quantized versions (e.g., Q2_K, Q3_K_S, etc.) to balance between size and quality.
Lightweight: Suitable for low - resource environments, edge devices, and mobile NLP.
Broad Application: Can be used in intent detection, command classification, and other NLP tasks.

📦 Model Information

Property	Details
Base Model	boltuix/NeuroBERT - Mini
Datasets	custom - dataset
Language	English
Library Name	transformers
License	MIT
Quantized By	mradermacher
Tags	BERT, NeuroBERT, transformer, pre - training, nlp, tiny - bert, edge - ai, transformers, low - resource, micro - nlp, quantized, iot, wearable - ai, offline - assistant, intent - detection, real - time, smart - home, embedded - systems, command - classification, toy - robotics, voice - ai, eco - ai, english, lightweight, mobile - nlp, ner

📚 Documentation

About

These are static quants of NeuroBERT-Mini. Weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion.

Provided Quants

(sorted by size, not necessarily quality. IQ - quants are often preferable over similar sized non - IQ quants)

Link	Type	Size/GB	Notes
GGUF	Q2_K	0.1
GGUF	Q3_K_S	0.1
GGUF	Q3_K_M	0.1	lower quality
GGUF	IQ4_XS	0.1
GGUF	Q3_K_L	0.1
GGUF	Q4_K_S	0.1	fast, recommended
GGUF	Q4_K_M	0.1	fast, recommended
GGUF	Q5_K_S	0.1
GGUF	Q5_K_M	0.1
GGUF	Q6_K	0.1	very good quality
GGUF	Q8_0	0.1	fast, best quality
GGUF	f16	0.1	16 bpw, overkill

Here is a handy graph by ikawrakow comparing some lower - quality quant types (lower is better):

And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.

📄 License

This project is licensed under the MIT license.

🙏 Thanks

I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご