đ NeoBERT Quantized Model
This project provides static quantizations of the NeoBERT model, offering various quantized versions for efficient usage.
đ Quick Start
If you're new to using GGUF files, check out TheBloke's READMEs for detailed guidance, including how to concatenate multi - part files.
⨠Features
- Static quantizations of the [chandar - lab/NeoBERT](https://huggingface.co/chandar - lab/NeoBERT) model.
- A variety of quantized versions are available, sorted by size.
đĻ Installation
No specific installation steps are provided in the original document.
đģ Usage Examples
No code examples are provided in the original document.
đ Documentation
About
This project offers static quants of [chandar - lab/NeoBERT](https://huggingface.co/chandar - lab/NeoBERT). Currently, weighted/imatrix quants seem unavailable. If they don't appear about a week after the static ones, it might be that they are not planned. You can request them by opening a Community Discussion.
Provided Quants
The provided quantized models are sorted by size (not necessarily quality). IQ - quants are often preferable over similar - sized non - IQ quants.
| Link |
Type |
Size/GB |
Notes |
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.Q2_K.gguf) |
Q2_K |
0.2 |
|
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.Q3_K_S.gguf) |
Q3_K_S |
0.2 |
|
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.Q3_K_M.gguf) |
Q3_K_M |
0.2 |
lower quality |
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.IQ4_XS.gguf) |
IQ4_XS |
0.2 |
|
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.Q3_K_L.gguf) |
Q3_K_L |
0.2 |
|
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.Q4_K_S.gguf) |
Q4_K_S |
0.2 |
fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.Q4_K_M.gguf) |
Q4_K_M |
0.2 |
fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.Q5_K_S.gguf) |
Q5_K_S |
0.3 |
|
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.Q5_K_M.gguf) |
Q5_K_M |
0.3 |
|
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.Q6_K.gguf) |
Q6_K |
0.3 |
very good quality |
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.Q8_0.gguf) |
Q8_0 |
0.3 |
fast, best quality |
| [GGUF](https://huggingface.co/mradermacher/NeoBERT - GGUF/resolve/main/NeoBERT.f16.gguf) |
f16 |
0.5 |
16 bpw, overkill |
Here is a useful graph by ikawrakow comparing some lower - quality quant types (lower is better):
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
FAQ / Model Request
For answers to your questions or if you want other models to be quantized, visit Model Requests.
đ§ Technical Details
No technical details are provided in the original document.
đ License
The project is licensed under the MIT license.
đ Thanks
I'm grateful to my company, nethype GmbH, for allowing me to use its servers and upgrading my workstation, enabling this work during my free time.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
