đ Clinician-Note-2.0a Quantized Model
This project provides quantized versions of the Clinician-Note-2.0a model, offering various quantization options for different use cases in medical text generation and summarization.
đ Quick Start
If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi - part files.
⨠Features
- Multiple Quantization Options: Offers a wide range of quantized models sorted by size, with different quality levels.
- Medical - Focused: Suitable for text generation, summarization, and other medical - related tasks.
- Access to Resources: Static quants are available at mradermacher/Clinician - Note - 2.0a - GGUF.
đĻ Installation
No specific installation steps are provided in the original document.
đ Documentation
About
Weighted/imatrix quants of ClinicianFOCUS/Clinician - Note - 2.0a. Static quants are available at mradermacher/Clinician - Note - 2.0a - GGUF.
Provided Quants
(sorted by size, not necessarily quality. IQ - quants are often preferable over similar sized non - IQ quants)
Link |
Type |
Size/GB |
Notes |
GGUF |
i1 - IQ1_S |
1.2 |
for the desperate |
GGUF |
i1 - IQ1_M |
1.2 |
mostly desperate |
GGUF |
i1 - IQ2_XXS |
1.3 |
|
GGUF |
i1 - IQ2_XS |
1.5 |
|
GGUF |
i1 - IQ2_S |
1.5 |
|
GGUF |
i1 - IQ2_M |
1.6 |
|
GGUF |
i1 - Q2_K_S |
1.7 |
very low quality |
GGUF |
i1 - Q2_K |
1.8 |
IQ3_XXS probably better |
GGUF |
i1 - IQ3_XXS |
1.8 |
lower quality |
GGUF |
i1 - IQ3_XS |
1.9 |
|
GGUF |
i1 - Q3_K_S |
2.0 |
IQ3_XS probably better |
GGUF |
i1 - IQ3_S |
2.0 |
beats Q3_K* |
GGUF |
i1 - IQ3_M |
2.1 |
|
GGUF |
i1 - Q3_K_M |
2.2 |
IQ3_S probably better |
GGUF |
i1 - Q3_K_L |
2.3 |
IQ3_M probably better |
GGUF |
i1 - IQ4_XS |
2.4 |
|
GGUF |
i1 - Q4_0 |
2.5 |
fast, low quality |
GGUF |
i1 - IQ4_NL |
2.5 |
prefer IQ4_XS |
GGUF |
i1 - Q4_K_S |
2.5 |
optimal size/speed/quality |
GGUF |
i1 - Q4_K_M |
2.6 |
fast, recommended |
GGUF |
i1 - Q4_1 |
2.7 |
|
GGUF |
i1 - Q5_K_S |
2.9 |
|
GGUF |
i1 - Q5_K_M |
3.0 |
|
GGUF |
i1 - Q6_K |
3.4 |
practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower - quality quant types (lower is better):

And here are Artefact2's thoughts on the matter:
Artefact2's Gist
FAQ / Model Request
See mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.
Thanks
I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
đ License
This project is licensed under the AGPL - 3.0 license.