đ Medra Quantized Model
This project provides static quantizations of the Medra model, offering various quantized versions for different use - cases in medical AI tasks.
đ Quick Start
If you are new to using this model, you can start by referring to the usage section below to understand how to work with the provided quantized files.
⨠Features
- Multi - dataset fine - tuned: Fine - tuned on multiple medical datasets such as
qiaojin/PubMedQA
, Mreeb/Dermatology - Question - Answer - Dataset - For - Fine - Tuning
, and lavita/MedQuAD
.
- Multilingual support: Supports languages including English (
en
) and Romanian (ro
).
- Diverse tasks: Suitable for tasks like text generation, medical question - answering, summarization, and dermatology - related tasks.
đĻ Installation
No specific installation steps are provided in the original document.
đ Documentation
General Information
Property |
Details |
Base Model |
drwlf/Medra |
Datasets |
qiaojin/PubMedQA, Mreeb/Dermatology - Question - Answer - Dataset - For - Fine - Tuning, lavita/MedQuAD |
Language |
en, ro |
Library Name |
transformers |
License |
apache - 2.0 |
Quantized By |
mradermacher |
Tags |
text - generation, medical - ai, question - answering, summarization, dermatology, gemma - 3, qlora, unsloth, fine - tuned |
About
This project provides static quants of drwlf/Medra. Weighted/imatrix quants are available at [mradermacher/Medra - i1 - GGUF](https://huggingface.co/mradermacher/Medra - i1 - GGUF).
Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's READMEs](https://huggingface.co/TheBloke/KafkaLM - 70B - German - V0.1 - GGUF) for more details, including on how to concatenate multi - part files.
Provided Quants
(sorted by size, not necessarily quality. IQ - quants are often preferable over similar sized non - IQ quants)
Link |
Type |
Size/GB |
Notes |
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.Q2_K.gguf) |
Q2_K |
1.8 |
|
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.Q3_K_S.gguf) |
Q3_K_S |
2.0 |
|
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.Q3_K_M.gguf) |
Q3_K_M |
2.2 |
lower quality |
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.Q3_K_L.gguf) |
Q3_K_L |
2.3 |
|
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.IQ4_XS.gguf) |
IQ4_XS |
2.4 |
|
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.Q4_K_S.gguf) |
Q4_K_S |
2.5 |
fast, recommended |
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.Q4_K_M.gguf) |
Q4_K_M |
2.6 |
fast, recommended |
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.Q5_K_S.gguf) |
Q5_K_S |
2.9 |
|
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.Q5_K_M.gguf) |
Q5_K_M |
2.9 |
|
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.Q6_K.gguf) |
Q6_K |
3.3 |
very good quality |
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.Q8_0.gguf) |
Q8_0 |
4.2 |
fast, best quality |
[GGUF](https://huggingface.co/mradermacher/Medra - GGUF/resolve/main/Medra.f16.gguf) |
f16 |
7.9 |
16 bpw, overkill |
Here is a handy graph by ikawrakow comparing some lower - quality quant types (lower is better):

And here are Artefact2's thoughts on the matter:
Artefact2's Gist
FAQ / Model Request
See mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.
Thanks
I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
đ License
This project is licensed under the apache - 2.0
license.