Compumacy Experimental 32B I1 GGUF

🚀 Compumacy-Experimental-32B Quantized Model

This project provides quantized versions of the Compumacy-Experimental-32B model, which is useful for various text - generation and medical - related tasks.

🚀 Quick Start

If you want to start using the quantized model, you need to know how to handle GGUF files. For detailed guidance, refer to TheBloke's READMEs, which also includes instructions on concatenating multi - part files.

✨ Features

• Quantized Model: Offers weighted/imatrix quants of the base model https://huggingface.co/Daemontatox/Compumacy-Experimental-32B.
• Multiple Quant Types: Provides a variety of quant types with different sizes and qualities, sorted by size.
• Useful Resources: Includes a comparison graph of lower - quality quant types and relevant thoughts from Artefact2.

📚 Documentation

About

The model is based on Daemontatox/Compumacy-Experimental-32B and uses datasets from Daemontatox/Psy-Data-books. The language supported is English, and it uses the transformers library. The license is apache - 2.0, and it is quantized by mradermacher.

The weighted/imatrix quants of the base model can be found at https://huggingface.co/Daemontatox/Compumacy-Experimental-32B. Static quants are available at https://huggingface.co/mradermacher/Compumacy-Experimental-32B-GGUF.

Provided Quants

The following table shows the provided quants, sorted by size (not necessarily quality. IQ - quants are often preferable over similar sized non - IQ quants):

Here is a handy graph by ikawrakow comparing some lower - quality quant types (lower is better):

And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

FAQ / Model Request

If you have any questions or want to request another model to be quantized, see https://huggingface.co/mradermacher/model_requests for answers.

📄 License

The model is licensed under the apache - 2.0 license.

Thanks

I express my gratitude to my company, nethype GmbH, for allowing me to use its servers and upgrading my workstation, which enables me to do this work in my free time. Additionally, I thank @nicoboss for giving me access to his private supercomputer. This access allows me to provide many more imatrix quants at a much higher quality than I could otherwise.