Psyonic Cetacean Ultra Quality 20b GGUF Imat Plus2

🚀 Ultra Quality High Remaster of the incredible: Psyonic - Cetacean 20b - Imatrix Plus 2

This project offers a high - quality remaster of the Psyonic - Cetacean 20b - Imatrix Plus 2 model. It focuses on enhancing precision and performance in text generation, especially in creative writing, role - playing, and other text - based scenarios.

✨ Features

• Floating Point 32 Upscale: All components and merges are remastered to floating point 32, including all merges recreated with master files and, where possible, substituting full FP32 models.
• New Imatrix Plus 2 Quants: Using a new in - house dataset merged with a master dataset to boost the performance of the ultra - quality remaster.
• High Precision: Carries forward maximum precision right up to the "GUFFed" point, with an F32 master file for GGUF at 78 GB.
• Performance Improvement: Significantly reduces perplexity across different quantization levels (Q2K, Q4KM, Q6, Q8), leading to better instruction following, new abilities, and improved prose quality.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

There are no specific code examples in the original document. However, for text generation, you can use common AI - based text - generation interfaces:

• In "KoboldCpp", "oobabooga/text - generation - webui", or "Silly Tavern", you can load the model and start generating text.
• For "text - generation - webui", if using GGUFs, you need to use "llama_HF" and download some config files from the source version of the model: [https://huggingface.co/collections/DavidAU/d - au - source - files - for - gguf - exl2 - awq - gptq - hqq - etc - etc - 66b55cb8ba25f914cbf210be](https://huggingface.co/collections/DavidAU/d - au - source - files - for - gguf - exl2 - awq - gptq - hqq - etc - etc - 66b55cb8ba25f914cbf210be)

Advanced Usage

• Smoothing Factor:
  • In "KoboldCpp" -> Settings -> Samplers -> Advanced -> "Smooth_F", set the "Smoothing_factor" to 1.5 to 2.5.
  • In "text - generation - webui" -> parameters -> lower right, set the "Smoothing_factor" to 1.5 to 2.5.
  • In "Silly Tavern", set the "Smoothing" to 1.5 to 2.5.
• Other Options:
  • Increase rep pen to 1.1 to 1.15 (not necessary if using "smoothing_factor").
  • If the interface supports "Quadratic Sampling" ("smoothing"), make the adjustment as noted.

📚 Documentation

• Model Performance:

  • The difference between F32 and BF16 is over 8 decimal places. Each merge and model modification causes "losses", which accumulate and affect model performance.
  • The remastered model shows significant drops in perplexity at different quantization levels:
    • At Q2K, a drop of 533 points in perplexity (VS: Q2K original base model: PPL = 9.8077 +/- 0.06821).
    • At Q4KM, a drop of 976 points in perplexity (VS: Q4km original base model -> PPL = 8.7858 +/- 0.06074).
    • At Q6, a drop of 234 points in perplexity (VS: Q6 original base model -> PPL = 8.6070 +/- 0.05907).
    • At Q8, a 150 - point improvement (PPL = 8.5850 +/- 0.05881 VS: BASE/ORGINAL: PPL = 8.6012 +/- 0.05900).

• User Feedback: According to Jeb Carter, the original creator of the model:

  • Instruction following has improved dramatically.
  • New abilities have emerged.
  • The need for specific instructions has reduced.
  • Prose, nuance, and depth have all improved.
  • Known issues with the original model have disappeared.

• Optimal Settings:

  • This is a "Class 2" model. For all settings, example generations, advanced settings guide, and methods to improve model performance for all use cases (including chat and role - play), please see [https://huggingface.co/DavidAU/Maximizing - Model - Performance - All - Quants - Types - And - Full - Precision - by - Samplers_Parameters](https://huggingface.co/DavidAU/Maximizing - Model - Performance - All - Quants - Types - And - Full - Precision - by - Samplers_Parameters)

🔧 Technical Details

The project focuses on maximizing precision by using floating point 32 upscale. All components and merges are remastered to FP32, and a new in - house dataset is merged with a master dataset. The methods used ensure that precision loss is minimized or eliminated, which is mathematically and theoretically sound.

📄 License

This project is licensed under the Apache - 2.0 license.