
Model Overview
Model Features
Model Capabilities
Use Cases
🚀 Pivot 0.1 Evil A
This project provides GGUF format model files for Pivot 0.1 Evil A, offering various quantization options for different inference needs.
🚀 Quick Start
If you want to use this model, you can choose the appropriate quantization file according to your hardware and requirements. For example, if you have limited memory, you can choose a smaller quantization file like pivot-0.1-evil-a.Q2_K.gguf
.
✨ Features
- Multiple Quantization Options: Different quantization methods are provided, such as Q2_K, Q3_K, Q4_K, etc., to meet the needs of different hardware and performance requirements.
- Wide Compatibility: Compatible with many clients and libraries, including llama.cpp, text-generation-webui, KoboldCpp, etc.
- Multilingual Support: Supports both English and Korean.
📦 Installation
Download GGUF Files
- Automatic Download: Clients/libraries like LM Studio, LoLLMS Web UI, and Faraday.dev can automatically download models for you.
- Manual Download: You can use the
huggingface-hub
Python library to download individual model files.pip3 install huggingface-hub huggingface-cli download TheBloke/PiVoT-0.1-Evil-a-GGUF pivot-0.1-evil-a.Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False
💻 Usage Examples
Basic Usage
./main -ngl 32 -m pivot-0.1-evil-a.Q4_K_M.gguf --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction:\n{prompt}\n\n### Response:"
Advanced Usage
from ctransformers import AutoModelForCausalLM
# Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
llm = AutoModelForCausalLM.from_pretrained("TheBloke/PiVoT-0.1-Evil-a-GGUF", model_file="pivot-0.1-evil-a.Q4_K_M.gguf", model_type="mistral", gpu_layers=50)
print(llm("AI is going to"))
📚 Documentation
About GGUF
GGUF is a new format introduced by the llama.cpp team on August 21st 2023, replacing GGML which is no longer supported by llama.cpp. Many clients and libraries support GGUF, such as llama.cpp, text-generation-webui, KoboldCpp, etc.
Repositories Available
- AWQ model(s) for GPU inference.
- GPTQ models for GPU inference, with multiple quantisation parameter options.
- 2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference
- Jeonghwan Park's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions
Prompt Template
### Instruction:
{prompt}
### Response:
Compatibility
These quantised GGUFv2 files are compatible with llama.cpp from August 27th onwards, as of commit d0cee0d. They are also compatible with many third party UIs and libraries.
Explanation of Quantisation Methods
Click to see details
The new methods available are:
- GGML_TYPE_Q2_K - "type-1" 2-bit quantization in super-blocks containing 16 blocks, each block having 16 weight. Block scales and mins are quantized with 4 bits. This ends up effectively using 2.5625 bits per weight (bpw)
- GGML_TYPE_Q3_K - "type-0" 3-bit quantization in super-blocks containing 16 blocks, each block having 16 weights. Scales are quantized with 6 bits. This end up using 3.4375 bpw.
- GGML_TYPE_Q4_K - "type-1" 4-bit quantization in super-blocks containing 8 blocks, each block having 32 weights. Scales and mins are quantized with 6 bits. This ends up using 4.5 bpw.
- GGML_TYPE_Q5_K - "type-1" 5-bit quantization. Same super-block structure as GGML_TYPE_Q4_K resulting in 5.5 bpw
- GGML_TYPE_Q6_K - "type-0" 6-bit quantization. Super-blocks with 16 blocks, each block having 16 weights. Scales are quantized with 8 bits. This ends up using 6.5625 bpw
Refer to the Provided Files table below to see what files use which methods, and how.
Provided Files
Name | Quant method | Bits | Size | Max RAM required | Use case |
---|---|---|---|---|---|
pivot-0.1-evil-a.Q2_K.gguf | Q2_K | 2 | 3.08 GB | 5.58 GB | smallest, significant quality loss - not recommended for most purposes |
pivot-0.1-evil-a.Q3_K_S.gguf | Q3_K_S | 3 | 3.16 GB | 5.66 GB | very small, high quality loss |
pivot-0.1-evil-a.Q3_K_M.gguf | Q3_K_M | 3 | 3.52 GB | 6.02 GB | very small, high quality loss |
pivot-0.1-evil-a.Q3_K_L.gguf | Q3_K_L | 3 | 3.82 GB | 6.32 GB | small, substantial quality loss |
pivot-0.1-evil-a.Q4_0.gguf | Q4_0 | 4 | 4.11 GB | 6.61 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
pivot-0.1-evil-a.Q4_K_S.gguf | Q4_K_S | 4 | 4.14 GB | 6.64 GB | small, greater quality loss |
pivot-0.1-evil-a.Q4_K_M.gguf | Q4_K_M | 4 | 4.37 GB | 6.87 GB | medium, balanced quality - recommended |
pivot-0.1-evil-a.Q5_0.gguf | Q5_0 | 5 | 5.00 GB | 7.50 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
pivot-0.1-evil-a.Q5_K_S.gguf | Q5_K_S | 5 | 5.00 GB | 7.50 GB | large, low quality loss - recommended |
pivot-0.1-evil-a.Q5_K_M.gguf | Q5_K_M | 5 | 5.13 GB | 7.63 GB | large, very low quality loss - recommended |
pivot-0.1-evil-a.Q6_K.gguf | Q6_K | 6 | 5.94 GB | 8.44 GB | very large, extremely low quality loss |
pivot-0.1-evil-a.Q8_0.gguf | Q8_0 | 8 | 7.70 GB | 10.20 GB | very large, extremely low quality loss - not recommended |
Note: the above RAM figures assume no GPU offloading. If layers are offloaded to the GPU, this will reduce RAM usage and use VRAM instead.
How to Run in Different Environments
- llama.cpp: Make sure you are using
llama.cpp
from commit d0cee0d or later../main -ngl 32 -m pivot-0.1-evil-a.Q4_K_M.gguf --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction:\n{prompt}\n\n### Response:"
- text-generation-webui: Further instructions can be found in the text-generation-webui documentation, here: text-generation-webui/docs/04 ‚Äê Model Tab.md.
- Python Code: You can use GGUF models from Python using the llama-cpp-python or ctransformers libraries.
from ctransformers import AutoModelForCausalLM # Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system. llm = AutoModelForCausalLM.from_pretrained("TheBloke/PiVoT-0.1-Evil-a-GGUF", model_file="pivot-0.1-evil-a.Q4_K_M.gguf", model_type="mistral", gpu_layers=50) print(llm("AI is going to"))
- LangChain: Guides on using llama-cpp-python and ctransformers with LangChain are available:
🔧 Technical Details
Model Information
Property | Details |
---|---|
Model Type | mistral |
Training Data | maywell/ko_wikidata_QA, kyujinpy/OpenOrca-KO, Anthropic/hh-rlhf |
Language | en, ko |
License | cc-by-sa-4.0 |
Model Creator | Jeonghwan Park |
Quantized By | TheBloke |
Tags | not-for-all-audiences |
Original Model Details
PivoT is a finetuned model based on Mistral 7B. It is a variation from Synatra v0.3 RP which has shown decent performance. PiVoT-0.1-Evil-a is an Evil tuned Version of PiVoT. It is finetuned by a specific method. PiVot-0.1-Evil-b has Noisy Embedding tuned and would have more variety in results.
📄 License
This project is licensed under the cc-by-sa-4.0 license.
Discord
For further support, and discussions on these models and AI in general, join us at:
Thanks, and how to contribute
Thanks to the chirper.ai team and Clay from gpus.llm-utils.org!
If you're able and willing to contribute, it will be most gratefully received and will help to keep providing more models and start work on new AI projects. Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.
- Patreon: https://patreon.com/TheBlokeAI
- Ko-Fi: https://ko-fi.com/TheBlokeAI
Special thanks to: Aemon Algiz.
Patreon special mentions: Brandon Frisco, LangChain4j, Spiking Neurons AB, transmissions 11, Joseph William Delisle, Nitin Borwankar, Willem Michiel, Michael Dempsey, vamX, Jeffrey Morgan, zynix, jjj, Omer Bin Jawed, Sean Connelly, jinyuan sun, Jeromy Smith, Shadi, Pawan Osman, Chadd, Elijah Stavena, Illia Dulskyi, Sebastain Graf, Stephen Murray, terasurfer, Edmond Seymore, Celu Ramasamy, Mandus, Alex, biorpg, Ajan Kanaga, Clay Pascal, Raven Klaugh, ÈòøÊòé, K, ya boyyy, usrbinkat, Alicia Loh, John Villwock, ReadyPlayerEmma, Chris Smitley, Cap'n Zoog, fincy, GodLy, S_X, sidney chen, Cory Kujawski, OG, Mano Prime, AzureBlack, Pieter, Kalila, Spencer Kim, Tom X Nguyen, Stanislav Ovsiannikov, Michael Levine, Andrey, Trailburnt, Vadim, Enrico Ros, Talal Aujan, Brandon Phillips, Jack West, Eugene Pentland, Michael Davis, Will Dee, webtim, Jonathan Leane, Alps Aficionado, Rooh Singh, Tiffany J. Kim, theTransient, Luke @flexchar, Elle, Caitlyn Gatomon, Ari Malik, subjectnull, Johann-Peter Hartmann, Trenton Dambrowitz, Imad Khwaja, Asp the Wyvern, Emad Mostaque, Rainer Wilmers, Alexandros Triantafyllidis, Nicholas, Pedro Madruga, SuperWojo, Harry Royden McLaughlin, James Bentley, Olakabola, David Ziegler, Ai Maven, Jeff Scroggin, Nikolai Manek, Deo Leter, Matthew Berman, Fen Risland, Ken Nordquist, Manuel Alberto Morcote, Luke Pendergrass, TL, Fred von Graf, Randy H, Dan Guido, NimbleBox.ai, Vitor Caleffi, Gabriel Tamborski, knownsqashed, Lone Striker, Erik Bj√§reholt, John Detwiler, Leonard Tan, Iucharbius
Thank you to all the generous patrons and donaters! And thank you again to a16z for their generous grant.

