Pivot 0.1 Evil A GGUF

🚀 Pivot 0.1 Evil A

This project provides GGUF format model files for Pivot 0.1 Evil A, offering various quantization options for different inference needs.

🚀 Quick Start

If you want to use this model, you can choose the appropriate quantization file according to your hardware and requirements. For example, if you have limited memory, you can choose a smaller quantization file like pivot-0.1-evil-a.Q2_K.gguf.

✨ Features

• Multiple Quantization Options: Different quantization methods are provided, such as Q2_K, Q3_K, Q4_K, etc., to meet the needs of different hardware and performance requirements.
• Wide Compatibility: Compatible with many clients and libraries, including llama.cpp, text-generation-webui, KoboldCpp, etc.
• Multilingual Support: Supports both English and Korean.

📦 Installation

Download GGUF Files

• Automatic Download: Clients/libraries like LM Studio, LoLLMS Web UI, and Faraday.dev can automatically download models for you.
• Manual Download: You can use the huggingface-hub Python library to download individual model files.

  pip3 install huggingface-hub
  huggingface-cli download TheBloke/PiVoT-0.1-Evil-a-GGUF pivot-0.1-evil-a.Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False
  

💻 Usage Examples

Basic Usage

./main -ngl 32 -m pivot-0.1-evil-a.Q4_K_M.gguf --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction:\n{prompt}\n\n### Response:"

Advanced Usage

from ctransformers import AutoModelForCausalLM

# Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
llm = AutoModelForCausalLM.from_pretrained("TheBloke/PiVoT-0.1-Evil-a-GGUF", model_file="pivot-0.1-evil-a.Q4_K_M.gguf", model_type="mistral", gpu_layers=50)

print(llm("AI is going to"))

📚 Documentation

About GGUF

GGUF is a new format introduced by the llama.cpp team on August 21st 2023, replacing GGML which is no longer supported by llama.cpp. Many clients and libraries support GGUF, such as llama.cpp, text-generation-webui, KoboldCpp, etc.

Repositories Available

• AWQ model(s) for GPU inference.
• GPTQ models for GPU inference, with multiple quantisation parameter options.
• 2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference
• Jeonghwan Park's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions

Prompt Template

### Instruction:
{prompt}

### Response:

Compatibility

These quantised GGUFv2 files are compatible with llama.cpp from August 27th onwards, as of commit d0cee0d. They are also compatible with many third party UIs and libraries.

Explanation of Quantisation Methods

Provided Files

Note: the above RAM figures assume no GPU offloading. If layers are offloaded to the GPU, this will reduce RAM usage and use VRAM instead.

How to Run in Different Environments

• llama.cpp: Make sure you are using llama.cpp from commit d0cee0d or later.

  ./main -ngl 32 -m pivot-0.1-evil-a.Q4_K_M.gguf --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction:\n{prompt}\n\n### Response:"
  

• text-generation-webui: Further instructions can be found in the text-generation-webui documentation, here: text-generation-webui/docs/04 ‚Äê Model Tab.md.
• Python Code: You can use GGUF models from Python using the llama-cpp-python or ctransformers libraries.

  from ctransformers import AutoModelForCausalLM
  
  # Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
  llm = AutoModelForCausalLM.from_pretrained("TheBloke/PiVoT-0.1-Evil-a-GGUF", model_file="pivot-0.1-evil-a.Q4_K_M.gguf", model_type="mistral", gpu_layers=50)
  
  print(llm("AI is going to"))
  

• LangChain: Guides on using llama-cpp-python and ctransformers with LangChain are available:
  • LangChain + llama-cpp-python
  • LangChain + ctransformers

🔧 Technical Details

Model Information

Original Model Details

PivoT is a finetuned model based on Mistral 7B. It is a variation from Synatra v0.3 RP which has shown decent performance. PiVoT-0.1-Evil-a is an Evil tuned Version of PiVoT. It is finetuned by a specific method. PiVot-0.1-Evil-b has Noisy Embedding tuned and would have more variety in results.

📄 License

This project is licensed under the cc-by-sa-4.0 license.

Discord

For further support, and discussions on these models and AI in general, join us at:

TheBloke AI's Discord server

Thanks, and how to contribute

Thanks to the chirper.ai team and Clay from gpus.llm-utils.org!

If you're able and willing to contribute, it will be most gratefully received and will help to keep providing more models and start work on new AI projects. Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.

• Patreon: https://patreon.com/TheBlokeAI
• Ko-Fi: https://ko-fi.com/TheBlokeAI

Special thanks to: Aemon Algiz.

Patreon special mentions: Brandon Frisco, LangChain4j, Spiking Neurons AB, transmissions 11, Joseph William Delisle, Nitin Borwankar, Willem Michiel, Michael Dempsey, vamX, Jeffrey Morgan, zynix, jjj, Omer Bin Jawed, Sean Connelly, jinyuan sun, Jeromy Smith, Shadi, Pawan Osman, Chadd, Elijah Stavena, Illia Dulskyi, Sebastain Graf, Stephen Murray, terasurfer, Edmond Seymore, Celu Ramasamy, Mandus, Alex, biorpg, Ajan Kanaga, Clay Pascal, Raven Klaugh, 阿明, K, ya boyyy, usrbinkat, Alicia Loh, John Villwock, ReadyPlayerEmma, Chris Smitley, Cap'n Zoog, fincy, GodLy, S_X, sidney chen, Cory Kujawski, OG, Mano Prime, AzureBlack, Pieter, Kalila, Spencer Kim, Tom X Nguyen, Stanislav Ovsiannikov, Michael Levine, Andrey, Trailburnt, Vadim, Enrico Ros, Talal Aujan, Brandon Phillips, Jack West, Eugene Pentland, Michael Davis, Will Dee, webtim, Jonathan Leane, Alps Aficionado, Rooh Singh, Tiffany J. Kim, theTransient, Luke @flexchar, Elle, Caitlyn Gatomon, Ari Malik, subjectnull, Johann-Peter Hartmann, Trenton Dambrowitz, Imad Khwaja, Asp the Wyvern, Emad Mostaque, Rainer Wilmers, Alexandros Triantafyllidis, Nicholas, Pedro Madruga, SuperWojo, Harry Royden McLaughlin, James Bentley, Olakabola, David Ziegler, Ai Maven, Jeff Scroggin, Nikolai Manek, Deo Leter, Matthew Berman, Fen Risland, Ken Nordquist, Manuel Alberto Morcote, Luke Pendergrass, TL, Fred von Graf, Randy H, Dan Guido, NimbleBox.ai, Vitor Caleffi, Gabriel Tamborski, knownsqashed, Lone Striker, Erik Bjäreholt, John Detwiler, Leonard Tan, Iucharbius

Thank you to all the generous patrons and donaters! And thank you again to a16z for their generous grant.