PocketDoc_Dans-PersonalityEngine Open Source Large Language Model - Freely Achieve Role-Playing, Story Creation, and Professional Tasks

Pocketdoc Dans PersonalityEngine V1.3.0 12b GGUF

Developed by bartowski

A 12B-parameter multilingual large language model based on llama.cpp quantization, supporting role-play, story creation, and multi-domain professional tasks

Large Language Model Open Source License:Apache-2.0 #Multi-domain Text Generation #High-precision Quantization #Role-play Optimization

Downloads 1,027

Release Time : 5/24/2025

Model Overview

This model is a 12B-parameter large language model developed by PocketDoc, optimized through llama.cpp quantization. It supports multiple languages and professional domain tasks, excelling particularly in role-play and creative text generation.

Model Features

Multilingual Support

Supports text generation and understanding in 9 languages

Multi-domain Adaptation

Covers professional fields including chemistry, biology, programming, law, medicine, finance, and more

Efficient Quantization

Offers multiple quantization levels from Q2 to Q8 to adapt to different hardware environments

Role-play Optimization

Specially optimized for role-play scenarios through training

Model Capabilities

Multilingual text generation

Role-play dialogue

Story creation

Professional domain Q&A

Code generation

Legal document processing

Medical information analysis

Use Cases

Creative Writing

Story Creation

Generates coherent long stories or novel chapters

Can produce creative texts with plot coherence

Role-play

Simulates dialogues and interactions of different characters

Maintains consistent character personalities in dialogues

Professional Assistance

Legal Document Processing

Assists in drafting and analyzing legal documents

Can generate professional texts compliant with legal standards

Medical Information Query

Answers medical-related questions

Provides professional responses based on medical knowledge

🚀 Llamacpp imatrix Quantizations of Dans-PersonalityEngine-V1.3.0-12b by PocketDoc

This project offers quantized versions of the Dans-PersonalityEngine-V1.3.0-12b model, leveraging the llama.cpp framework. It provides various quantization types to meet different performance and quality requirements, enabling users to run the model efficiently on different hardware platforms.

🚀 Quick Start

Quantization Details

The project uses llama.cpp release b5466 for quantization. The original model can be found at https://huggingface.co/PocketDoc/Dans-PersonalityEngine-V1.3.0-12b. All quantizations are made using the imatrix option with a dataset from here.

Running the Model

Using LM Studio: You can run the quantized models in LM Studio.
Using llama.cpp: Run the models directly with llama.cpp, or any other llama.cpp - based project.

✨ Features

Supported Languages

en, ar, de, fr, es, hi, pt, ja, ko

Datasets

The model is trained on a wide range of datasets, including:

PocketDoc/Dans-Prosemaxx-RP
PocketDoc/Dans-Personamaxx-Logs-2
... (and many others as listed in the original document)

Base Model

Base model: PocketDoc/Dans-PersonalityEngine-V1.3.0-12b
Thumbnail: https://huggingface.co/PocketDoc/Dans-PersonalityEngine-V1.3.0-12b/resolve/main/resources/pe.png
Base model relation: quantized
License: apache - 2.0

📦 Installation

Prompt Format

The prompt format for the model is as follows:

[gMASK]<sop><|system|>{system_prompt}<|endoftext|><|user|>{prompt}<|endoftext|><|assistant|>

Downloading Files

You can download specific files from the following table:

Filename	Quant type	File Size	Split	Description
Dans-PersonalityEngine-V1.3.0-12b-bf16.gguf	bf16	24.50GB	false	Full BF16 weights.
Dans-PersonalityEngine-V1.3.0-12b-Q8_0.gguf	Q8_0	13.02GB	false	Extremely high quality, generally unneeded but max available quant.
... (and other files as listed in the original document)	...	...	...	...

Embed/Output Weights

Some of the quantizations (Q3_K_XL, Q4_K_L etc) use the standard quantization method, with the embeddings and output weights quantized to Q8_0 instead of the default values.

Downloading using huggingface - cli

Click to view download instructions

First, make sure you have hugginface - cli installed:

pip install -U "huggingface_hub[cli]"

Then, you can target the specific file you want:

huggingface-cli download bartowski/PocketDoc_Dans-PersonalityEngine-V1.3.0-12b-GGUF --include "PocketDoc_Dans-PersonalityEngine-V1.3.0-12b-Q4_K_M.gguf" --local-dir ./

If the model is bigger than 50GB, it will have been split into multiple files. To download them all to a local folder, run:

huggingface-cli download bartowski/PocketDoc_Dans-PersonalityEngine-V1.3.0-12b-GGUF --include "PocketDoc_Dans-PersonalityEngine-V1.3.0-12b-Q8_0/*" --local-dir ./

You can either specify a new local - dir (PocketDoc_Dans-PersonalityEngine-V1.3.0-12b-Q8_0) or download them all in place (./)

🔧 Technical Details

ARM/AVX Information

Previously, Q4_0_4_4/4_8/8_8 were downloaded, and their weights were interleaved in memory to improve performance on ARM and AVX machines by loading more data in one pass.

Now, there is "online repacking" for weights. Details can be found in this PR. If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.

As of llama.cpp build b4282, you cannot run the Q4_0_X_X files and need to use Q4_0 instead.

Additionally, if you want slightly better quality, you can use IQ4_NL thanks to this PR, which will also repack the weights for ARM (only the 4_4 for now). The loading time may be slower, but it will result in an overall speed increase.

Click to view Q4_0_X_X information (deprecated)

I'm keeping this section to show the potential theoretical uplift in performance from using the Q4

📄 License

The project is licensed under the apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご