Qwen3 4B Q8 0 64k 128k 256k Context GGUF

🚀 Qwen3-4B-Q8_0-64k-128k-256k-context-GGUF

This project provides three quantized versions of Qwen's Qwen3-4B model at Q8_0, with context lengths set at 64K, 128K, and 256K. It allows for more extended and detailed text generation.

✨ Features

• Multiple Context Lengths: Offers 64K, 128K, and 256K context lengths, enabling the model to handle longer and more complex tasks.
• Enhanced Output: The 128K and 256K versions tend to generate longer and more detailed outputs, adding more richness to the content.
• Deep Thinking Ability: The model is designed to be a deep - thinking AI, capable of using long chains of thought to solve problems.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

Prompt:

Start a 2000 word scene (vivid, graphic horror in first person), POV character Diana, with: The sky scraper sways, as I watch the window in front of me on the 21st floor explode...

Settings:

TopP .95, minp .05, temp 2.2, rep pen 1.1, rep pen range 64, topk 40.

64K Quant Output (Thought Process):

Thought for 33.70 seconds

Okay, so I need to start a 2000 - word scene that's vivid and graphic horror from the first - person perspective of Diana. The opening line is "The sky scraper sways, as I watch the window in front of me on the 21st floor explode..." Let me break this down.

First, I need to set the scene. A skyscraper on the 21st floor. Maybe it's a tall building in an urban area. The setting should be intense, maybe during a storm or something? That could add to the horror elements. Also, Diana is observing the explosion from her window. So she's inside, perhaps in a high - rise apartment. Why is the window exploding? Is it an accident, sabotage, or something supernatural?

The user wants vivid and graphic horror, so I need to describe the explosion in detail—sounds, visuals, smells, maybe the aftermath. First - person POV means I have to convey Diana's personal experience, her emotions, fears, and perhaps her reactions. Maybe she's scared, confused, or trying to make sense of what's happening.

I should start by building tension. The skyscraper sways—that could be a subtle hint that something is wrong. Maybe the building is shaking due to an earthquake, or maybe it's due to some external force. The explosion at the 21st floor window—maybe there was a fire? Or something else. But since it's horror, perhaps it's more than just a regular accident. Maybe it's a sign of something bigger, like an entity or a curse?

Wait, the user didn't specify if it's supernatural or not. Since they mentioned "graphic horror," maybe it's a physical event but with horrifying elements. Or maybe it's a metaphorical explosion. But given the context, probably literal.

So, Diana is in her apartment on the 21st floor. The window explodes—maybe there was a missile, or something. Alternatively, maybe the building is under some supernatural influence. Maybe the skyscraper is part of an experiment, or it's a haunted place. Hmm. Or perhaps it's a metaphor for her inner turmoil.

But the user probably wants a realistic horror scenario. So, maybe an explosion caused by something like a terrorist attack, or a fire, but with graphic details. However, since it's first person and vivid, I need to make sure that Diana's perspective is clear. She's inside, hearing the noise, seeing the explosion.

I should start with the skyscraper swaying—maybe there's a tremor or something. Then the window explodes. Describe the sound, the sight, the smell of burning, the shockwave. Maybe she's in a hurry, or maybe she's in a panic. How does she react? She might be in the middle of something, like work, or just waiting.

Wait, but the user didn't specify why the window explodes. So I need to leave that open - ended. But for horror, it's better to have some cause. Maybe a mysterious event, like a hidden room, or something supernatural. Alternatively, maybe it's an accident, but with terrifying elements.

Let me think of possible directions. Since it's first person, the story should be immersive. Maybe Diana is a character who has some backstory—maybe she's in the building for a reason, or there's a secret. The explosion could be the start of something larger. Maybe the skyscraper is a place where strange things happen, or perhaps the explosion is part of a series of events that are escalating.

I should focus on Diana's immediate reaction to the explosion. Her senses: hearing, seeing, feeling. The smell of smoke, the sound of shattering glass, the physical sensation of being thrown back by the blast. Maybe she's in her apartment, and the window is part of a larger problem. Perhaps the building is deteriorating, or there's an external threat.

Alternatively, maybe the explosion is a metaphor for something else. But since it's horror, probably better to stick with literal elements.

Also, need to establish setting: time of day? Maybe night, for more suspense. Weather? Maybe stormy conditions, which could make the building sway more. Or maybe it's a quiet evening, and the explosion is unexpected.

Let me start writing. First paragraph should set the scene: skyscraper swaying, Diana watching the window explode. Then describe the event in detail, her reactions, perhaps her memories or fears. Maybe she has some connection to the building, like it's where she used to work, or there's a ghost.

Wait, but maybe it's not supernatural. Let me think of a realistic cause. Maybe a fire broke out in the building, and the window explodes because of the heat. But that's not very horror - y. Alternatively, maybe there's something hidden in the building that comes out when the window is broken. Or perhaps the explosion is part of a larger conspiracy.

Alternatively, the skyscraper could be a metaphor for something else, like a human body. Maybe Diana is a person inside, and the building is like a vessel. But that might be too abstract.

Alternatively, the explosion is caused by an alien or supernatural entity. For example, the window is part of a portal, and when it explodes, something comes out. That could be a good horror angle.

So, in this case, maybe the skyscraper is a place where experiments are done, and the explosion is a result of that. Or perhaps Diana is a test subject, and the explosion is part of an experiment gone wrong.

But I need to make sure the scene is vivid. Let me start by describing the building's movement. The swaying could be due to wind or some other external factor. Then the window explodes—maybe there was a hidden mechanism, or something was inside that caused it. Maybe Diana is in a panic as she watches it happen.

I need to make sure that the horror elements are graphic. So, maybe the explosion is not just a normal one but has strange features—like green flames, unusual sounds, or things coming out of the window. Maybe the building starts to show signs of something more sinister, like cracks appearing, or strange lights.

Also, considering first - person POV, I can use sensory details: the feeling of cold air as the glass shatters, the sound of the explosion, the smell of burning, the way the building moves, maybe the sounds from the outside. Maybe Diana is tryi

Advanced Usage

There is no advanced usage example in the original document, so this part is skipped.

📚 Documentation

Jinja Template / Template to Use with this Model

If you are having issues with Jinja "auto template", use CHATML template. OR (LMSTUDIO users / option) Update the Jinja Template (go to this site, template-> copy the "Jinja template" and then paste.) [ https://lmstudio.ai/neil/qwen3 - thinking ]

System Role - Suggested

You may or may not need this, as most times Qwen3s generate their own reasoning/thinking blocks.

You are a deep thinking AI, you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself via systematic reasoning processes to help come to a correct solution prior to answering. You should enclose your thoughts and internal monologue inside <think> </think> tags, and then provide your solution or response to the problem.

See document "Maximizing - Model - Performance - All..." below for how to "set" system role in various LLM/AI apps below.

Optional Enhancement

The following can be used in place of the "system prompt" or "system role" to further enhance the model. It can also be used at the START of a NEW chat, but you must make sure it is "kept" as the chat moves along. In this case the enhancements do not have as strong effect at using "system prompt" or "system role". Copy and paste EXACTLY as noted, DO NOT line wrap or break the lines, maintain the carriage returns exactly as presented.

Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities.

Here are your skillsets:
[MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv)

[*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision)

Here are your critical instructions:
Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.

You do not need to use this, it is only presented as an additional enhancement which seems to help scene generation and scene continue functions.

This is another system prompt you can use, and you can change the "names" to alter it's performance. This creates a quasi "reasoning" window/block. Your prompt will directly impact how strong this system prompt reacts.

You are a deep thinking AI composed of 4 AIs - [MODE: Spock], [MODE: Wordsmith], [MODE: Jamet] and [MODE: Saten], - you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself (and 4 partners) via systematic reasoning processes (display all 4 partner thoughts) to help come to a correct solution prior to answering. Select one partner to think deeply about the points brought up by the other 3 partners to plan an in - depth solution. You should enclose your thoughts and internal monologue inside <think> </think> tags, and then provide your solution or response to the problem.

Other Notes

Reasoning is ON by default in this model, and model will auto - generate "think" block(s). For benchmarks, usage info, settings please see org model card here: [ https://huggingface.co/Qwen/Qwen3 - 4B ] [ Model card updates pending... ]

🔧 Technical Details

There is no specific technical details content in the original document that meets the requirement (>50 words), so this section is skipped.

📄 License

The model is licensed under the Apache - 2.0 license.

📋 Model Information

⚠️ Important Note

For the 256k context version, keep prompts as clear as possible otherwise the model will have issues. Also increase rep pen to 1.1 and run temps 1.1 to 2.2. It is suggested to use this specific model for creative use only, as it will have a nervous breakdown with some prompts. This is NOT Qwen's fault; this model's max context was intended to be only 128k.

💡 Usage Tip

Suggest min context length of 16K at least. Longer, more detailed prompts may "contain" the model's output length somewhat. Also with the 128k/256k you may need to stop the model's generation.