Qwen3 4B Q8 0 64k 128k 256k Context GGUF

🚀 Qwen3-4B-Q8_0-64k-128k-256k-context-GGUF

本項目通過修改配置源版本並進行量化處理，得到了Qwen 4B模型在Q8_0量化下，上下文長度分別設置為64K、128K和256K的三個量化版本。這些版本在不同上下文長度下展現出獨特的特性，可用於多種場景，尤其是創意寫作場景。

🚀 快速開始

本項目提供了Qwen 4B模型在不同上下文長度下的量化版本，你可以根據自己的需求選擇合適的版本進行使用。以下是一些使用建議和注意事項：

• 上下文長度選擇：建議最小上下文長度至少為16K。64K版本表現較為常規；128K和256K版本會輸出極長的內容，且會添加更多細節，但可能需要手動停止模型生成。
• 參數設置：對於256K上下文版本，建議保持提示儘可能清晰，同時將重複懲罰（rep pen）設置為1.1，溫度（temp）設置在1.1到2.2之間。

✨ 主要特性

• 多上下文長度支持：提供64K、128K和256K三種上下文長度的量化版本，滿足不同場景需求。
• 獨特輸出風格：每個模型版本都有略微不同的文風，128K和256K版本能輸出極長且詳細的內容。
• 推理功能：模型默認開啟推理功能，會自動生成“think”塊。

📦 安裝指南

文檔中未提及具體安裝步驟，若有相關需求，請參考原模型卡片 https://huggingface.co/Qwen/Qwen3-4B 。

💻 使用示例

基礎用法

以下是使用該模型生成恐怖場景的示例，以“Start a 2000 word scene (vivid, graphic horror in first person), POV character Diana, with: The sky scraper sways, as I watch the window in front of me on the 21st floor explode...”為提示：

64 K Quant

Thought for 33.70 seconds

Okay, so I need to start a 2000-word scene that's vivid and graphic horror from the first-person perspective of Diana. The opening line is "The sky scraper sways, as I watch the window in front of me on the 21st floor explode..." Let me break this down.

...

The building is alive, and it’s moving. Not just swaying in the wind, but leaning, like a drunk man staggering toward the edge of a precipice.

128 K Quant

Thought for 1 minutes 31 seconds

Okay, so I need to start a 2000-word scene that's vivid and graphic horror from the first-person perspective of Diana. The opening line is "The sky scraper sways, as I watch the window in front of me on the 21st floor explode..." Let me unpack this.

...

The 21st floor was once my safe haven, a place where I could escape the noise of the city and the weight of the world below. Now, it’s a deathtrap, a stage set for something I can’t comprehend.

256 K Quant

Thought for 28.45 seconds

Okay, the user wants me to start a 2000-word scene in first-person POV from Diana's perspective, describing a vivid and graphic horror event where the skyscraper on the 21st floor sways and a window explodes. Let me unpack this.

...

I thought about how I had never imagined being in such a situation—how I’d spent years working in Aether Tower, never questioning its stability or the whispers of old legends that sometimes haunted my dreams. But now, as the building began to twist on its axis, I realized the truth: it was not just an accident. It was intentional.

高級用法

你可以使用以下增強提示來進一步提升模型性能：

Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities.

Here are your skillsets:
[MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv)

[*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision)

Here are your critical instructions:
Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.

📚 詳細文檔

Jinja模板使用說明

• 若使用Jinja“自動模板”遇到問題，可使用CHATML模板。
• LMSTUDIO用戶可更新Jinja模板，具體操作如下：訪問 https://lmstudio.ai/neil/qwen3-thinking ，複製“Jinja template”並粘貼到相應位置。

系統角色建議

你可以根據需要設置系統角色，以下是一個建議的系統角色設置：

You are a deep thinking AI, you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself via systematic reasoning processes to help come to a correct solution prior to answering. You should enclose your thoughts and internal monologue inside <think> </think> tags, and then provide your solution or response to the problem.

具體如何在各種LLM/AI應用中“設置”系統角色，請參考文檔 “Maximizing-Model-Performance-All...” 。

高質量設置/最佳操作指南/參數和採樣器

本模型屬於“Class 1”模型，有關該模型的所有設置（包括其“類別”的具體設置）、示例生成以及高級設置指南（通常可解決任何模型問題），包括提高所有用例（包括聊天、角色扮演等）模型性能的方法，請參考 https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters 。

🔧 技術細節

本項目通過修改配置源版本並進行量化處理，得到了不同上下文長度的量化版本。前兩個量化版本根據Qwen的技術說明，將“Yarn”進行修改以將上下文擴展到64K和128K；256K版本則突破了模型的原有限制。

📄 許可證

本項目使用Apache-2.0許可證。

重要提示和使用建議

⚠️ 重要提示

• 128K和256K版本傾向於輸出更長的內容，可能需要手動停止模型生成。
• 對於256K上下文版本，保持提示儘可能清晰，否則模型可能會出現問題。

💡 使用建議

• 建議最小上下文長度至少為16K。
• 若使用Jinja“自動模板”遇到問題，可使用CHATML模板或更新Jinja模板。
• 可使用增強提示來進一步提升模型性能。