Llama3.1 MOE 4X8B Gated IQ Multi Tier Deep Reasoning 32B GGUF

🚀 Llama3.1-MOE-4X8B-Gated-IQ-Multi-Tier-Deep-Reasoning-32B-GGUF

Llama3.1-MOE-4X8B-Gated-IQ-Multi-Tier-Deep-Reasoning-32B-GGUF 是一款強大的文本生成模型，它結合了多個模型的優勢，具備深度推理能力，適用於各種場景，如創意寫作、問題解決等。

🚀 快速開始

• 語言支持：英語、法語、德語、西班牙語、葡萄牙語、意大利語、日語、韓語、俄語、中文、阿拉伯語、波斯語、印尼語、馬來語、尼泊爾語、波蘭語、羅馬尼亞語、塞爾維亞語、瑞典語、土耳其語、烏克蘭語、越南語、印地語、孟加拉語。
• 許可證：Apache-2.0。
• 標籤：涵蓋所有用例、創意寫作、各種體裁、工具調用、問題解決、深度思考、推理等。
• 基礎模型：DavidAU/Llama3.1-MOE-4X8B-Gated-IQ-Multi-Tier-Deep-Reasoning-32B。
• 管道標籤：文本生成。

✨ 主要特性

• 可變控制推理：“Gated - IQ - Multi - Tier - Deep - Reasoning”是一種可變控制推理模型，可在所有溫度/設置下運行，適用於所有用例。
• 獨特內部結構：在“推理”階段，所有4個模型均可運行，推理模型在過程中起主導作用，輸出生成時會進行切換。
• 直接控制模型：用戶可通過提示、名稱和關鍵字直接控制一個或多個模型。
• 推理速度和質量提升：核心推理模型結合3個非推理核心模型，推理速度和質量最多可提升300%。
• 支持工具調用：由於嵌入了Meta Llama 3.1 Instruct，支持工具調用和使用。
• 模型切換和構建：獨特的超級結構允許切換“推理模型”和支持/輸出生成模型，還可構建更大、更強大的模型。

📦 安裝指南

文檔未提及具體安裝步驟，可參考相關模型使用說明進行安裝。

💻 使用示例

基礎用法

以下是解決謎題的示例：

Answer the following riddle:

Six brothers were spending their time together.

The first brother was reading a book alone.
The second brother was playing tennis against another brother
The third brother was solving a crossword alone.
The fourth brother was watering the lawn alone.
The fifth brother was drawing a picture alone.
Question: what was the sixth brother doing?  

模型推理過程：

Thought for 17.57 seconds

Okay, let's break this down. We have six brothers, and each of them is doing something different. The first brother is reading a book alone, the second is playing tennis against another brother, the third is solving a crossword alone, the fourth is watering the lawn alone, and the fifth is drawing a picture alone.

So, the common theme here is that each of the first five brothers is doing something by themselves, and there's one brother who is not doing something by themselves. The second brother is playing tennis against another brother, which means there are at least two brothers involved in that activity.

Now, let's think about what the sixth brother could be doing. If we look at the activities of the first five brothers, they're all solo activities, like reading, solving crosswords, watering the lawn, and drawing a picture. There's no indication that any of these activities require more than one person.

But the second brother is playing tennis against another brother, which means there are at least two people involved in that activity. If we assume that the sixth brother is not doing a solo activity, then it's likely that he's involved in the activity that requires two people.

So, the sixth brother must be the one who is playing tennis with the second brother. This fits the pattern of the riddle, where each brother is doing something different, and the sixth brother is not doing a solo activity.

輸出結果：

The sixth brother was playing tennis with the second brother.

高級用法

以下是創意寫作的示例：

Using the following "story idea" below, write the first scene in the novel introducing the young woman. This scene should start in the middle of the action, include dialog, vivid passages, and end on a cliffhanger relevant to the story idea but it should also be unexpected. The scene should be 1000 words long and escalate in conflict and suspense and be written in first person, present tense with the point of view character being the young woman. The pov character will CURSE AND SWEAR, and generally have
a "filthy" mouth.

Story idea:
In a world ruled by dictatorship, a rebel young woman leads a rebellion against the system. Despite the risks, she fights to overthrow the dictator and restore democracy to her country. The government executes her for treason, but she sticks to her beliefs and is responsible for starting the revolution.

模型推理過程：

Thought for 25.07 seconds

Okay, let's break this down. We have a scene that's supposed to be intense, in the middle of action, and first person, present tense. The POV character is a rebel young woman fighting against a dictatorship. I need to make sure the language reflects her personality, as well as the tension of the situation.

First, I'll start with a situation where our character is in a tense, high - stakes moment. Perhaps she's being chased by government agents. She needs to use all her wits to stay alive. The scene will also be her introduction, so I should make sure the reader knows who she is and what she's fighting for.

Now, let's think about the language. It's not enough just to have her use swear words and be tough. I need to make her voice sound authentic, like someone who has lived through harsh conditions and fought for what she believes in. Her dialogue will be fast - paced, and she'll use phrases that show her resourcefulness and determination.

As the scene progresses, the danger will escalate. Maybe she'll find a way out, or maybe she won't. The ending should leave the reader on the edge of their seat, eager to find out what happens next. The twist has to be relevant to the story idea, but unexpected enough to be shocking. Perhaps our character will do something that surprises even herself, or there will be an element of the unexpected that propels the scene forward.

I'll also need to describe the setting, but not too much. We don't want the reader to get distracted by too much description. The focus will be on the action, our character's thoughts and emotions, and the dialogue.

One last thing to consider is the tone of the scene. It will be intense and high - stakes, but also show a hint of our character's resilience. We should leave room for character development in the scenes that come after this one.

I think I'm ready to start writing. Let's do it.

輸出結果：

"Shit. They've found us." I grit my teeth as I hear the thundering of boots behind me, the sound echoing off the concrete walls of the city's underpass. My heart pounds in my chest, and I know I have to keep moving if I want to stay alive.

"Come on, Naya!" my friend Kael yells from up ahead. "You can do this!"

I don't bother replying. I just keep running, my sneakers slapping against the damp ground as I round a corner. The sound of my pursuers is getting closer, and I can hear their muffled voices, the clanging of their rifles against the concrete walls.

"Shit," I curse again, ducking into a narrow alleyway. My breath is coming in ragged gasps as I lean against the cold brick, trying to catch my breath. My eyes dart around, searching for a way out or a place to hide. But there's nothing here but trash and the smell of decay.

I hear Kael's voice again, this time calling out, "Naya, where are you?"

I don't answer. I can't risk revealing my position. I have to stay quiet and keep moving if I want to stay alive.

I push away from the wall and keep running, my heart pounding in my chest like a drum. The alleyway opens up into a main street, and I'm hit by a blast of bright sunlight that makes me squint. The smell of cooking food and fresh air is a welcome respite from the stench of the underpass.

I weave through the crowded streets, dodging between pedestrians and market stalls. People look at me warily, but no one stops me. They know not to get involved in our business, the business of revolutionaries.

As I round another corner, I see Kael, his rifle at the ready, looking like he's about to take on the world. I grin at him, feeling a rush of excitement. This is what we've been fighting for: freedom, a chance to make things right.

But my moment of elation is cut short by the sound of a shot, and suddenly Kael is screaming. He spins around, clutching his shoulder, his face twisted in pain.

"Oh my god, Kael!" I yell, dropping to my knees beside him. "Kael, hang on!"

I glance around wildly, looking for any sign of the shooter. But it's too late; they're gone, melting into the crowd. I look back at Kael, and my heart breaks at the sight of his bloody shoulder.

"Naya," he whispers, his voice barely above a whisper. "Get out of here. Don't stop until you get to the safe house."

I shake my head, determination setting in. "No, we stick together. We'll make it out of this, just like we always do."

Kael tries to smile, but his face twists in pain. "I can't move," he gasps, his eyes already growing glassy.

I know what I have to do. I grit my teeth, my hands shaking as I grab Kael's rifle. "I'll carry you," I say, trying to sound strong.

But as I look up, I see a group of government agents storming down the street, their faces twisted with cruelty. And they're not alone; they have a prisoner in tow, a woman with a scarred face and a look of utter terror in her eyes.

I freeze, my blood running cold. The agents stop in front of Kael and me, their leader a towering figure with a cruel smile on his face.

"Well, well," he says, his eyes lingering on Kael. "What do we have here? A wounded bird and its little nestling?"

I clench my fists, anger building inside me. "Get away from him," I growl.

The agent chuckles. "Or what? You'll do something about it?"

He knows the answer; I'm out of options. My hands tremble with rage, but all I can think about is getting Kael out of here.

Suddenly, something clicks in my mind. This is what we've been working for: our moment, the chance to strike back at the system that's been crushing us for so long. The dictator, the regime; they all think they're invincible, but they're not. We have the power to bring them down.

I raise Kael's rifle, aiming at the agent. My hand shakes, but I don't look away.

The agent's smile grows. "You think a little rebellion is going to change anything? You're nothing but ants scurrying around, trying to make a difference."

But I know that's not true. We're more than that. We're a revolution.

And right now, it's time to make our move.

📚 詳細文檔

重要操作說明

• 溫度偏好：該模型喜歡較高的溫度，如1.2+、2.2+等，較低溫度可能無法產生最佳內容。
• 提示長度：作為指令模型，使用中長提示效果最佳。
• 推理開關：通過系統提示控制推理的開啟和關閉。
• 上下文要求：建議最小上下文為4k，8k更佳。
• 量化選擇：較大的量化意味著更好、更強的推理能力。

已知問題

• 有時需要重新生成以激活“思考/推理”或獲得良好的“思考塊”。
• 有時第二次或第三次生成的結果最佳，對於特定創意用途，建議至少生成5次。
• 有時“思考塊”結束後，需要手動提示模型生成輸出。

生成高質量輸出的方法

• 多輪改進：通過多次提示逐步完善內容，如先給出初稿，再進行評估和改進。
• 使用特定標籤和名稱：通過特定的標籤和名稱直接訪問一個或多個模型，控制推理和輸出。

系統角色/系統提示

系統提示用於控制模型的推理、指令遵循和輸出生成，不同的系統提示可實現不同的功能，如基本推理、多階段推理、創意推理等。使用時需保留格式，部分提示中的名稱為模型內部引用，請勿更改。

額外支持文檔

• 如何使用推理/思考模型並創建它們：詳細介紹如何使用推理/思考模型並獲得最大性能，包括所有推理/思考模型的鏈接以及將任何“常規”模型轉換為“推理/思考”模型的適配器。
• 通過採樣器和參數最大化模型性能：詳細介紹所有參數、設置、採樣器和高級採樣器，以充分發揮模型的最大潛力。

軟件補丁

適用於Silly Tavern的軟件補丁：用於控制和改善任何AI模型的輸出生成，尤其適用於一些“創意”模型，減少參數/採樣器調整。

🔧 技術細節

該模型基於Llama 3.1架構，採用MOE（Mixture of Experts）配置，由四個8B模型（1個推理模型和3個非推理模型）組成，實際具有32B參數，但“權重”為25B。推理技術源自NousResearch的原始Llama 3.1 “DeepHermes”模型，該版本保留了原模型約100%的推理能力，並通過3個核心模型的協助，總推理能力比原DeepHermes 8B模型最多強300%。

📄 許可證

本模型使用Apache - 2.0許可證。