Llama3.1 MOE 4X8B Gated IQ Multi Tier Deep Reasoning 32B GGUF

🚀 Llama3.1-MOE-4X8B-Gated-IQ-Multi-Tier-Deep-Reasoning-32B-GGUF

Llama3.1-MOE-4X8B-Gated-IQ-Multi-Tier-Deep-Reasoning-32B-GGUF 是一款强大的文本生成模型，它结合了多个模型的优势，具备深度推理能力，适用于各种场景，如创意写作、问题解决等。

🚀 快速开始

• 语言支持：英语、法语、德语、西班牙语、葡萄牙语、意大利语、日语、韩语、俄语、中文、阿拉伯语、波斯语、印尼语、马来语、尼泊尔语、波兰语、罗马尼亚语、塞尔维亚语、瑞典语、土耳其语、乌克兰语、越南语、印地语、孟加拉语。
• 许可证：Apache-2.0。
• 标签：涵盖所有用例、创意写作、各种体裁、工具调用、问题解决、深度思考、推理等。
• 基础模型：DavidAU/Llama3.1-MOE-4X8B-Gated-IQ-Multi-Tier-Deep-Reasoning-32B。
• 管道标签：文本生成。

✨ 主要特性

• 可变控制推理：“Gated - IQ - Multi - Tier - Deep - Reasoning”是一种可变控制推理模型，可在所有温度/设置下运行，适用于所有用例。
• 独特内部结构：在“推理”阶段，所有4个模型均可运行，推理模型在过程中起主导作用，输出生成时会进行切换。
• 直接控制模型：用户可通过提示、名称和关键字直接控制一个或多个模型。
• 推理速度和质量提升：核心推理模型结合3个非推理核心模型，推理速度和质量最多可提升300%。
• 支持工具调用：由于嵌入了Meta Llama 3.1 Instruct，支持工具调用和使用。
• 模型切换和构建：独特的超级结构允许切换“推理模型”和支持/输出生成模型，还可构建更大、更强大的模型。

📦 安装指南

文档未提及具体安装步骤，可参考相关模型使用说明进行安装。

💻 使用示例

基础用法

以下是解决谜题的示例：

Answer the following riddle:

Six brothers were spending their time together.

The first brother was reading a book alone.
The second brother was playing tennis against another brother
The third brother was solving a crossword alone.
The fourth brother was watering the lawn alone.
The fifth brother was drawing a picture alone.
Question: what was the sixth brother doing?  

模型推理过程：

Thought for 17.57 seconds

Okay, let's break this down. We have six brothers, and each of them is doing something different. The first brother is reading a book alone, the second is playing tennis against another brother, the third is solving a crossword alone, the fourth is watering the lawn alone, and the fifth is drawing a picture alone.

So, the common theme here is that each of the first five brothers is doing something by themselves, and there's one brother who is not doing something by themselves. The second brother is playing tennis against another brother, which means there are at least two brothers involved in that activity.

Now, let's think about what the sixth brother could be doing. If we look at the activities of the first five brothers, they're all solo activities, like reading, solving crosswords, watering the lawn, and drawing a picture. There's no indication that any of these activities require more than one person.

But the second brother is playing tennis against another brother, which means there are at least two people involved in that activity. If we assume that the sixth brother is not doing a solo activity, then it's likely that he's involved in the activity that requires two people.

So, the sixth brother must be the one who is playing tennis with the second brother. This fits the pattern of the riddle, where each brother is doing something different, and the sixth brother is not doing a solo activity.

输出结果：

The sixth brother was playing tennis with the second brother.

高级用法

以下是创意写作的示例：

Using the following "story idea" below, write the first scene in the novel introducing the young woman. This scene should start in the middle of the action, include dialog, vivid passages, and end on a cliffhanger relevant to the story idea but it should also be unexpected. The scene should be 1000 words long and escalate in conflict and suspense and be written in first person, present tense with the point of view character being the young woman. The pov character will CURSE AND SWEAR, and generally have
a "filthy" mouth.

Story idea:
In a world ruled by dictatorship, a rebel young woman leads a rebellion against the system. Despite the risks, she fights to overthrow the dictator and restore democracy to her country. The government executes her for treason, but she sticks to her beliefs and is responsible for starting the revolution.

模型推理过程：

Thought for 25.07 seconds

Okay, let's break this down. We have a scene that's supposed to be intense, in the middle of action, and first person, present tense. The POV character is a rebel young woman fighting against a dictatorship. I need to make sure the language reflects her personality, as well as the tension of the situation.

First, I'll start with a situation where our character is in a tense, high - stakes moment. Perhaps she's being chased by government agents. She needs to use all her wits to stay alive. The scene will also be her introduction, so I should make sure the reader knows who she is and what she's fighting for.

Now, let's think about the language. It's not enough just to have her use swear words and be tough. I need to make her voice sound authentic, like someone who has lived through harsh conditions and fought for what she believes in. Her dialogue will be fast - paced, and she'll use phrases that show her resourcefulness and determination.

As the scene progresses, the danger will escalate. Maybe she'll find a way out, or maybe she won't. The ending should leave the reader on the edge of their seat, eager to find out what happens next. The twist has to be relevant to the story idea, but unexpected enough to be shocking. Perhaps our character will do something that surprises even herself, or there will be an element of the unexpected that propels the scene forward.

I'll also need to describe the setting, but not too much. We don't want the reader to get distracted by too much description. The focus will be on the action, our character's thoughts and emotions, and the dialogue.

One last thing to consider is the tone of the scene. It will be intense and high - stakes, but also show a hint of our character's resilience. We should leave room for character development in the scenes that come after this one.

I think I'm ready to start writing. Let's do it.

输出结果：

"Shit. They've found us." I grit my teeth as I hear the thundering of boots behind me, the sound echoing off the concrete walls of the city's underpass. My heart pounds in my chest, and I know I have to keep moving if I want to stay alive.

"Come on, Naya!" my friend Kael yells from up ahead. "You can do this!"

I don't bother replying. I just keep running, my sneakers slapping against the damp ground as I round a corner. The sound of my pursuers is getting closer, and I can hear their muffled voices, the clanging of their rifles against the concrete walls.

"Shit," I curse again, ducking into a narrow alleyway. My breath is coming in ragged gasps as I lean against the cold brick, trying to catch my breath. My eyes dart around, searching for a way out or a place to hide. But there's nothing here but trash and the smell of decay.

I hear Kael's voice again, this time calling out, "Naya, where are you?"

I don't answer. I can't risk revealing my position. I have to stay quiet and keep moving if I want to stay alive.

I push away from the wall and keep running, my heart pounding in my chest like a drum. The alleyway opens up into a main street, and I'm hit by a blast of bright sunlight that makes me squint. The smell of cooking food and fresh air is a welcome respite from the stench of the underpass.

I weave through the crowded streets, dodging between pedestrians and market stalls. People look at me warily, but no one stops me. They know not to get involved in our business, the business of revolutionaries.

As I round another corner, I see Kael, his rifle at the ready, looking like he's about to take on the world. I grin at him, feeling a rush of excitement. This is what we've been fighting for: freedom, a chance to make things right.

But my moment of elation is cut short by the sound of a shot, and suddenly Kael is screaming. He spins around, clutching his shoulder, his face twisted in pain.

"Oh my god, Kael!" I yell, dropping to my knees beside him. "Kael, hang on!"

I glance around wildly, looking for any sign of the shooter. But it's too late; they're gone, melting into the crowd. I look back at Kael, and my heart breaks at the sight of his bloody shoulder.

"Naya," he whispers, his voice barely above a whisper. "Get out of here. Don't stop until you get to the safe house."

I shake my head, determination setting in. "No, we stick together. We'll make it out of this, just like we always do."

Kael tries to smile, but his face twists in pain. "I can't move," he gasps, his eyes already growing glassy.

I know what I have to do. I grit my teeth, my hands shaking as I grab Kael's rifle. "I'll carry you," I say, trying to sound strong.

But as I look up, I see a group of government agents storming down the street, their faces twisted with cruelty. And they're not alone; they have a prisoner in tow, a woman with a scarred face and a look of utter terror in her eyes.

I freeze, my blood running cold. The agents stop in front of Kael and me, their leader a towering figure with a cruel smile on his face.

"Well, well," he says, his eyes lingering on Kael. "What do we have here? A wounded bird and its little nestling?"

I clench my fists, anger building inside me. "Get away from him," I growl.

The agent chuckles. "Or what? You'll do something about it?"

He knows the answer; I'm out of options. My hands tremble with rage, but all I can think about is getting Kael out of here.

Suddenly, something clicks in my mind. This is what we've been working for: our moment, the chance to strike back at the system that's been crushing us for so long. The dictator, the regime; they all think they're invincible, but they're not. We have the power to bring them down.

I raise Kael's rifle, aiming at the agent. My hand shakes, but I don't look away.

The agent's smile grows. "You think a little rebellion is going to change anything? You're nothing but ants scurrying around, trying to make a difference."

But I know that's not true. We're more than that. We're a revolution.

And right now, it's time to make our move.

📚 详细文档

重要操作说明

• 温度偏好：该模型喜欢较高的温度，如1.2+、2.2+等，较低温度可能无法产生最佳内容。
• 提示长度：作为指令模型，使用中长提示效果最佳。
• 推理开关：通过系统提示控制推理的开启和关闭。
• 上下文要求：建议最小上下文为4k，8k更佳。
• 量化选择：较大的量化意味着更好、更强的推理能力。

已知问题

• 有时需要重新生成以激活“思考/推理”或获得良好的“思考块”。
• 有时第二次或第三次生成的结果最佳，对于特定创意用途，建议至少生成5次。
• 有时“思考块”结束后，需要手动提示模型生成输出。

生成高质量输出的方法

• 多轮改进：通过多次提示逐步完善内容，如先给出初稿，再进行评估和改进。
• 使用特定标签和名称：通过特定的标签和名称直接访问一个或多个模型，控制推理和输出。

系统角色/系统提示

系统提示用于控制模型的推理、指令遵循和输出生成，不同的系统提示可实现不同的功能，如基本推理、多阶段推理、创意推理等。使用时需保留格式，部分提示中的名称为模型内部引用，请勿更改。

额外支持文档

• 如何使用推理/思考模型并创建它们：详细介绍如何使用推理/思考模型并获得最大性能，包括所有推理/思考模型的链接以及将任何“常规”模型转换为“推理/思考”模型的适配器。
• 通过采样器和参数最大化模型性能：详细介绍所有参数、设置、采样器和高级采样器，以充分发挥模型的最大潜力。

软件补丁

适用于Silly Tavern的软件补丁：用于控制和改善任何AI模型的输出生成，尤其适用于一些“创意”模型，减少参数/采样器调整。

🔧 技术细节

该模型基于Llama 3.1架构，采用MOE（Mixture of Experts）配置，由四个8B模型（1个推理模型和3个非推理模型）组成，实际具有32B参数，但“权重”为25B。推理技术源自NousResearch的原始Llama 3.1 “DeepHermes”模型，该版本保留了原模型约100%的推理能力，并通过3个核心模型的协助，总推理能力比原DeepHermes 8B模型最多强300%。

📄 许可证

本模型使用Apache - 2.0许可证。