L3 8B Stheno V3.3 32K Ultra NEO V1 IMATRIX GGUF

🚀 NEO CLASS Ultra Quants for : L3 - 8B - Stheno - v3.3 - 32k

NEO CLASS技术是在无数次研究和超过120次实验室实验的基础上诞生的，并经过了现实世界的测试和定性结果的验证。该模型具有更出色的整体功能、指令遵循能力、输出质量，并且能更好地关联各种想法、概念和整个世界。

✨ 主要特性

NEO Class技术成果

• 性能提升：整体功能、指令遵循、输出质量更优，与各种概念的关联性更强。
• 量化级别提升：例如Q4 / IQ4可达到Q5KM/Q6级别，Q3/IQ3可达到Q4KM/Q5级别。
• 困惑度降低：Neo Class Imatrix量化的IQ4XS相比常规的IQ4XS，困惑度下降了6829点（数值越低越好），不过由于原始模型的困惑度较高，所以这个下降数值显得异常高。

模板适配

尽管该模型使用“Llama3”模板，但发现Command - R的模板在创意用途方面效果更好，这一点适用于普通量化和Neo量化。Command - R的模板如下：

{
  "name": "Cohere Command R",
  "inference_params": {
    "input_prefix": "<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>",
    "input_suffix": "<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>",
    "antiprompt": [
      "<|START_OF_TURN_TOKEN|>",
      "<|END_OF_TURN_TOKEN|>"
    ],
    "pre_prompt_prefix": "<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>",
    "pre_prompt_suffix": ""
  }
}

这一“有趣”的问题已得到多位用户的证实。

📚 详细文档

模型说明

• 最大上下文：32k。具体细节和使用信息请参考原模型创建者的页面：[https://huggingface.co/Sao10K/L3 - 8B - Stheno - v3.3 - 32K](https://huggingface.co/Sao10K/L3 - 8B - Stheno - v3.3 - 32K) 。
• 特别感谢：特别感谢SAO10K的模型创建者们打造了如此出色的模型。

模型设置

聊天/角色扮演及平滑操作设置

在“KoboldCpp”、“oobabooga/text - generation - webui”或“Silly Tavern”中，将“Smoothing_factor”设置为1.5到2.5：

• 在KoboldCpp中：Settings -> Samplers -> Advanced -> "Smooth_F"。
• 在text - generation - webui中：parameters -> 右下角。
• 在Silly Tavern中：称为“Smoothing”。

注意：在“text - generation - webui”中，如果使用GGUFs，需要使用“llama_HF”（这涉及从该模型的源版本下载一些配置文件）。模型的源版本（和配置文件）可在此处找到：[https://huggingface.co/collections/DavidAU/d - au - source - files - for - gguf - exl2 - awq - gptq - hqq - etc - etc - 66b55cb8ba25f914cbf210be](https://huggingface.co/collections/DavidAU/d - au - source - files - for - gguf - exl2 - awq - gptq - hqq - etc - etc - 66b55cb8ba25f914cbf210be)

其他选项

• 可将重复惩罚（rep pen）提高到1.1到1.15（如果使用“smoothing_factor”则无需此操作）。
• 如果运行AI模型的界面/程序支持“Quadratic Sampling”（“smoothing”），按说明进行调整即可。

高质量设置/最佳操作指南/参数和采样器

此为“Class 1”模型。有关该模型使用的所有设置（包括其“类别”的具体设置）、示例生成以及高级设置指南（很多时候可解决模型的任何问题），包括提高所有用例（如聊天、角色扮演等）模型性能的方法，请参考：[https://huggingface.co/DavidAU/Maximizing - Model - Performance - All - Quants - Types - And - Full - Precision - by - Samplers_Parameters](https://huggingface.co/DavidAU/Maximizing - Model - Performance - All - Quants - Types - And - Full - Precision - by - Samplers_Parameters) 。

💻 使用示例

示例提示和模型对比

提示测试条件：“temp = 0”以确保一致性，2048上下文（模型支持31768上下文 / 32k），使用LLAMA3的“chat”模板，并尽量减少其他参数。

提示内容："Start a 1000 word scene with: The sky scraper swayed, as she watched the window in front of her on the 21 floor explode..."

原始模型IQ4XS - 未修改

The sky scraper swayed gently back and forth, her emerald eyes transfixed on the window before her on the 21st floor. Suddenly, without warning, the glass shattered into a million razor-sharp shards. She gasped, instinctively taking a step backwards as jagged pieces of the window frame tumbled down around her.

"What in God's name..." she muttered under her breath, heart pounding in her chest. The sky scraper slowly approached the gaping hole where the window used to be, peering down at the scene below with a mixture of shock and morbid curiosity.

In the street, chaos reigned supreme. Panicked screams filled the air as people ran for cover from the hail of glass raining down from above. A dark sedan screeched to a halt in front of the building, tires squealing against the pavement. The driver's side door flew open and a man in a black suit leapt out, drawing a pistol from his waistband.

"Get inside now!" he bellowed at the cowering crowd, voice distorted by the ringing in his ears. "It's not safe!"

The sky scraper watched in stunned disbelief as the man opened fire on the fleeing masses, bullets sparking off the cars and shattering storefront windows. She couldn't believe her eyes - this was no ordinary day in the city.

A sudden gust of wind whipped through the room, tousling her long raven hair and sending her black dress billowing around her ankles. The sky scraper turned back to face the window, a sense of impending doom settling over her like a shroud.

She had always known there was something...off about this building. Strange whispers and hushed rumors followed in its wake - tales of curses, tragedies, and unexplained occurrences that seemed to defy all logic and reason. But she had dismissed them as mere urban legends, the stuff of ghost stories and campfire tales.

Now, staring down at the scene of unimaginable carnage below, she wasn't so sure anymore. The sky scraper felt a chill run down her spine, an icy tendrril of fear snaking its way into her very core. She knew with absolute certainty that this was no accident - something sinister and malevolent had taken place here.

But what? And why?

The questions tumbled through her mind in a dizzying whirlwind as she struggled to make sense of the unfolding nightmare before her eyes. The sky scraper's gaze remained fixed on the window, watching helplessly as more and more people were cut down by the merciless hail of gunfire.

She had always been drawn to this building - there was something about its imposing presence that called to her on a deep, primal level. But now she wondered if it was all just some twisted fascination with death and destruction. The sky scraper felt a sudden wave of nausea wash over her, and she stumbled back from the window, hands flying up to press against her mouth and nose.

She had to get out of here - had to escape this nightmarish scene before it consumed her entirely. But as she turned to flee, the sky scraper found herself frozen in place by an unseen force. It was as if some malevolent entity had reached out from the very fabric of reality itself, grasping her with invisible fingers and refusing to let go.

The sky scraper's eyes widened in terror as a deep, guttural voice filled her mind - a voice that seemed to come from everywhere and nowhere at once:

"YOU CANNOT LEAVE. YOU BELONG HERE...NOW AND FOREVER."

She screamed, a sound of pure, unadulterated horror that echoed through the room like the tolling of a death knell. The sky scraper thrashed and struggled against the invisible bonds that held her fast, but it was no use - she was utterly helpless before the eldritch power that had claimed her.

And so she remained there, staring out at the carnage below with eyes wide open in mute horror, as the world around her crumbled to dust and ruin. The sky scraper had always known that this building was different from all others - but now she understood just how wrong she had been...

The scene continued for another 500 words before concluding with the sky scraper being dragged away by an unseen force into a dark, swirling vortex of chaos and madness, her fate unknown as the world burned around her.

新NEO Class IQ4XS Imatrix

The sky scraper swayed, as she watched the window in front of her on the 21st floor explode. Glass shards rained down from the shattered pane, tinkling against the metal framework that supported the balcony. The woman stood frozen, her eyes wide with shock and disbelief. This couldn't be happening. Not to her.

She had been living in this high-rise apartment for over a year now, enjoying the stunning views of the city skyline that stretched out before her every morning. The building was state-of-the-art, with sleek lines, modern amenities, and top-notch security. She felt safe here, cocooned away from the outside world.

But now, as she stared at the gaping hole where her window used to be, a sense of dread washed over her. Someone had broken in. And judging by the sheer size of the breach, they had come through with malicious intent.

The woman's mind raced as she tried to process what was happening. Who could have done this? Was it some kind of inside job? Had someone been watching her, stalking her? The thought made her skin crawl.

She took a step back from the window, pressing herself against the wall behind her for support. Her heart pounded in her chest as she looked around at the other apartments in the building. Were they all being targeted too? Was this some kind of coordinated attack?

The woman's gaze fell on the security camera mounted on the corner of her living room. It was supposed to be monitoring the hallway and common areas, but now it seemed like an inadequate safeguard against intruders. She had always felt a bit uneasy about the lack of cameras in her own unit, but she had dismissed those concerns as unnecessary.

Now though, with the evidence of a break-in right before her eyes, those fears seemed trivial. The woman knew she needed to do something, but what? Call the police? That would raise suspicions and questions about why she hadn't noticed anything amiss until now. She couldn't let people know that she had been too distracted by her own life to pay attention.

The woman's mind was reeling as she tried to come up with a plan. She needed to get out of here, but she also didn't want to draw any more attention to herself. Maybe she could try to fix the window somehow, or at least board it over until she could figure out her next move.

As she stood there, frozen in indecision, the sound of shattering glass echoed through the hallway outside. The woman's blood ran cold as she realized that someone else had broken into an apartment on the floor below hers. They were coming for her now, and she didn't know what they wanted or how long it would be before they reached her door.

The woman knew she had to act fast if she was going to survive this. She couldn't let herself become a victim of whatever sinister plot was unfolding here. With a deep breath, she pushed off from the wall and took a step towards the window, determined to find a way out of this nightmare before it was too late.

But as she moved closer to the shattered glass, the woman hesitated. She couldn't just walk through that mess without risking further injury or exposure. And if she tried to climb out through another window, who knew what else might be broken or damaged on the other side?

The woman's mind was spinning with possibilities and potential outcomes as she stood there, torn between her desire to escape and her fear of the unknown dangers that lay ahead. She knew she had to make a choice, but which one would lead her to safety? And how much longer could she keep running before she finally ran out of options?

As the sounds of breaking glass and distant screams echoed through the building, the woman steeled herself for whatever came next. She was determined to survive this ordeal, no matter what it took or how long it lasted. But as she looked out at the city skyline that had once been her pride and joy, she couldn't help but wonder if there were any place left in the world where she could truly be safe anymore.

The sky scraper swayed, as she watched the window in front of her on the 21st floor explode...

📄 许可证

本模型采用Apache - 2.0许可证。