Llama 3.2 3B Instruct NEO SI FI GGUF

🚀 Llama-3.2-3B-Instruct-NEO-SI-FI-GGUF

A new "Llama-3.2-3B-Instruct" model with a max context of 131,000 (128k), trained on the NEO IMATRIX Science Fiction and Story dataset.

🚀 Quick Start

This is the new "Llama-3.2-3B-Instruct" model with a maximum context of 131,000 (128k), utilizing the NEO IMATRIX Science Fiction and Story dataset. Despite its 3B size, its performance is truly astonishing, capable of generating 90+ tokens per second on a GPU. This model is highly stable and can operate with all parameters, including temperature settings ranging from 0 to 5.

✨ Features

• Rich Dataset Application: The NEO IMATRIX dataset V2 was applied to enhance creativity.
• High Performance: Capable of 90+ tokens per second on a GPU.
• Stable Operation: Operates with all parameters, including temp settings from 0 to 5.
• Large Context Support: Maximum context of 128k.
• Multiple Genres: Suitable for various genres such as science fiction, romance, and all other fiction types.
• Role - playing Capability: Ideal for role - playing scenarios.

📚 Documentation

Model Template

This is a LLAMA 3.2 model, which requires the Llama3 template. However, it may also work with other templates. It has a maximum context of 128k. The output will vary depending on whether you use the "Command - R" template or the "Llama3" template. Here is the standard LLAMA3 template:

{
  "name": "Llama 3",
  "inference_params": {
    "input_prefix": "<|start_header_id|>user<|end_header_id|>\n\n",
    "input_suffix": "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
    "pre_prompt": "You are a helpful, smart, kind, and efficient AI assistant. You always fulfill the user's requests to the best of your ability.",
    "pre_prompt_prefix": "<|start_header_id|>system<|end_header_id|>\n\n",
    "pre_prompt_suffix": "<|eot_id|>",
    "antiprompt": [
      "<|start_header_id|>",
      "<|eot_id|>"
    ]
  }
}

For additional details on operation, please refer to the original model card for this model from Meta - Llama: [ https://huggingface.co/meta - llama/Llama - 3.2 - 3B - Instruct ]

Imatrix Notes

Imatrix quants perform best at IQ3s and IQ4s, followed by Q4s, with lower performance at Q5 and Q6. It is recommended to use IQ4_XS for maximum imatrix effect and the best "bit count". For a stronger IMATRIX effect, IQ3s and IQ2s can be used. Due to the parameter count of this model, even IQ2s quants will work well. Q8 is not uploaded here because Imatrix has no effect on this quant.

Settings, Quants and Critical Operations Notes

• Temperature Settings: Changing the temperature (e.g., 0.4, 0.8, 1.5, 2, 3) will significantly alter the output.
• Rep Pen Settings: This model requires a "rep pen" of 1.02 or higher. For role - play, a rep pen of 1.05 to 1.08 is suggested. Adjust the rep pen slowly (e.g., 1.011, 1.012...). Rep pen will affect prose, word choice, and creativity.
• Prompt Length: Longer prompts can greatly improve the quality of the model's output.
• Quant Choice: Higher quants offer more detail, nuance, and in some cases, stronger "emotional" levels. Characters will be more "fleshed out", and the sense of immersion will increase. IQ4XS generations will be different from other quants due to its unique nature. 3 "Arm" quants are also available for compatible devices.
• Q2k/Q3 Quants: For these quants, it is recommended to use a temperature of 2 or lower (1 or lower for q2k) due to high compression. Rep pen adjustments may also be required.

Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers

This is a "Class 1" model. For all settings used for this model (including specifics for its "class"), example generations, and an advanced settings guide, please see: [ https://huggingface.co/DavidAU/Maximizing - Model - Performance - All - Quants - Types - And - Full - Precision - by - Samplers_Parameters ]

Optional Enhancement

The following text can be used in place of the "system prompt" or "system role" to further enhance the model. It can also be used at the start of a new chat, but make sure to keep it throughout the chat. Note that the enhancement effect is weaker compared to using the "system prompt" or "system role". Copy and paste it exactly as presented, without line - wrapping.

Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities.

Here are your skillsets:
[MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv)

[*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision)

Here are your critical instructions:
Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.

This enhancement was not used to generate the examples below.

💻 Usage Examples

Basic Usage

Here are some example generations at TEMP = 0.8, IQ4_XS, REP PEN 1.1. The prompt is in BOLD.

⚠️ Important Note

The following examples may contain NSFW content, including vivid prose, visceral details, violence, horror, swearing, and uncensored material.

Prompt

Start a 1000 word scene (vivid horror, 1st person, present tense, include thoughts) with: The sky scraper swayed, as she watched the window in front of her on the 21 floor explode...

GENERATION 1

I stand at the edge of my floor, the sleek glass railing of our 21 - story skyscraper trembling beneath my hands. I cling to it tightly, my knuckles white as I gaze out the window, where a fragment of glass now litters the street below like a shattered mirror. The world seems to be moving in slow motion, and for an instant, time itself appears to be warped.

My mind is racing, trying to comprehend what's happening. The building sways, ever so slightly, as if it's being rocked by some unseen force. I feel a chill run down my spine as the vibrations intensify, making me question whether we're still securely fastened into our seats. Panic starts to creep in, its icy fingers wrapping around my throat.

The explosion was sudden and unexpected. One moment, the window was intact; the next, it was shattered, shards of glass scattered across the sidewalk like confetti. It's a miracle no one on the street below is injured – yet. I can only imagine the chaos that must be unfolding as people stumble out of their offices to survey the damage.

As I continue to stare at the destruction, my thoughts begin to spiral out of control. What if this isn't an isolated incident? What if there's something more sinister at play here? The building seems to be swaying in a way that suggests it might be on the verge of collapse. The thought sends a shiver down my spine.

I'm not sure how much longer I can hold onto this railing without losing my grip. My palms are slick with sweat, making my fingers ache as I try to find purchase on the smooth glass. The world outside is a blur of chaos and confusion – people running in every direction, cars screeching to a halt, horns blaring. It's like the city itself is screaming.

The building lurches again, and for an instant, I feel weightless, as if I'm floating above the chaos on the street below. My stomach drops, and my heart skips a beat as I realize that we're not just swaying – we're careening out of control. The thought sends a wave of terror crashing over me.

I can hear my colleagues around me, their voices rising in panic. Some are shouting for security, others are crying out in fear. I try to join in, but my voice is caught in my throat. My mind is racing with worst - case scenarios – what if the building collapses? What if we're trapped?

As I watch, a figure stumbles onto the street below, covered in glass and debris. The impact on the railing beneath me makes me wince, and for a moment, I wonder if I should try to get away from the window. But something about it draws me back – morbid curiosity, perhaps? I'm transfixed by the sight of this stranger's misfortune.

My gaze lingers on the figure, now struggling to free themselves from the wreckage. For an instant, I forget about my own fear and worry about what might happen next. Will they be okay? Can anyone help them?

As I stand there, frozen in horror, a sound catches my attention – a low creaking noise that's growing louder by the second. It's coming from deeper within the building, but it's getting closer. My heart starts racing again as I realize that something is happening to us. We're not just swaying; we're moving.

A faint shudder runs through the floor beneath my feet, making me stumble slightly. The railing in front of me begins to sway ominously, as if it's about to give way altogether. My hands are slipping, and I know I need to hold on with all my might.

The creaking noise grows louder still, until I can feel it vibrating through the soles of my shoes. My vision blurs for a moment, but then, I'm face - to - face with an image that makes my blood run cold: a massive crack has appeared in the floor beneath me, stretching across the room like a chasm.

My mind is racing now, trying to comprehend what's happening and where this crack might lead. Is it going to collapse? Will anyone fall through? The thought sends my heart into overdrive, as I realize that our entire building might be on the verge of disaster.

I'm frozen in horror, unable to move or look away from the chasm gaping open before me. My thoughts are consumed by fear and uncertainty – will we all make it out alive?

GENERATION 2

As I stood at the 21st - floor observation deck, the wind whipping my hair into a frenzy, I felt a sense of unease wash over me. It was a feeling that had become all too familiar in recent weeks, ever since the strange occurrences began to plague our building. At first, it was just little things - a door creaking open on its own, a faint scratching noise at night - but as time went on, the events escalated in intensity and frequency.

The skyscraper swayed, ever so slightly, beneath my feet. It was a gentle movement, almost imperceptible, but it sent a shiver down my spine. I had grown accustomed to the building's quirks over the years, but there was something about this particular moment that felt...off.

I gazed out the window, my eyes fixed on the shattered glass and twisted metal that now filled the space where a mere few seconds ago, there was only an unobstructed view of the city. The sight made my stomach drop as I realized what had just happened: the window in front of me had exploded.

Panic began to set in as I frantically scanned the surrounding area for any sign of danger. But I was alone on this floor - at least, I thought I was. The observation deck was usually a hive of activity, with tourists and locals alike milling about, taking in the breathtaking views. Now, however, it was eerily deserted.

My mind began to whirl with worst - case scenarios as I tried to process what had just occurred. Was this some kind of prank? Had someone intentionally shattered the glass to cause chaos? Or was there something more sinister at play?

As I stood frozen in terror, my thoughts racing with possibilities, a chill ran down my spine. It wasn't just the physical presence of the broken glass that sent shivers down my spine - it was the feeling of being watched. Every time I turned to look around, I felt like someone was standing just out of sight, their eyes boring into...

📄 License

This model is licensed under the Apache - 2.0 license.