Llama 3.1 Nemotron Nano 8B V1 GGUF

base_model: nvidia/Llama-3.1-Nemotron-Nano-8B-v1 language:

• en library_name: transformers license: other license_name: nvidia-open-model-license license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/ pipeline_tag: text-generation tags:
• nvidia
• llama-3
• pytorch
• TensorBlock
• GGUF

nvidia/Llama-3.1-Nemotron-Nano-8B-v1 - GGUF

This repo contains GGUF format model files for nvidia/Llama-3.1-Nemotron-Nano-8B-v1.

The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4882.

Our projects

Awesome MCP Servers	TensorBlock Studio
A comprehensive collection of Model Context Protocol (MCP) servers.	A lightweight, open, and extensible multi-LLM interaction studio.
👀 See what we built 👀	👀 See what we built 👀

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>

{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Model file specification

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, downoad the individual model file the a local directory

huggingface-cli download tensorblock/Llama-3.1-Nemotron-Nano-8B-v1-GGUF --include "Llama-3.1-Nemotron-Nano-8B-v1-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/Llama-3.1-Nemotron-Nano-8B-v1-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'