Nemotron H 8B Base 8K

🚀 Nemotron-H-8B-Base-8K

NVIDIA Nemotron-H-8B-Base-8K is a large language model designed for text completion. It uses a hybrid architecture and supports multiple languages with a context length of 8K.

🚀 Quick Start

To use the Nemotron-H-8B-Base-8K model, you can follow this simple Python example:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the tokenizer and model
tokenizer  = AutoTokenizer.from_pretrained("nvidia/Nemotron-H-8B-Base-8K", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("nvidia/Nemotron-H-8B-Base-8K", torch_dtype=torch.bfloat16, trust_remote_code=True).cuda()

prompt = "When was NVIDIA founded?"

outputs = model.generate(**tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device))
print(tokenizer.decode(outputs[0]))

✨ Features

• Hybrid Architecture: Utilizes a hybrid model architecture mainly composed of Mamba-2 and MLP layers, combined with just four Attention layers.
• Multi - Language Support: Supports multiple languages including English, German, Spanish, French, Italian, Korean, Portuguese, Russian, Japanese, and Chinese.
• 8K Context Length: Capable of handling text sequences with a context length of up to 8K.
• Customization: Can be customized using the NeMo Framework suite of tools, such as Parameter - Efficient Fine - Tuning and Model Alignment.

📚 Documentation

Model Overview

NVIDIA Nemotron-H-8B-Base-8K is a large language model (LLM) developed by NVIDIA for text completion. For more detailed information on the model architecture, training, and evaluation, please see the project page and the technical report.

License/Terms of Use

Use of this model is governed by the NVIDIA Internal Scientific Research and Development Model License.

Use Case

This model is intended for developers and researchers building LLMs.

Release Date

4/14/2025

References

• [2504.03624] Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models

Model Architecture

Input

Output

Software Integration

Model Version

• v1.0

Prompt Format

As this is a base model, no explicit prompt format is recommended or required.

Training, Testing, and Evaluation Datasets

Training & Testing Datasets

The training corpus consists of English and multilingual text, as well as code. It covers various document types and domains. Data collection and labeling are hybrid (Automated, Human, Synthetic).

Evaluation Datasets

We used multiple datasets to evaluate the model, including those for commonsense understanding, coding, math, and general knowledge.

Commonsense Understanding Evaluations

• ARC (Ai2 reasoning challenge) - Challenge: Dataset
• Hellaswag: Dataset
• Winogrande: Dataset
• CommonsenseQA: Dataset

Coding Evaluations

• MBPP (Mostly Basic Python Programming Problems): Dataset
• MBPP+: Dataset
• HumanEval: Dataset

Math Evaluations

• GSM8K (Grade School Math 8K): Dataset
• MATH: Dataset
• MATH Lvl 5: Dataset
• MATH - 500: Dataset

General Evaluations

• MMLU Pro: Dataset
• MMLU: Dataset

Potential Known Risks for Usage

The model was trained on data with toxic language and societal biases. It may amplify these biases and return toxic responses, especially with toxic prompts. It may also generate inaccurate, incomplete, or irrelevant text.

Inference

Ethical Considerations

NVIDIA believes in Trustworthy AI. For more detailed information on ethical considerations, please see the Responsible Use Guide at http://nvidia.com/nemotron - responsible - use. Report security vulnerabilities or NVIDIA AI Concerns here.

📄 License

This model is licensed under the NVIDIA Internal Scientific Research and Development Model License.