Saanvi-C0-12B Open-source Large Language Model - Efficient and Accurate Handling of Long Texts, Supports RAG Technology

Saanvi C0 12B

Developed by riple-saanvi-lab

A 12-billion-parameter large language model optimized for speed, efficiency, and contextual accuracy, supporting RAG-enhanced technology and a 128k context window.

Large Language Model

Transformers

Open Source License:Apache-2.0 #12B parameter large model #4-bit quantization efficient inference #RAG-enhanced retrieval

Downloads 170

Release Time : 4/25/2025

Model Overview

A next-generation 12-billion-parameter large language model featuring RAG-enhanced technology, 4-bit quantization, Flash Attention 2, and bfloat16 precision, optimized for inference capabilities, logical reasoning, language fluency, task completion, and mathematical abilities.

Model Features

Flash Attention 2

2.7x faster inference speed

4-bit quantization

Runs on GPUs with just 8GB VRAM

Instruction tuning

Superior task execution

RAG-enhanced

More accurate context retrieval

Math expert

Precise mathematical knowledge

Model Capabilities

Text generation

Logical reasoning

Task completion

Mathematical computation

Contextual memory

Use Cases

Text generation

Dialogue systems

Building intelligent chatbots

Smooth and natural conversational experience

Content creation

Generating articles, stories, and other text content

High-quality, coherent text output

Mathematical computation

Math problem solving

Solving complex mathematical problems

Accurate mathematical knowledge output

🚀 Saanvi-C0-12B 🤖⚡

A next-generation 12B LLM optimized for speed, efficiency, and contextual accuracy. Powered by RAG-based enhancements, 4-bit quantization, Flash Attention 2, bfloat16, and a 128k context window.

✨ Features

🚀 Why Upgrade to Saanvi-C0-12B?

Saanvi-C0-12B brings a huge leap in capability over smaller models, maintaining efficiency while significantly improving reasoning, fluency, task completion, and math!

Feature	Benefit
⚡ Flash Attention 2	Up to 2.7× faster inference
🧠 4-bit Quantization	Runs on 8GB VRAM GPUs
🎯 Instruction-Tuned	Better task performance
🔥 RAG-Enhanced	More precise contextual retrieval
➗ Math-Expert	Precise Mathematics knowledge

🖥️ Optimized for Mid-Tier GPUs

Runs on mid-range GPUs with 8GB+ VRAM (RTX 3050, RTX 2060, etc.).
More robust than our 3B model with better contextual retention and instruction-following.
4-bit quantization minimizes VRAM usage without sacrificing quality.

🚀 Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "riple-saanvi-lab/Saanvi-C0-12B"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype="bfloat16", device_map="auto")

while True:
    user_input = input("\n👤 You: ").strip()
    if user_input.lower() == "exit":
        break
    inputs = tokenizer(user_input, return_tensors="pt").to(model.device)
    output = model.generate(**inputs, max_length=2048, do_sample=True)
    print("🤖 AI:", tokenizer.decode(output[0], skip_special_tokens=True))

📦 Installation

pip install torch transformers

📚 Documentation

📊 Benchmarks

A100-40GB Performance

Batch Size	Throughput	Latency	VRAM Usage
1	42 tok/sec	85ms	8.2GB
8	218 tok/sec	430ms	12.5GB

🚀 On Mid-Tier GPUs (RTX 3050, RTX 2060, RTX 3060 12GB)

VRAM Usage: ~8.2GB (single batch)
Speed: ~10-15 tok/sec
Best Practices: Stick to smaller batch sizes for best performance.

📄 License

Licensed under the Apache 2.0 License. See the LICENSE file for details.

💡 Usage Tip

For maximum efficiency, use torch.compile() and CUDA graphs on high-end GPUs!

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご