Qwen2 96M

🚀 Qwen2-96M

Qwen2-96M is a small language model based on the Qwen2 architecture. It's trained from scratch on English datasets with a context length of 8192 tokens. With only 96 million parameters, it serves as a lightweight base model for specific task fine - tuning.

🚀 Quick Start

Qwen2-96M is a small - scale language model. It offers a lightweight option for those looking to fine - tune a model for specific tasks. However, it has limitations compared to larger models.

✨ Features

• Compact Architecture: With only 96 million parameters, it's a lightweight base model.
• English - trained: Trained from scratch on English datasets.
• Long Context: Supports a context length of 8192 tokens.

📦 Installation

To use Qwen2-96M, you need to install the required libraries:

pip install transformers==4.49.0 torch==2.6.0

💻 Usage Examples

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
import torch

model_path = "Felladrin/Qwen2-96M"
prompt = "I've been thinking about"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path).to(device)
streamer = TextStreamer(tokenizer)
inputs = tokenizer(prompt, return_tensors="pt").to(device)
model.generate(
    inputs.input_ids,
    attention_mask=inputs.attention_mask,
    max_length=tokenizer.model_max_length,
    streamer=streamer,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id,
    repetition_penalty=1.1,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    top_k=0,
    min_p=0.1,
)

⚠️ Important Note

Due to its compact size, the model has significant limitations in reasoning, factual knowledge, and general capabilities compared to larger models. It may produce incorrect, irrelevant, or nonsensical outputs. Additionally, as it was trained on internet text data, it may contain biases and potentially generate inappropriate content.