Solar Pro Preview Instruct

🚀 Solar Pro Preview: The most intelligent LLM on a single GPU

Solar Pro Preview is an advanced large language model with 22 billion parameters, designed to fit into a single GPU and offering superior performance.

🚀 Quick Start

Solar Pro Preview is an instruction - tuned large language model (LLM) with 22 billion parameters. It's designed to run on a single GPU with 80GB of VRAM. This pre - release version shows great potential, though it has limitations in language coverage and a 4K maximum context length. The official Solar Pro will be released in November 2024 with extended language support and longer context windows.

✨ Features

• Single - GPU Compatibility: It can fit into a single GPU, making it accessible for users with standard GPU setups.
• High Performance: Outperforms many LLMs with less than 30 billion parameters and rivals models over three times its size, like Llama 3.1 with 70 billion parameters.
• Enhanced Training: Developed using an enhanced depth up - scaling method, trained on a carefully curated dataset, which significantly improves performance on benchmarks like MMLU - Pro and IFEval.

📦 Installation

To use Solar Pro Preview, you need to install the necessary libraries. Here are the installation commands:

# Install requirements
# !pip install transformers==4.44.2 torch==2.3.1 flash_attn==2.5.8 accelerate==0.31.0

💻 Usage Examples

Basic Usage

# Load model
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("upstage/solar-pro-preview-instruct")
model = AutoModelForCausalLM.from_pretrained(
    "upstage/solar-pro-preview-instruct",
    device_map="cuda",  
    torch_dtype="auto",  
    trust_remote_code=True,
)
# Apply chat template
messages = [
    {"role": "user", "content": "Please, introduce yourself."},
]
prompt = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
# Generate text
outputs = model.generate(prompt, max_new_tokens=512)
print(tokenizer.decode(outputs[0]))

Advanced Usage

Solar Pro Preview is also available as an API in Upstage Console. For more information and other easy - to - use methods, visit our blog page.

📚 Documentation

Chat Template

As an instruction - tuned model, Solar Pro Preview uses the ChatML template for optimal performance in conversational and instruction - following tasks. Here is an example:

<|im_start|>user
Please, introduce yourself.<|im_end|>
<|im_start|>assistant

Note that system prompts are not currently supported in Solar Pro Preview. This feature will be available in the official release.

🔧 Technical Details

Solar Pro Preview is developed using an enhanced version of the previous depth up - scaling method. It scales a Phi - 3 - medium model with 14 billion parameters to 22 billion parameters. The carefully curated training strategy and dataset have significantly enhanced performance, especially on the MMLU - Pro and IFEval benchmarks.

📄 License

Solar Pro Preview is released under the MIT License.

📊 Evaluation

Solar Pro Preview is evaluated over a variety of benchmarks. The following table shows the comparison with other models:

(*) Since the model tends to generate a chat template, the score can't be accurately determined.

Evaluation Protocol

The following table lists the evaluation tools and settings used:

The results may vary slightly for different batch sizes and experimental environments such as GPU type.

📞 Contact us

For any questions and suggestions regarding the model, please visit the discussion board.

Learn more:

• Chat with Solar Pro Preview
• Solar Pro Preview blog
• Solar Pro Preview developer documents

Also try out:

• Document Parse: An industry - leading model for converting complex document files to LLM - compatible HTML formats.
• Solar DocVision Preview: A vision LLM specialized on documents.