🚀 Sarashina2-Vision-14B
Sarashina2-Vision-14B is a Japanese Large Vision Language Model trained by SB Intuitions. It combines the power of Sarashina2-13B and the Image Encoder of Qwen2-VL-7B. As of 2025/03/07, it has achieved the highest scores in 4 benchmarks compared to other Japanese VLMs.
🚀 Quick Start
📦 Installation
First, you need to install the necessary dependencies:
pip install -U transformers==4.47.0 torch torchvision pillow protobuf sentencepiece accelerate
💻 Usage Examples
Basic Usage
The following script demonstrates how to load the model and perform inference:
import requests
from PIL import Image
from transformers import AutoModelForCausalLM, AutoProcessor
model_path = "sbintuitions/sarashina2-vision-14b"
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_path,
device_map="cuda",
torch_dtype="auto",
trust_remote_code=True,
)
message = [{"role": "user", "content": "この写真に写っているもので、最も有名と考えられる建築物は何でどこに写っていますか?"}]
text_prompt = processor.apply_chat_template(message, add_generation_prompt=True)
"""text_prompt: <s><|prefix|><|file|><|suffix|>A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.
### Human: この写真に写っているもので、最も有名と考えられる建築物は何でどこに写っていますか?
### Assistant:"""
sample_image_url = "https://huggingface.co/sbintuitions/sarashina2-vision-14b/resolve/main/sample.jpg"
image = Image.open(requests.get(sample_image_url, stream=True).raw).convert("RGB")
inputs = processor(
text=[text_prompt],
images=[image],
padding=True,
return_tensors="pt",
)
inputs = inputs.to("cuda")
stopping_criteria = processor.get_stopping_criteria(["\n###"])
output_ids = model.generate(
**inputs,
max_new_tokens=128,
temperature=0.0,
do_sample=False,
stopping_criteria=stopping_criteria,
)
generated_ids = [
output_ids[len(input_ids) :] for input_ids, output_ids in zip(inputs.input_ids, output_ids)
]
output_text = processor.batch_decode(
generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True
)
print(output_text[0])
"""この写真に写っているもので、最も有名と考えられる建築物は東京タワーです。東京タワーは、東京の街並みの右側に写っています。"""
Example
Here is an example of input and output:

Prompt |
Output |
この写真に写っているもので、最も有名と考えられる建築物は何でどこに写っていますか? |
この写真に写っているもので、最も有名と考えられる建築物は東京タワーです。東京タワーは、東京の街並みの右側に写っています。 |
真ん中に映っている赤と白の物は何ですか? |
赤と白の物はクレーンです。 |
🔧 Technical Details
Training
Sarashina2-Vision is developed through a three-stage learning process:
- Tune the parameters in the projector using caption datasets.
- Tune the parameters in the Vision Encoder and projector using caption datasets.
- Tune the parameters in the projector and LLM using Visual Instruction datasets.
Evaluation Results
The following table shows the evaluation results of different models on several benchmarks:
- Evaluated only single image samples (1,286 samples). If answer extraction failed, it was treated as incorrect (score 0) instead of making a random choice to eliminate stochasticity.
- GPT-4o (gpt-4o-2024-08-06) was used for LLM-as-a-Judge.
📚 Documentation
Ethical Considerations and Limitations
Sarashina2-Vision may generate some meaningless sequences, inaccurate instances, or biased/objectionable outputs. Developers are advised to tune the models based on human preferences and safety considerations before use.
📄 License
This project is licensed under the MIT License.