Perception LM 1B

🚀 Perception Language Model (PLM)

Perception Language Model (PLM) is a state - of - the - art, fully open and reproducible MLLM designed for transparent research in image and video understanding. It addresses the challenges in these fields by leveraging advanced techniques and high - quality data, providing valuable insights and tools for researchers.

🚀 Quick Start

The training and evaluation code for PLM is available at perception_models codebase. You can find detailed instructions and more information in the GitHub repo.

✨ Features

• Advanced Architecture: PLM consists of a vision encoder paired with a small - scale (<8B parameters) LLM decoder.
• Data - Driven Approach: It starts with an analysis of standard training pipelines using available data, without relying on proprietary model distillation.
• Large - Scale Synthetic Data: Investigates large - scale synthetic data and establishes key scaling laws to identify data gaps in video understanding, especially for spatio - temporal reasoning and fine - grained understanding tasks.
• High - Quality Human - Labeled Data: To fill the identified gaps, 2.8M high - quality human - labeled data is created, which is nearly an order of magnitude larger than the largest existing video datasets.

📚 Documentation

Model Overview

PLM was introduced in "PerceptionLM: Open - Access Data and Models for Detailed Visual Understanding". You can also refer to the tech report and the GitHub repository.

Resources

Benchmark Results

PLM Image Benchmark Results

PLM Video Benchmark Results

📄 License

The use of PLM is subject to the FAIR Noncommercial Research License. By clicking “I Accept” or using or distributing any portion of the Research Materials, you agree to be bound by this Agreement.

Key Terms

• Acceptable Use Policy: The FAIR Acceptable Use Policy applicable to Research Materials.
• Agreement: The terms and conditions for using, reproducing, distributing, and modifying the Research Materials.
• Documentation: Specifications, manuals, and documentation accompanying the Research Materials distributed by Meta.
• Licensee: You, your employer, or any other person or entity entering into this Agreement.
• Meta: Meta Platforms Ireland Limited (if in the EEA or Switzerland) or Meta Platforms, Inc. (outside the EEA or Switzerland).
• Noncommercial Research Uses: Non - commercial research use cases not primarily for commercial advantage or monetary compensation.
• Research Materials: Documentation, models, software, algorithms, and related elements distributed by Meta under this Agreement.

Prohibited Uses

You agree not to use the Research Materials for:

• Illegal or Unlawful Activities: Such as violence, terrorism, exploitation of children, human trafficking, sexual solicitation, and other criminal activities.
• Harassment and Discrimination: Engaging in, promoting, or facilitating harassment, abuse, discrimination, or other unlawful or harmful conduct.
• Unauthorized Professional Practice: Unauthorized or unlicensed practice of any profession.
• Sensitive Information: Collecting, processing, or disclosing sensitive personal information without proper consent.
• Infringement: Engaging in actions that infringe, misappropriate, or violate third - party rights.
• Malicious Code: Creating or facilitating the creation of malicious code or anything that could harm a website or computer system.
• Dangerous Activities: Engaging in activities presenting a risk of death or bodily harm, such as military, warfare, or illegal weapon - related activities.
• Deception: Intentionally deceiving or misleading others, including generating fraud, disinformation, or spam.
• Failure to Disclose: Failing to appropriately disclose any known dangers of the Research Materials.

Please report any violations of this Policy at [https://docs.google.com/forms/d/e/1FAIpQLSeb11cryAopJ7LNrC4nxEUXrHY26hfkXQMf_uH - oFgA3WlYZQ/viewform].

📚 Citation

If you find our code useful for your research, please consider citing:

@article{cho2025PerceptionLM,
  title={PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding},
  author={Jang Hyun Cho and Andrea Madotto and Effrosyni Mavroudi and Triantafyllos Afouras and Tushar Nagarajan and Muhammad Maaz and Yale Song and Tengyu Ma and Shuming Hu and Hanoona Rasheed and Peize Sun and Po-Yao Huang and Daniel Bolya and Suyog Jain and Miguel Martin and Huiyu Wang and Nikhila Ravi and Shashank Jain and Temmy Stark and Shane Moon and Babak Damavandi and Vivian Lee and Andrew Westbury and Salman Khan and Philipp Kr\"{a}henb\"{u}hl and Piotr Doll{\'a}r and Lorenzo Torresani and Kristen Grauman and Christoph Feichtenhofer},
  journal={arXiv},
  year={2025}
}

@article{bolya2025PerceptionEncoder,
  title={Perception Encoder: The best visual embeddings are not at the output of the network},
  author={Daniel Bolya and Po-Yao Huang and Peize Sun and Jang Hyun Cho and Andrea Madotto and Chen Wei and Tengyu Ma and Jiale Zhi and Jathushan Rajasegaran and Hanoona Rasheed and Junke Wang and Marco Monteiro and Hu Xu and Shiyu Dong and Nikhila Ravi and Daniel Li and Piotr Doll{\'a}r and Christoph Feichtenhofer},
  journal={arXiv},
  year={2025}
}