Theia Base Patch16 224 Cdiv

🚀 Theia

Theiaは、ロボット学習用のビジョン基礎モデルです。様々なビジョンタスクで学習された複数の既存のビジョン基礎モデルを蒸留しています。Theiaの豊富な視覚表現は多様な視覚知識をエンコードし、下流のロボット学習を強化します。このモデルは論文 Theia: Distilling Diverse Vision Foundation Models for Robot Learning で紹介されており、この論文にはTheiaが教師モデルや以前のロボット学習モデルを上回り、少ない学習データと小さなモデルサイズで動作することを示す実験結果も含まれています。デモビデオはプロジェクトページで見ることができます。

The AI Institute

🚀 クイックスタート

Theiaは、ロボット学習用の革新的なビジョン基礎モデルです。多様なビジョン基礎モデルを蒸留することで、豊富な視覚表現を獲得し、下流のロボット学習タスクを強化します。

✨ 主な機能

• 複数の既存のビジョン基礎モデルを蒸留し、多様な視覚知識をエンコード。
• 少ない学習データと小さなモデルサイズで、教師モデルや以前のロボット学習モデルを上回る性能を発揮。

📚 ドキュメント

モデル詳細

theia-base-patch16-224-cdiv モデルは、DeiT-Base をバックボーンとして使用し、CLIP、DINOv2、および ViT を同時に蒸留します。使用方法の詳細については、Theiaリポジトリ をご覧ください。

引用

もしあなたの研究でTheiaを使用する場合は、以下のBibTeXエントリを使用してください。

@article{shang2024theia,
  author    = {Shang, Jinghuan and Schmeckpeper, Karl and May, Brandon B. and Minniti, Maria Vittoria and Kelestemur, Tarik and Watkins, David and Herlant, Laura},
  title     = {Theia: Distilling Diverse Vision Foundation Models for Robot Learning},
  journal   = {arXiv},
  year      = {2024},
}

使用方法

Theiaとともに公開された事前学習済みモデルの重みとコードは、The AI Institute License の下で使用できます。以下にライセンス全文を再掲します。

Copyright (c) 2024 Boston Dynamics AI Institute LLC

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the copyright notice included
with the software, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the copyright notice, this
list of conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.
3. Modified versions of the software must be conspicuously marked as such.
4. The software may only be used for non-commercial research purposes.
For profit enterprises may use the software, subject to this limitation.

THIS SOFTWARE IS PROVIDED BY THE AI INSTITUTE AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, NON-
INFRINGEMENT,TITLE, MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE AI INSTITUTE OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, PUNITIVE OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, DAMAGES ARISING OUT OF CLAIMS OF
INTELLECTUAL PROPERTY RIGHTS INFRINGEMENT; PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

📄 ライセンス

本プロジェクトは The AI Institute License の下で提供されています。

Copyright (c) 2024 Boston Dynamics AI Institute LLC

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the copyright notice included
with the software, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the copyright notice, this
list of conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.
3. Modified versions of the software must be conspicuously marked as such.
4. The software may only be used for non-commercial research purposes.
For profit enterprises may use the software, subject to this limitation.

THIS SOFTWARE IS PROVIDED BY THE AI INSTITUTE AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, NON-
INFRINGEMENT,TITLE, MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE AI INSTITUTE OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, PUNITIVE OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, DAMAGES ARISING OUT OF CLAIMS OF
INTELLECTUAL PROPERTY RIGHTS INFRINGEMENT; PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.