vqbet_pusht Open-source Behavior Generation Model - Trained for the PushT Environment to Aid Behavior Creation

Home

Vqbet Pusht

Developed by lerobot

VQ-BeT is a behavior generation model trained for the PushT environment, designed based on latent action principles

Image Generation

Transformers

Open Source License:Apache-2.0 #Robot Control #Latent Action Generation #PushT Environment

Downloads 68

Release Time : 7/3/2024

Model Overview

This model is based on the VQ-BeT architecture, specifically trained for the robot pushing task (PushT) environment, capable of generating action sequences for robot control

Model Features

Latent Action Representation

Uses vector quantization techniques to learn discrete latent action spaces, improving the robustness of behavior generation

Efficient Training

Achieves good performance after only 250,000 training steps, demonstrating high training efficiency

Multimodal Input Processing

Capable of processing RGB image inputs, suitable for real-world robot applications

Model Capabilities

Robot Action Generation

Visual Input Processing

Pushing Task Execution

Use Cases

Robot Control

Object Pushing Task

Controls the robot to push an object to a target position

In 500 tests, achieved an average maximum overlap rate of 89.5% and a success rate of 63.8%

🚀 Model Card for VQ-BeT/PushT

VQ-BeT is trained for the PushT environment, offering advanced solutions for robotics tasks.

🚀 Quick Start

See the LeRobot library (particularly the evaluation script) for instructions on how to load and evaluate this model.

✨ Features

VQ-BeT Policy: Based on the VQ-BeT architecture, trained for the PushT environment.
Robotics Application: Specifically designed for robotics tasks, with a focus on the PushT scenario.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

To load and evaluate the model, refer to the LeRobot library and the evaluation script.

📚 Documentation

Training Details

Trained with LeRobot@3c0a209. The model was trained using LeRobot's training script and with the pusht dataset, using this command:

python lerobot/scripts/train.py \
    --output_dir=outputs/train/vqbet_pusht \
    --policy.type=vqbet \
    --dataset.repo_id=lerobot/pusht \
    --env.type=pusht \
    --seed=100000 \
    --batch_size=64 \
    --steps=250000 \
    --eval_freq=25000 \
    --save_freq=25000 \
    --wandb.enable=true

The training curves may be found at https://wandb.ai/aliberts/lerobot/runs/3i7zs94u. The current model corresponds to the checkpoint at 200k steps.

Model Size

Property	Details
RGB Encoder	11.2M
Remaining VQ-BeT Parts	26.3M

Evaluation

The model was evaluated on the PushT environment from gym-pusht. There are two evaluation metrics on a per-episode basis:

Maximum overlap with target (seen as eval/avg_max_reward in the charts above). This ranges in [0, 1].
Success: whether or not the maximum overlap is at least 95%.

Here are the metrics for 500 episodes worth of evaluation.

Metric	Value
Average max. overlap ratio for 500 episodes	0.895
Success rate for 500 episodes (%)	63.8

The results of each of the individual rollouts may be found in eval_info.json. It was produced after training with this command:

python lerobot/scripts/eval.py \
    --policy.path=outputs/train/vqbet_pusht/checkpoints/200000/pretrained_model \
    --output_dir=outputs/eval/vqbet_pusht/200000 \
    --env.type=pusht \
    --seed=100000 \
    --eval.n_episodes=500 \
    --eval.batch_size=50 \
    --device=cuda \
    --use_amp=false

📄 License

This project is licensed under the Apache-2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご