Octo Base

🚀 Octo Base

Octo Base is a model in the field of robotics. It can predict 7 - dimensional actions 4 steps into the future using a diffusion policy with a window size of 2. This model provides a new solution for robotic control and decision - making.

🚀 Quick Start

See https://github.com/octo - models/octo for instructions for using this model.

✨ Features

• Model Structure: Octo Base is a Transformer with 93M parameters (equivalent to a ViT - B).
• Tokenization: Images are tokenized by preprocessing with a lightweight convolutional encoder and then grouped into 16x16 patches. Language is tokenized by applying the T5 tokenizer and then applying the T5 - Base language encoder.
• Observation and Task Specification: It has specific specifications for observations and tasks, which are described in detail below.
• Inference Flexibility: At inference, you may pass in any subset of the observation and task keys, with a history window up to 2 timesteps.

🔧 Technical Details

Octo Base is trained with a window size of 2, predicting 7 - dimensional actions 4 steps into the future using a diffusion policy. The model is a Transformer with 93M parameters (equivalent to a ViT - B).

Observation Specification

Observations conform to the following spec:

{
    image_primary: ('batch', 'history_window', 256, 256, 3),
    image_wrist: ('batch', 'history_window', 128, 128, 3),
}

Task Specification

Tasks conform to the following spec:

{
    image_primary: ('batch', 256, 256, 3),
    image_wrist: ('batch', 128, 128, 3),
    language_instruction: {
        attention_mask: ('batch', 16),
        input_ids: ('batch', 16),
    },
}

Training Datasets

This model was trained on a mix of datasets from the Open X - Embodiment dataset. The proportion of each dataset in the batch is as follows:

📄 License

This project is licensed under the MIT license.