Octo Base 1.5

🚀 Octo Base

Octo Base is a robotics model. It can predict 7 - dimensional actions 4 steps into the future using a diffusion policy with a window size of 2. This model provides a solution for robotic action prediction and is trained on a mix of datasets from the Open X - Embodiment dataset.

🚀 Quick Start

See https://github.com/octo-models/octo for instructions for using this model.

✨ Features

• Octo Base is trained with a window size of 2, predicting 7 - dimensional actions 4 steps into the future using a diffusion policy.
• The model is a Transformer with 93M parameters (equivalent to a ViT - B).
• Images are tokenized by preprocessing with a lightweight convolutional encoder, then grouped into 16x16 patches.
• Language is tokenized by applying the T5 tokenizer, and then applying the T5 - Base language encoder.

🔧 Technical Details

Observation and Task Specification

Observations and tasks conform to the following spec:

Observations:

{
    image_primary: ('batch', 'history_window', 256, 256, 3),
    image_wrist: ('batch', 'history_window', 128, 128, 3),
}

Tasks:

{
    image_primary: ('batch', 256, 256, 3),
    image_wrist: ('batch', 128, 128, 3),
    language_instruction: {
        attention_mask: ('batch', 16),
        input_ids: ('batch', 16),
    },
}

At inference, you may pass in any subset of these observation and task keys, with a history window up to 2 timesteps.

Training Datasets

This model was trained on a mix of datasets from the Open X - Embodiment dataset.

📚 Documentation

Updates for Version 1.5

• Language task tokens are now repeated at every timestep in the context window.
• Augmented the language instructions in the data with rephrasings from GPT - 3.5.
• Bug fixes:
  • Turned off dropout in the diffusion head due to incompatibility with layer norm.
  • Fixed an off - by - one error with the attention mask.
  • Fixed an issue where different image augmentations did not get fresh random seeds.

📄 License

This project is licensed under the MIT license.