ML Agents SnowballFight 1vs1

🚀 Snowball Fight ☃️, a multi-agent environment for ML-Agents made by Hugging Face

A multi - agent environment using Unity ML - Agents Toolkit where two agents compete in a 1vs1 snowball fight game.

🚀 Quick Start

A multi - agent environment using Unity ML - Agents Toolkit where two agents compete in a 1vs1 snowball fight game.

👉 You can play it online at this link.

⚠️ Important Note

You need to have some skills in ML - Agents if you want to use it. If it's not the case, check the documentation

✨ Features

The Environment

• Two agents compete in a 1 vs 1 snowball fight game.
• The goal is to hit the opponent team while avoiding the opponent's snowballs ❄️.

Observation Space

• Ray - casts:

  • 10 ray - casts forward distributed over 100 degrees: detecting opponent.
  • 10 ray - casts forward distributed over 100 degrees: detecting walls, shelter and frontier.
  • 10 ray - casts forward distributed over 100 degrees: detecting snowballs.
  • 3 ray - casts backward distributed over 45 degrees: detecting wall and shelter.

• Vector Observations:

  • Bool canShoot (you can only shoot a snowball every 2 seconds).
  • Float currentHealth: normalized [0, 1]
  • Vector3 vertical speed
  • Vector3 horizontal speed
  • Vector3 "home" position

Action Space (Discrete)

• Vector Action space:
  • Four branched actions corresponding to forward, backward, sideways movement, rotation, and snowball shoot.

Agent Reward Function (dependant):

• If the team is injured:
  • 0.1 to the shooter.
• If the team is dead:
  • (1 - accumulated time penalty): when a snowball hits the opponent, the accumulated time penalty decreases by (1 / MaxStep) every fixed update and is reset to 0 at the beginning of an episode.
  • (-1) When a snowball hit our team.

Addendum

• There is no friendly fire, which means that an agent can't shoot himself, or in the future, in a 2vs2 game can't shoot a teammate.

📦 Installation

Set - up the environment

1. Clone this project git clone https://huggingface.co/ThomasSimonini/ML-Agents-SnowballFight-1vs1
2. Open Unity Hub and create a new 3D Project
3. In the cloned project folder, open .\ML-Agents-SnowballFight-1vs1\packages and copy manifest.json and package.lock.json
4. Paste these two files in Your Unity Project\Packages => this will install the required packages.
5. Drop the SnowballFight - 1vs1 unity package to your Unity Project.

Watch the trained agents

6. If you want to watch the trained agents, open Assets\1vs1\Scenes\1vs1_v2_Training. place the \ML-Agents-SnowballFight-1vs1\saved_model\SnowballFight1vs1-4999988.onnx into BlueAgent and PurpleAgent Model.

Train the agent

6. If you want to train it again, the scene is Assets\1vs1\Scenes\1vs1_v2_Training.

📚 Documentation

Training info

• SnowballFight1vs1 was trained with 5100000 steps.
• The final ELO score was 1766.452.

Config File

behaviors:
  SnowballFight1vs1:
    trainer_type: ppo
    hyperparameters:
      batch_size: 2048
      buffer_size: 20480
      learning_rate: 0.0003
      beta: 0.005
      epsilon: 0.2
      lambd: 0.95
      num_epoch: 3
      learning_rate_schedule: constant
    network_settings:
      normalize: false
      hidden_units: 512
      num_layers: 2
      vis_encode_type: simple
    reward_signals:
      extrinsic:
        gamma: 0.99
        strength: 1.0
    keep_checkpoints: 40
    checkpoint_interval: 200000
    max_steps: 50000000
    time_horizon: 1000
    summary_freq: 50000
    self_play:
      save_steps: 50000
      team_change: 200000
      swap_steps: 2000
      window: 10
      play_against_latest_model_ratio: 0.5
      initial_elo: 1200.0

📄 License

This project is licensed under the Apache - 2.0 license.