Sentis Face Landmarks

🚀 Face Landmark from Google Mediapipe validated for Unity Sentis (Version 1.4.0-pre.2*)

This project provides a Face Landmark model from Google in the Sentis format. It can detect 468 3D markers on a centered face in an image, which can be used for various applications like mesh control.

🚀 Quick Start

Prerequisites

• Unity 2023 installed on your system.

Steps

1. Create a new scene in Unity 2023.
2. Place the face_landmark.sentis file in the Assets/StreamingAssets folder.
3. Install the com.unity.sentis package version 1.4.0-pre.2 from the package manager.
4. Put a video in the Assets/StreamingAssets folder and set the videoName variable to the video name.
5. Create a RawImage and place it in your scene. Link to this image in the previewUI field.

✨ Features

• Accurate Face Landmark Detection: The model can detect 468 3D markers on a face centered in an image.
• Versatile Application: These markers can be used, for example, to control a Mesh, such as this one in obj format.

💻 Usage Examples

Basic Usage

To use the model, follow the steps in the "Quick Start" section. After setting up the environment, the model will start detecting face landmarks in the provided video.

Advanced Usage

For images with faces not in the center or for multiple faces, you need to use another model to detect and crop the faces before feeding them into this model. For example, you can use Blaze Face to detect the faces.

📚 Documentation

Important Note

• The face needs to be centered and cropped to fit the image. If the faces are not centered or there are multiple faces in the image, you need to use an additional face detection model to crop the faces before using this model.
• Version 1.3.0 Sentis files are not compatible with Sentis 1.4.0 and need to be recreated/downloaded.

Model Details

This is the Face Landmark model from Google in the Sentis format.

Preview

If you get it working, it should look like this (original image from pexels.com):

📄 License

All Google Mediapipe models are open source under the Apache 2.0 license. The accompanying C# source code we provide can be used in your applications for commercial purposes.

⚠️ Important Note

The face needs to be centered and cropped to fit the image. For images with faces not in the center or for multiple faces, you will need another model to detect faces, and crop them before feeding them into this model. For example you could use Blaze Face to detect them.

💡 Usage Tip

Make sure to install the correct version of the com.unity.sentis package (version 1.4.0-pre.2) from the package manager. Also, ensure that the face_landmark.sentis file and the video are placed in the Assets/StreamingAssets folder.