Face mesh tracking with ARKit + SceneKit

Related tags

ARKit FaceMeshDemo
Overview

Face Mesh with iOS ARKit + SceneKit

So I was exploring ways to deploy face landmark detector onto a mobile environment. The only way I know how was to either deploy with Tensorflow Lite or PyTorch Mobile.

Both of them are great in terms of inference speed and accuracy, but nothing beats the native SDK! Check out the Face API from Apple here

The advantages of ARKit over custom ML models here being:

  1. Ultra fast inference speed
  2. Short warmup time
  3. Buttery smooth animation
  4. Less than 50 lines of code! And no machine learning knowledge needed!

Disadvantages:

  1. Only supports devices from iPhone X and above
  2. It is not portable for Android counterparts. It means for business purposes, it is not possible to deliver the same experience for our customers using Android devices :(

Demo video :)

RPReplay_Final1620136247.mov
Owner
Rex Low
Staff @RevenueMonster
Rex Low
This library uses ARKit Face Tracking in order to catch user's smile.

SmileToUnlock Make your users smile before opening the app :) Gif with the demonstration Installation Cocoapods The most preferable way to use this li

Ruslan Serebriakov 622 Nov 8, 2021
Make your UIScrollView scroll automatically when user is looking 👀 by tracking face using ARKit and AVFoundation

Baraba from Korean: meaning, Look at me Make your UIScrollView scroll automatically when user is looking at the screen ?? Features Automatic scrolling

Soojin Ro 289 Nov 8, 2021
A demonstration of vertical planes "tracking" and occlusions with ARKit+Scenekit

arkit-occlusion-demo Bjarne Lundgren / [email protected] / @bjarnel This ARKit+SceneKit project shows how to "track" vertical planes (relative to horizo

Bjarne Lundgren 345 Sep 29, 2021
Show your GitHub commit records in 3D with ARKit and SceneKit. 用 ARKit 展示你的 GitHub 提交图

ARGitHubCommits Show your GitHub commit records in 3D with ARKit and SceneKit. Listed in Awesome ARKit now. Welcome to Star ⭐️ ~ Preview Requirement i

Kuixi Song 282 Nov 8, 2021
A library that allows you to generate and update environment maps in real-time using the camera feed and ARKit's tracking capabilities.

ARKitEnvironmentMapper Example To run the example project, clone the repo, and run pod install from the Example directory first. Installation ARKitEnv

SV Hawks 87 Nov 24, 2021
A library that allows you to generate and update environment maps in real-time using the camera feed and ARKit's tracking capabilities.

ARKitEnvironmentMapper Example To run the example project, clone the repo, and run pod install from the Example directory first. Installation ARKitEnv

SV Hawks 86 Jul 22, 2021
Quick demo of 3d drawing in ARKit using metal + SceneKit. NOTE: This is experimental code!

ARBrush Quick demo of 3d drawing in ARKit using metal + SceneKit. Code adapted from: https://github.com/lapfelix/ARKit-line-drawing and: https://www.r

laan labs 774 Nov 10, 2021
Tic-Tac-Toe implemented using ARKit+Scenekit

arkit-tictactoe Bjarne Lundgren / [email protected] / @bjarnel In this example I have taken the simple scenekit tic-tac-toe (another of my repositories)

Bjarne Lundgren 383 Nov 14, 2021
Augmented Reality Tetris made with ARKit and SceneKit

ARTetris Augmented Reality Tetris made with ARKit and SceneKit We are a development agency building phenomenal apps. FAQ App crashes when running on i

Exyte 1.5k Oct 1, 2021
Make VR with SceneKit & ARKit.

ARKitPlusVR VR with SceneKit & ARKit. This project demonstrates the using of ARKit to make the movement in VR scene possible. Requirements A9 ( iPhone

WorkerAmo 81 Oct 21, 2021
An implementation of a Super Mario Bros-like game in augmented reality with ARKit and SceneKit.

arkit-smb-homage Bjarne Lundgren / [email protected] / @bjarnel This project is a homage to Super Mario Bros. It does not, however, contain ANY original

Bjarne Lundgren 244 Nov 6, 2021
Shell Game built with ARKit and SceneKit

ARKit Shell Game Shell Game built with ARKit and SceneKit. Good sample to learn AR/3D basics. Requirements XCode 9 Device running iOS, with an A9 chip

Handsome 85 Jul 10, 2021
Record your SceneKit and ARKit scenes easily.

SceneKitVideoRecorder Example To run the example project, clone the repo, and run pod install from the Example directory first. Apps using SceneKitVid

SV Hawks 225 Nov 13, 2021
Voxel graphics framework using ARKit + SceneKit

ARVoxelKit Lightweight Framework for Voxel graphic using AR + SceneKit Requirements ARVoxelKit requires iOS 11 and devices, which support ARKit Usage

null 71 Jun 8, 2021
Clean class to create a portal in SceneKit for use in ARKit.

SceneKit-PortalMask See my Medium post about this CocoaPod This class PortalMask creates an occluding box around any rectangular frame, including a tr

Max Cobb 53 Sep 8, 2021
Augmented Reality Movie Trailer made with ARKit and SceneKit.

ARTrailer Identify the movie title from a poster, and use SceneKit to display the movie trailer in AR. Overview This app runs an ARKit world-tracking

Wai Ting Cheung 14 Oct 30, 2021
Easily use ARKit to detect facial gestures. FaceTrigger is a simple to use class that hides the details of using ARKit's ARSCNView to recognize facial gestures via ARFaceAnchor.BlendShapeLocations

FaceTrigger Introduction FaceTrigger is a simple to use class that hides the details of using ARKit's ARSCNView to recognize facial gestures via ARFac

Michael Peterson 65 Sep 4, 2021
An ARKit version of WeChat Bottle Jump game. ARKit 版微信跳一跳游戏

ARBottleJump An ARKit version of WeChat Bottle Jump game. Welcome to Star ⭐️ ~ Instruction After launching, please move your iPhone around in order to

Kuixi Song 262 Nov 9, 2021
Detects faces using the Vision-API and runs the extracted face through a CoreML-model to identiy the specific persons.

FaceRecognition in ARKit This is a simple showcase project, that detects faces using the Vision-API and runs the extracted face through a CoreML-model

Novatec Consulting GmbH 779 Nov 23, 2021