Pose Recognition Unity SDK

This was another Unity Plugin SDK project for iOS and Android, but with particular emphasis on optimisation.

Nexus Studios were using Tensorflow in an AR dance game to track player’s movements and apply particle effects as they moved. For this to perform well, data needed to be sent to Tensorflow and the resulting pose data retrieved back as quickly as possible – otherwise the effects would be out of sync with the player movement.

This involved optimising the AR image pipeline, the C# to native interop, image manipulation within Tensorflow itself, and delivering the data back to my C# API for the game to use.