Accurat Unity Plugin SDK

Another Unity plugin project – I seem to get a lot of these! – exposing native iOS and Android libraries to Unity and constructing a C# friendly SDK around them.

This project was initially problematic, as both libraries inherited a huge number of Google libraries, leading to conflicts and linking problems. Not only that, the underlying libraries’ kept updating with behaviour changes, so what started out as a simple three week contract lasted about a year!

Deathtrap Dungeon: The Interactive Video Adventure

Forming Branching Narrative with Matt Spall, we spent 18 months developing this game together, with the infamous Stoo Cambridge providing graphics and UI.

The game is entirely written in JavaScript, contained within bespoke native code wrappers. This enables us to publish as native apps on PC, Mac, iOS, Android and console; even allowing an online streaming version, which we use as a playable demo directly on branching narrative.com.

We’re planning on releasing more Fighting Fantasy titles with a variety of actors in the future.

Buy it now!

Pose Recognition Unity SDK

This was another Unity Plugin SDK project for iOS and Android, but with particular emphasis on optimisation.

Nexus Studios were using Tensorflow in an AR dance game to track player’s movements and apply particle effects as they moved. For this to perform well, data needed to be sent to Tensorflow and the resulting pose data retrieved back as quickly as possible – otherwise the effects would be out of sync with the player movement.

This involved optimising the AR image pipeline, the C# to native interop, image manipulation within Tensorflow itself, and delivering the data back to my C# API for the game to use.

Headcaster

Headcaster was a next-generation messaging app for iOS and Android, allowing you to deliver animated, lip-synced messages via a variety of characters – ranging from cute kittens, through to the zombied reincarnation of Margaret Thatcher. Celebrities use the app too – notably Stephen Fry and Rio Ferdinand – to deliver podcasts via custom made caricatures of themselves.


We also developed a desktop version for broadcast purposes, for a series of short satirical pieces for BT Sport. This allowed for camera movements, green backgrounds and multiple characters.


We developed this over 18 months using Unity, with myself as CTO overseeing a team of six coders and 4 artists whilst developing native code plugins for video sharing. Challenges included building a patented lip sync system which worked across all languages, and building a server architecture able to withstand spikes in traffic from Rio tweeting a headcast to his 7 million followers.

Playora

Playora is an iOS and Android app allowing you to play PC or Flash games on a phone/tablet, or stream content from a PC or phone/tablet to a Smart TV.


Challenges included low-latency video encoding, so games could be played without any noticable lag, and DLL injection coding, effectively hijacking a game or app’s visual and audio output from within the app itself, rather than simply recording the screen.