This was another Unity Plugin SDK project for iOS and Android, but with particular emphasis on optimisation.
Nexus Studios were using Tensorflow in an AR dance game to track player’s movements and apply particle effects as they moved. For this to perform well, data needed to be sent to Tensorflow and the resulting pose data retrieved back as quickly as possible – otherwise the effects would be out of sync with the player movement.
This involved optimising the AR image pipeline, the C# to native interop, image manipulation within Tensorflow itself, and delivering the data back to my C# API for the game to use.
Another Unity native code Plugin project – this time across PC, PS4 and XBone! As well as developing the cross-platform C# plugin SDK, I had to also develop the underlying native console device drivers, delivering USB data from the MARS hub up to the C# SDK for parsing, resulting in Unity-friendly events and delegates.
Headcaster was a next-generation messaging app for iOS and Android, allowing you to deliver animated, lip-synced messages via a variety of characters – ranging from cute kittens, through to the zombied reincarnation of Margaret Thatcher. Celebrities use the app too – notably Stephen Fry and Rio Ferdinand – to deliver podcasts via custom made caricatures of themselves.
We also developed a desktop version for broadcast purposes, for a series of short satirical pieces for BT Sport. This allowed for camera movements, green backgrounds and multiple characters.
We developed this over 18 months using Unity, with myself as CTO overseeing a team of six coders and 4 artists whilst developing native code plugins for video sharing. Challenges included building a patented lip sync system which worked across all languages, and building a server architecture able to withstand spikes in traffic from Rio tweeting a headcast to his 7 million followers.
Playora is an iOS and Android app allowing you to play PC or Flash games on a phone/tablet, or stream content from a PC or phone/tablet to a Smart TV.
Challenges included low-latency video encoding, so games could be played without any noticable lag, and DLL injection coding, effectively hijacking a game or app’s visual and audio output from within the app itself, rather than simply recording the screen.
A six month labour of love (read “made no money at all”), Gridrunner++ was a conversion of Jeff Minter’s classic game. Initially ported to ancient mobile phones running J2ME (through careful use of magic, alchemy and ritual sacrifice), the game was built using my own cross platform engine; allowing the game to also run within web browsers running Java and Flash. The Flash version also had an outing on Facebook, where it contained a graphical highscore table showing how your score compared with those of your friends.