The main software components for the implementation of the DECT are: Frapper (Agent Framework), PsychoPy, and the SMI’s RED-X eye tracker API.
Frapper, the Filmakademie Application Framework, provides the back-end and front-end for the DECT. The Agent Framework is a compendium of all the functionalities for the generation of real-time facial animation, which is the core of our project. In order to satisfy all the requirements for SARA, Frapper has been empowered and enhanced to run using OpenGL 3+, allowing the generation of real-time hair rendering and real-time non-photorealistic rendering (NPR) algorithms. Frapper relies on Ogre 1.10 as the rendering engine. Regarding the characters, these have been created in different DCC tools, and then animated using our Facial Animation Toolset (FAT), which is distributed as a plugin for Maya.
Frapper is also the central element in the communication with PsychoPy and the SMI eye tracker. The communication with PsychoPy is established via sockets, which allow the exchange of messages indicating Frapper when to generate the emotional facial animation of a given character with a given speed, intensity and level of abstraction (i.e. in the case a certain NPR style has been applied). The communication with the SMI eye tracker is established using the API provided that the eye tracker.
Download SARA Frapper Deploy (510 MB). The deploy contains: an example scene, the NPR-DECT, the i-DECT, and all characters - Nikita, Sara, Hank, Gunnar - separately.