Frapper Deoploy
The main software components for the implementation of the DECT are: Frapper (Agent Framework), PsychoPy, and the SMI’s RED-X eye tracker API.
Frapper, the Filmakademie Application Framework, provides the back-end and front-end for the DECT. The Agent Framework is a compendium of all the functionalities for the generation of real-time facial animation, which is the core of our project. In order to satisfy all the requirements for SARA, Frapper has been empowered and enhanced to run using OpenGL 3+, allowing the generation of real-time hair rendering and real-time non-photorealistic rendering (NPR) algorithms. Frapper relies on Ogre 1.10 as the rendering engine. Regarding the characters, these have been created in different DCC tools, and then animated using our Facial Animation Toolset (FAT), which is distributed as a plugin for Maya.
Frapper is also the central element in the communication with PsychoPy and the SMI eye tracker. The communication with PsychoPy is established via sockets, which allow the exchange of messages indicating Frapper when to generate the emotional facial animation of a given character with a given speed, intensity and level of abstraction (i.e. in the case a certain NPR style has been applied). The communication with the SMI eye tracker is established using the API provided that the eye tracker.
Downloads
Download SARA Frapper Deploy (510 MB).
The deploy contains: an example scene, the NPR-DECT, the i-DECT, and all characters - Nikita, Sara, Hank, Gunnar - separately.
Download Microsoft Visual C++ 2010 SP1 Redistributable Package (x64)
(necessary if Visual Studio is not installed in the system where Frapper will run)
Frapper Deploy Tutorials
The following tutorials intend to be a guideline for the installation and use of the SARA tests and Frapper.
(For a better resolution play the video in the Youtube player with a resolution of 1080p. This might take longer to load
PsychoPy
PsychoPy is an open-source application, written in Python, to allow the presentation of stimuli and collection of data for a wide range of neuroscience, psychology and psychophysics experiments. For this experiments we are using the 1.79.00 version.
PsychoPy is used by the University Medical Center Freiburg to design the DECT itself and store the results for subsequent statistical analysis. The protocol is that PsychoPy sends Frapper all the parameters require to generate the facial animations, and in turn Frapper sends back timing information and acknowledgement that a trial has been finished.
Download PsychoPy from their official website.
Eye Tracker
The SMI RED-250 eye tracker is the device used at the University Medical Center Freiburg for the experiments with children and adolescents with ASD and ADHD. The communication with Frapper is established through the SMI iView X™ API.
To this end, we have created in Frapper a plugin sMIEyeTracker that allows the initialization, calibration, tracking, writing of gaze data and creation of Areas of Interest (AOIs) using the eye tracker’s libraries. In order to use the sMIEyeTracker plugin (node) it is necessary to have the eye tracker and the corresponding SDK installed.
For more information about obtaining the eye tracker please visit the SMI official site. The partners of the SARA project are not responsible for providing neither the device nor the SMI libraries.