Semantic Animation describes using high level descriptors to direct the performances of special character assets which are capable of determining their own detailed low-level behaviours and scene interactions. The required semantic information can be created in a manual authoring process or by machine learning. It includes our work in both virtual production and physically based animation.
- To develop and demonstrate real-time control systems for authoring animated content using smart assets, automatically synthesising new scenes from existing ones and integrating smart assets into virtual production scenarios with editable cameras and lights
- To test the prototype technologies and tool-kits in a series of experimental productions and evaluate their performance in realistic contexts of professional use.
In this context Filmakademie has crafted and released the PHS Motion Library. It contains over an hour of bipedal reference motion capture data and videos.
Semantic Character Animation is realized with dedicated focus on virtual production scenarios. Filmakademie developed an open character streaming protocol for its VPET toolset. The entire character (including weights, skeleton etc.) can be transferred to the tablets at run time. The newly developed API then allows arbitrary external animation solving engines to animate the character through streamed bone animations.
This interface extends the open architecture of VPET. Although the client is based on Unity, any host (providing the scene to the VPET clients) could be connected. We have demonstrated this with Foundry Katana in the past. The latest addition allows to connect an arbitrary animation engine.
Semantic Scene Understanding allows to extract labels for (SMART) assets of a scene by the means of machine learning. They can be used to prepare the assets for their dedicated production scenario or to dynamically create character animations.
Filmakademie released the Love & 50 Megatons - Miniature City Assets. This 2020 VES nominated production made heavy use of a virtual production scenario and scanned miniature assets.