Research, Innovation, and the Endless Possibilities of Virtual Production
How Technical Directing students at the Animationinstitut contribute to the latest developments in film and media production.
Technical directors are responsible for the technical equipment in film productions. Their main task is to provide the best possible technological infrastructure for the realization of cinematic ideas. In other words, they build the bridge between art and technology.
The academic field of Technical Directing is concerned with improving the production tools and methods by which films and other media products are created. The focus is on the latest technological developments that can be harnessed in the media and entertainment industry.
This applies to numerous areas and fields of application. The film and media world is considered to be extremely innovative. It was one of the early pioneers during the digitization. For decades, almost all productions use complex software and hardware systems.
LED Wall Workshop 2021
Combining Game and Film: The Virtual Production Revolution
Technical directors are also fulfilling a vital role in the innovation process that is currently taking place in the film and media sector and is expanding cinematic possibilities in a remarkable way. Virtual production is bringing about one of the most profound changes ever seen in the production of film and video.
In a nutshell, virtual productions merge real footage with digitally generated images in real time. Previously, the combination of digital imagery and real actors was only possible via complex post-production processes. In virtual production, all of this happens directly during the production itself.
This is made possible by virtual sets that are displayed on high resolution LED walls, and by advances in real time (gaming) technology. The so-called “game engines” are nowadays so powerful that they enable photorealistic real-time rendering when using the LED walls. The combination of both worlds is done with the help of (volumetric) camera and tracking techniques.
From a creative point of view, virtual production changes production processes in a serious way. Among other things, the director has better control over the cinematic world. Further, production costs are reduced due to the less complex rendering process in addition to small travel expenses as many shots can be realized in one studio. In a recently published paper, the Filmakademie demonstrated that virtual production can make a significant contribution to reducing energy demands in film productions with VFX components.
It is also boosting the usage of extended reality (XR) applications (read more about LED technology here).
Virtual Production Expertise at the Animationsinstitut
These days, working with LED screen studios is no longer a distant dream. All over the world, there are more and more production facilities specializing in virtual production. Numerous film and advertising projects have already been shot with them.
At the same time, there is a growing demand in the industry for technical directors who have specialized in virtual production. The Animationsinstitut has been training filmmakers in Technical Directing for years. The study program is closely linked to the institute's Research & Development (R&D) department. The R&D of the Animationsinstitut is considered one of the most important research facilities for the film and media field in Europe.
The R&D has contributed to the technological breakthrough of virtual production in a meaningful way. It has undertaken important research, for example, in the development of open-source software for virtual productions, or, in the optimization of real-time pipelines.
Participation in Groundbreaking Research Collaborations
As a result, the Animationsinstitut has become a center for research and development of virtual production technology. This is illustrated, among other things, by the R&D's participation in an E.U.-funded research and innovation program.
The program is called MAX-R (Mixed Augmented and eXtended Reality Media Pipeline) and aims to "define, develop, and demonstrate a complete pipeline of tools to create, process, and deliver XR content at maximum quality in real time."
Within MAX-R, the Animationsinstitut is collaborating with an illustrious group of companies and other research institutions. Participating film schools include the Filmakademie Baden-Württemberg, Pompeu Fabra University (Barcelona), which is coordinating the project, and Hasselt University (Belgium).
Furthermore, seven European media companies are involved in MAX-R: film equipment supplier Arri, which already operates several mixed reality studios, the BBC (British Broadcasting Cooperation), which wants to expand its virtual studio and metaverse capabilities, provider of virtual set solutions Brainstorm, performing arts pioneers CREW, experts of live visuals Disguise, post-production specialist FilmLight, software company Foundry (Nuke), and metaverse world professionals Improbable. (Read more about MAX-R here)
The companies and universities involved in MAX-R will undoubtedly have a decisive impact on the future of the media and entertainment industry in Europe.
The TD students will be able to actively participate in the program's projects by aligning their own research activities accordingly
Prof. Volker Helzle, Head of Research & DevelopmentGreat Career Prospects for TDs with Know-How in Virtual Production
Senior engineer of the R&D, Simon Spielmann, emphasizes that the Animationsinstitut is at the heart of developing industry standards by sharing expertise with such accomplished partners. "In the MAX-R project, we can contribute with our great amount of experience in virtual production, gained through research projects such as SAUCE or Dreamspace, to support the advancement of the technology. Our research on AI-based machine learning tools for character animation also adds to this," Spielmann says.
Students from the Technical Directing course will benefit from the Animationsinstitut’s involvement in MAX-R in particular. "The TD students will be able to actively participate in the program's projects by aligning their own research activities accordingly," emphasizes head of R&D Volker Helzle.
Click on the Images!
For Spielmann and Helzle, studying TD and becoming an expert in virtual production is currently one of the most promising career options one can take in the film business; the industry is looking for professionals in this field worldwide at the moment.
Evidence for this are the students from the Technical Directing program with expertise in virtual production who are already successfully working in the industry during their mobility years.
One example for this is the student Vincent Maurer, who first worked at the studio of Hyperbowl and later as a real time TD at Industrial Light & Magic, the famous visual effects and virtual production company.
"The exchange of know-how with influential partners within MAX-R also means that the Animationsinstitut can connect its students with industry players, opening up interesting career paths for them," Spielmann notes.
Training at the Filmakademie with State-of-the-Art Technology
Through the R&D's research activities, the possibilities of virtual production also found their way into the curriculum of the Animationsinstitut early on. For example, a LED wall studio has been available for students since 2020 as part of the Set Extension Workshop (SEW). (Read more about the SEW here)
This virtual production environment is being used more and more in student projects at the Filmakademie. In recent years, it has been utilized to create high-profile productions, such as the pilot of the series concept CYBERCITY LOVESTORY, which innovatively pushes the boundaries between film and gaming.
This also benefits Technical Directing students because they gain more experience in dealing with virtual production. In addition to their research work, they are responsible for the technical implementation of several productions within project teams during their studies.
"Students from all disciplines at the Filmakademie are very interested in virtual production and increasingly want to use it in their projects," Spielmann notes. He interprets this as another sign that these technologies are becoming more and more prevalent in the film industry.
Therefore, the R&D also ensures that all students at the Filmakademie are increasingly coming into contact with the technology. For example, at the Filmakademie XR Day, the R&D introduces the possibilities of virtual productions to the entire university along with other technologies.
Students from all disciplines at the Filmakademie are very interested in virtual production and increasingly want to use it in their projects
Simon Spielmann, Senior Engineer R&DIn addition, the R&D is collaborating with the Academy of Performing Arts Baden-Wuerttemberg, which plans to introduce its acting students to virtual production environments. Showtime!
You want to help shape the future of filmmaking and media production? Read more about the Technical Directing program at the Animationsinstitut here.
