Each year, Animationsinstitut hosts the Set Extension Workshop (SEW). In this course, students of Filmakademie learn state-of-the-art methods to create visual effects in live-action films by developing and realizing a short film with a high proportion of VFX. Since 2020, high-resolution LED screen walls have also been used in this process. The screens enable a new, forward-looking shooting technique that is currently conquering the film and media industry. Set shots can thus be merged with computer-generated images (CGI) in real time. As a result, this offers impressive visual possibilities, as shown in the SEW 2021 short film AWAKENING.
“We are just experiencing the convergence of cinema and video games.”Théophile Farant, animation student and director of the 2021/2022 Set Extension Workshop project.
This is the storyline of the short film: A woman wakes up from sleep in a gigantic ice cave with a giant tree in the middle. She seems to be frozen, slowly moving her limbs. Confused, she looks around and carefully walks towards the tree. When she reaches and touches it, there’s suddenly a light in the background. At the place where it originates, she discovers a frozen body in the ice, with roots growing from its wrists, connected to the tree. The woman notices marks on her wrists, which are in the same place where the roots grow from the frozen body. She recoils. Was she also connected to the tree? As she begins to explore the tree insecurely, iridescent particles of light fall from it. Individually, they land on her marks, whereupon a dark mass oozes from them, gradually encircling her arms and forming a tattoo-like pattern on her body. Her fear seems to vanish, and she realizes the tree is well-disposed.
"The film is all about atmosphere and the emotional mood it creates," explains Théophile Farant. The animation student is the director of the short film project. The story could be read as a metaphor for the empowering connection between humankind and nature, but it can be interpreted in different ways, he says. "To develop the storyline, we formed three teams in the workshop, each of which came up with a storyline for the VFX film," the director continues. In the end, there was a vote to decide which story would be realized. "The script I worked on at first was not elected, but this made the challenge of directing even more exciting for me." Théo is actually studying at MoPa L'ecole internationale du cinéma d'animation in Arles, France. He is completing an exchange semester at Animationsinstitut in the winter term of 2021/22. His role as director of AWAKENING was also determined by the participants of the SEW in a vote.
The SEW is part of the curriculum of Filmakademie Baden-Württemberg. The participants of the SEW come from the study specializations Cinematography, Production Design as well as the programs animation and VFX of Animationsinstitut. The course is about learning the state-of-the-art techniques that can be used to combine animation and visual effects with live-action footage. In 2021, students were supervised and trained by animator and illustrator David Maas, technical artist Tonio Freitag, set designer Thomas Stammer and cinematographer Thomas Merker.
Since 2020, shooting with an LED wall has been part of the SEW. The wall has always been set up during the first half of the respective winter semester in one of the studios on the campus of Filmakademie. Visual effects in live-action films are increasingly being created with the help of such high-resolution LED screen walls. The new technology offers the possibility of shooting footage on film sets in real time with computer-generated imagery (CGI). The content on the screens serves as a virtual set that can be merged with the conventional set design in the camera’s image. One of the first major productions to make impressive use of the technology was the Disney series MANDOLORIAN. Nowadays, there are studios around the world that have set up LED screen sets, in Germany for example in Cologne, Berlin, Mannheim and Munich.
In recent years, Animationsinstitut of Filmakademie Baden-Württemberg contributed to the development of the cinematic use of LED screens with its research work. Its Research & Development department is regarded as one of the leading research institutions in the field of virtual production worldwide. As a result, Animationsinstitut also plays a leading role in the training of animation and VFX experts who can work and realize films with LED screens.
"The fact that we have the chance to shoot with the LED wall is a great opportunity," says Théo. "When we were shooting, it felt like we were really working with the newest thing possible in the industry. It's incredible that students at Animationsinstitut can try out these new tools and shoot with them." However, he says it was also particularly challenging. For him, working with the screen was something completely novel: "In a way, the technology is still innocent. As is so often the case with a new technology, it comes down to figuring out how to use it well," the director emphasizes. Artistically, he says, new technology can advance films on the one hand, but on the other hand, it can be a pain when technical limitations restrict the content. "I've only done 3D animation before, and there, you can basically do anything you imagine," Théo explains. "With AWAKENING, I already had to realize during the storyboard phase that a lot of my direction ideas were technically not possible."
Théo emphasizes, however, that, now, the general task is to find a good workflow with the LED screens. The shortcomings of the technology must be worked around in a certain way. For example, it is not possible to shoot long shots from a greater distance or to focus on the wall because of the moirée effect, which causes, that the light pattern of the LED screens can be seen in the camera shoots. “There were certain constraints in post-production or in the performance of the game engine that had to be kept in mind during filming,” Théo adds. "Basically, while shooting, we had to be careful not to end up with more work in post-production with the scenes than without the screens."
The VFX supervisors and 3D artists on the SEW team Benjamin Gätzschmann, Philipp Dörrer, Eileen Kammer, David Sick and Jan Klingner avoided potential problems by creating detailed pre-visualizations of the set and a mockup of the studio. This allowed them to plan the shooting sequences precisely. For instance, the team avoided shooting scenes with problematic camera angles accidentally. "Our shoot at the studio was also only limited to two days, so we had not much time to experiment, and so this planning helped us immensely," Théo explains. In addition, the team also worked with fully computer-generated images for many frames of the film. These eventually complemented the set shots, and in the end, the combination produced a visually stunning result.
In general, the director of AWAKENING is enthusiastic about LED screen shooting technology. "Especially for acting, the technology has enormous advantages. In front of a green screen, the actress would not have been able to play the scene that authentically," he enthuses. If the technical workflows can be adjusted and the imponderables can be solved – something that is being researched intensively at Animationsinstitut and to which the experiences of the SEW team further contribute – the LED wall technology holds great opportunities in his opinion. In the area of AR and VR alone, he said, the screens have great potential, also in telling stories. "We are just experiencing the convergence of cinema and video games," says Théo, "It's very exciting to be able to be so close to this new development here at Animationsinstitut."