The LED Screen Revolution
The latest filming methods with LED screens create virtual film sets of unprecedented quality and could fundamentally change the film industry. Starting in October, researchers and students will be experimenting with a test system at Animationsinstitut.
Up to now only a few productions use high-resolution LED screens on their film sets. One of the first was the Disney series Mandalorian. The impressive images created by the VFX company Industrial Light & Magic already show how groundbreaking this technology could be for the film industry.
"With large display technologies, virtual content can be brought directly onto the film set," explains Simon Spielmann. The mathematician and media computer scientist is a senior engineer in the research and development department of Animationsinstitut and a lecturer for Technical Directing. For years he has been researching, among other things, how to make developments in real-time computer graphics usable for the film industry.
So far, scenes playing in front of computer-generated landscapes or photographic backgrounds have been shot mainly with blue and green screen technology. In this process, people or objects are filmed in front of blue or green surfaces and later placed in the desired virtual worlds. In the future, high-resolution LED walls could replace this method.
More precision and flexibility in filming
"Working with LED screens on set, has numerous advantages over previous methods," says Spielmann. One major disadvantage of green and blue screen technology, he says, is that the detailed design of the scenes regarding the match between the play and the visual environment is done retrospectively. This often presents an acting challenge, because the actors are forced to project their acting almost blindly or only indirectly into the environment. Shooting mistakes, Recording errors, which have to be corrected in post-production, also happen quite often.
The LED screens, on the other hand, function as a kind of stage. The scenery seen with the naked eye on set is thus close to the visual shot for the film. "This gives you more control and greater accuracy during shooting," explains Spielmann. "In addition, the screens are now so bright that you can use them to illuminate as well". So well, in fact, that LED walls could soon supplement the lighting technology on the set.
Workshops for students in Studio 1
For the technology to establish itself outside of the budget-strong productions in Hollywood - only they can afford the experiments with it so far - technical details still need to be improved and meaningful workflows for filming established. For this reason, Animationsinsitut’s research and development department is setting up a test set for four weeks starting in October in Studio 1 on the Filmakademie campus. A large LED wall measuring four by seven meters and three smaller units for lighting will then be available to the researchers and students of Animationsinstitut. It was planned to set up the system as early as April, but the corona crisis meant that the construction had to be postponed until the winter semester. In addition to extensive test series, there will then be workshops for the students with a limited number of participants so that they can get to know the technology and try it out for their own projects.
"We still have a whole range of problems with the technology that need to be solved," stresses the chief engineer. His department is, for instance, working on software solutions, that can simplify the work of director. If, for example, they want to make changes to the picture on set, the projections could easily be redesigned using tablet applications like the ones developed by Animationsinstitut (VPET).
Research expertise in virtual production has a long tradition at Animationsinstitut
It is also important to find suitable ways for matching the screens with the camera to produce the best possible final image. For a long time now, attempts have been made to technically synchronize computer-animated image worlds with real-life actions in a way that the spectacle and the image interact live, for example in terms of optical perspectives. A camera tracking system that spatially synchronizes the shot with the generated background is crucial here.
"The difficulty here is always the enormous computing power required for each individual image, which can only be achieved with great effort," Spielmann notes. In this field, many improvements were realized in recent years. Large productions, such as Steven Spielberg's "AI" or James Cameron's "Avatar" achieved pioneering work in this field. Animationsinstitut has also been researching this topic for a long time. In 2009, animation student Johannes Appelt shot his graduation film Motherland entirely in a computer-generated world with so-called On-Set VFX supervision and direction.
An important factor in the progress of this technology are developments in the field of real-time computer graphics for computer games. Virtual film productions also work with game engines, i.e. software systems that provide the visual representation of the game flow in computer games. Game engines are becoming more and more powerful, making the real-time projection of computer graphics on the LED screens of film sets possible.
LED screen technology gets a boost from corona crisis
Although it is not yet fully developed, the new filming technique is receiving a lot of attention in the film industry due to the corona crisis. At the moment, due to hygiene regulations, filming is only possible to a very limited extent all over the world. Since screen film sets can be constructed in basically the same ways in different studios, it is possible that filming could take place at different locations, thus avoiding or limiting travel by film crews.
"After all, this technology could also open up a whole new field of business," explains Spielmann. "It is imaginable that lighting settings and VFX set designs will be offered as ready-made presets by service providers, because they can be digitally uploaded to any LED screen studio in the world.”
Shooting in the LED Screen Studio
Research on LED technology, as a side effect of the corona crisis, has also intensified at Animationsinstitut. The Mannheim-based company Rent Event Tec, from which Animationsinstitut rents the LED screens for Studio 1 and who is normally renting screens for events, set up an LED screen studio in its warehouse. Rent Event Tec invited Animationsinstitut to conduct its tests on the technology in Mannheim as well. In August even the diploma project Neoshin, a dystopian music animation film, was able to record scenes using the new technology in Mannheim. The first shoots and tests already show that we can expect some breathtaking VFX projects of new quality by Animationsinstitut soon.