EINSTEIN – Digital Actor

The EINSTEIN (2017) project of Animationsinstiut’s Research Lab is an impressive achievement! After all, this was a project that set out to explore the possibilities and boundaries of digital actors created entirely through art. No easy thing to do. The EINSTEIN project resonates quite strongly with me as I was also involved in a project with the Research Lab around 15 years ago, exploring the feasibility of making digital humans believable.

“It should not be done.”—This is something we were told around the time we began our project as we were trying to clear the rights to the footage we wanted to use for our endeavour. To be perfectly frank, this statement kept ringing in my ear all the way, and I could never quite shake it. After all, when attempting to create a deceased person there are always moral implications. However, we found ourselves way past that crossroad and there was no turning back — the funding and the team were secured, the wheels were in motion, no second thoughts.

Not even with such loaded and burdensome words coming from someone who shared such a significant legacy with the actor we had planned to portray. Whilst we were determined to proceed with the utmost respect to the actor’s legacy, his family and his estate, this sentence lingered in the back of my mind, causing nasty little shivers of doubt. As with everything we create, we did not want to disappoint—not just ourselves, but more importantly, the sceptics. Maybe, hopefully, we would prove, to anyone who had doubts regarding the practical and moral feasibility, that this could be done with talent and respect. Still — and to be even more frank — at a certain point early on, I was convinced that we were doomed.

I do hope that the people involved in these imaginative and fresh creations featuring a digital Einstein were not burdened with similar doubts. One thing has certainly been true for both projects, and I believe it still holds true to this day—digital humans, despite all the advancements made, continue to be very difficult to achieve and to deceive with. The thing is, we know everything about our fellow humans. Their joys and fears, their truths and deceptions, all are laid out bare in our counterpart’s faces. Millions of years of evolutionary programming have made sure we can read each other. Overcoming this familiarity, this invisible and indescribable hurdle — that is the inherent difficulty when creating digital humans.

When I said that I thought our project back then was doomed, I was actually jesting in part. However, the joke kept reverberating in my head at the time, I think to allow myself to create a space in which making all of this seemed less serious and less likely to fail. At that time around 2006 or 2007, there weren’t that many great examples of digital humans that would pass the litmus test and that would overcome the uncanny valley, which seems deep, dark and insurmountable when you’re standing at its precipice.

There was THE SCORPION KING, the MATRIX digi doubles, Lemony Snicket’s SUNNY BAUDELAIRE — a few less or more successful examples done with the backing of far more expertise and money than the Research Labs projects had. Mind you, THE CURIOUS CASE OF BENJAMIN BUTTON was still a year out. Maybe, unknowingly, when trying to create digital humans, we do set ourselves up for a speck of failure? But then again, the traps we lay for ourselves never seem to have stopped anyone from anything ever.

Arguably, digital humans have always been, and still are, the holy grail of visual effects. Digital humans are notoriously hard to create and seemingly more so to make convincing. Depending on what you try to achieve, you can get away with it by doing certain things like keeping your digital actors schtum, or keeping shots short and cameras busy, all to avoid falling into the trap of giving viewers a chance to dissect the creation for too long. The EINSTEIN shorts resort to none of that, as the team chose a certain path and did not allow themselves to use any of these tricks. Just like in the project I was involved in a long time ago, they chose a path known to them and that matched the collective skill set, yet still pushed the boundaries of what is known.

When opting for a fully artistic approach, the task at hand is to test the limits of what is possible with the skills and tools you have available. EINSTEIN also used the Research Project’s own development, the Adaptable Facial Setup, to drive the facial animation. Both projects started off with a life-size silicone bust, a literal silicone bust that would inevitably turn into a figurative one. Neither project, in essence, was really concerned with performance capture or with recreating a likeness based on measurements taken. They are both about exploring what is possible from an artist’s perspective without constraints imposed by the real world. After all, a narrative was invented here, an impression of what Einstein might look like aged 80 or 90. All of it was made up! What needed to be invented is Einstein known, yet unknown. The EINSTEIN team dared to dream up an expression of Albert and achieved something I’ll happily watch time and time again—and hopefully, any sceptics might agree.

Now, 15 years on from the project I was involved in, the playing field has changed. With the advent of Machine Learning in visual effects, the promise of overcoming the uncanny valley is greater than ever and the barriers seemingly lower. What was previously very difficult to achieve, namely the creation of a believable digital human, is now more easily achievable. In my career so far, I have seen deepfakes applied, with the faces of actors superimposed onto stunt doubles, including a level of performance, and I have been stunned by the quality and level of the deception. However, it is still hard work and requires skill, passion and determination, although I am uncertain how much fun creating all of that is these days.

Machine Learning is something of a black box and requires a lot of training of, and supervised learning with, the model. The creative tools at our disposal have been changing, and this has allowed us to inch closer to unprecedented believability of digital humans. In years to come fewer people will remember how hard it was in the beginning to fool our fellow (real) humans.

The Animationsinstitut is a wondrous place. The fact that both of these projects took off there is a testament to the curiosity, the sheer fearlessness this place induces. I am immensely grateful for having had the opportunity to spend time there going through the Technical Director study course, participating in the projects of fellow students, deep diving into each other’s minds and craft, honing valuable skills and knowledge. The creative opportunities this place allows to unfold, the friendships that are cemented there through collective leaps of faith, hard work and determination, this stuff will stay with you for life.

I believe that EINSTEIN is an expression of that. Albeit not a student project per se, I believe it was done in a similar spirit, in what are pure and unadulterated possibilities. At the end of it all, my time there would endow me with the beginnings of things, things needed to start and pursue my journey through the visual effects industry—a journey that is ongoing and for which my passion is undimmed.

Find all EINSTEIN-episodes here.