Synthetic Characters and the Impact of them on the Filmmaking Process
By Nikolaos Manitaris, Head of 3D/ CGI, Burlington Books Greece & Cyprus
The purpose of this article is to provide a background vision of how the Synthetic Characters (made as Digital Doubles, Digital Agents, CFX—Creature Visual Effects, Deepfake technique, use of Artificial Intelligence) shaped the CGI/VFX industry and how they might influence the future of Filmmaking process.
Since the dawn of visual effects, the VFX/CGI industry has received many reboots and enhancements. Over the last decades, the industry’s main focus varied from the creation of enormous beasts to realistic portray of alien creatures resampling human beings.
James Cameron and his creation, Avatar, set new standards for the industry. With his team of experts, he managed to create high-quality alien-like creatures, skin shaders that realistically interact with light, and anatomical accuracy that resamples those of a human being.
The technical team at Weta Digital (now WetaFX), along with their highly skilled supervisors, even managed to create a proprietary tool called Tissue which allowed them to use x-rays for an accurate depiction of bone structure and how the bones interact with all the separate layers of muscles.
The same company years before was responsible for the creation of Gollum (Andy Serkis provided both the performance capture and the voice-over) for the needs of Lord of the Rings, which at that time was a filmmaking marvel. The ability to incorporate into live-action shots a digital creature that breaths, acts, behaves, and interacts with characters was beyond everyone’s expectations.
These achievements not only allowed the filmmaking process to evolve in a way that we couldn’t imagine before, but they also paved the way for what we experience today with this plethora of movies depicting many characters made up entirely by CGI.
With the creation of Gollum, it was obvious that we have moved into the era of VFX/CGI Filmmaking and the industry declared that there are new ways of expression. This not only facilitated the industry to tell new stories with a variety of visual content (beasts, fairies, monsters e.t.c) but also became match more visually engaging for the audience; hence the income was increased.
Moreover, the creation of a Synthetic Character allows the creators to take key decisions even before principal photography or even go back after shooting and reshoot a shot digitally or even add new shots that they were not planned for shooting or even use the digital double for safety purposes. Given the above, the advantages of using what is now called Synthetic Characters were astonishing and beyond any controversy.
This gave the green light for many studios to spend more money to research and produce new techniques for the creation of Synthetic Characters.
However, the need to decline the costs is always a subject of further investigation, which brings us to Artificial Intelligence.
Before moving to AI, here are some of the most recognizable movies that depict Synthetic Characters, some of them made up entirely with CGI, and others were a mixture of both makeup effects and CGI.
Lord of the Rings
Harry Potter
Avatar
The planet of the apes
Avengers: End Game
Despite the good will and the artistry of CGI artists, time is cost-effective, and the industry seeks new ways to produce many shots efficiently.
It does not matter to create a digital, realistic character if you cannot bring him to life for the requested shots, hence the creation of Synthetic Characters must fit in a production pipeline. For that, the number of CGI employees was drastically increased; you can simply check that by watching the credits of a movie.
This is where AI appears to step in. Soon there might be a possibility that will allow a studio to incorporate Synthetic Characters into shots automatically. All you have to do is just to build the character.
Deepfake looks promising. Recent results of artists incorporating a famous actor’s face, Tom Cruise, were beyond unreal. This raised concerns about the legitimacy of the videos that circulate on the web and the authenticity of information that the depicted person carries. This is also a subject for another analysis.
Deepfake appears to do pretty well what it claims to do, so that we can expect in the near future more.
Apart from Deepfake, Adapt’s proprietary Plato process of advanced machine learning and neural rendering sets new boundaries for dubbing and subtitles.
Plato by Adapt company has the ability to replace the original language performance of long feature film by maintaining the performance even with extensive use of makeup or under difficult light conditions. Moreover, the amazing aspect of Plato is that it can fit inside a production pipeline for the production of hundreds of shots. This gives a new meaning to the filmmaking process as it appears to facilitate the production of foreign movies for a wider audience. It appears that AI is the game changer in the industry of Filmmaking, and we can expect more in the coming years. Meanwhile, as technology evolves, it is expected that in the near future, more and more filmmakers will be able to engage a bigger audience and in the process, the diversity of storytelling will increase, concluding a fresh and a new industry.