Kim Libreri, an award-winning visual effects artist based in Northern California, has worked on movies including Artificial Intelligence and War of the Planet of the Apes.
For nine years he has been working with a piece of technology better known for computer games, in particular the smash-hit Fortnite.
The Unreal Engine, owned by Epic Games, provides the building blocks and tools that a computer game developer needs, but is increasingly an attractive technology for TV and film producers.
The latest version of technology, Unreal Engine 5, is coming out next year, and Epic has been heavily trailing its features.
It should allow visual effects artists like Mr Libreri to slot graphics and images straight into a scene, with little fuss.
“With traditional filmmaking, a director and cinematographer might shoot a scene on set -then down the line, hand footage and creative direction off to a team of virtual reality artists and designers, who enhance that material with visual effects and computer-generated imagery in a separate phase of production, Mr Libreri says.
Directors can play with lighting and other effects inside a games engine
With Unreal Engine collaboration between the director, cinematographer, production designer and virtual reality teams can occur simultaneously as an interactive process on set.
“Unreal Engine 5 promises to further free the artistic process by making it easier to take virtual worlds developed for feature film and television, and run them in the game engine in real time,” says Mr Libreri.
Anthony Hunt, is the chief executive of Cinesite, a multinational digital entertainment group, whose visual effects film work includes blockbusters such as Avengers: Endgame, Independence Day: Resurgence, Iron Man 3 and more.
His company recently used Unreal Engine on a live stunt show for a theme park, which includes chase scenes, punishing fistfights, and death-defying leaps.
“It all happens right in front of you with live performers, interactive props and an immense LED screen – making it impossible to determine where the live-action ends and the screen begins,” says Mr Hunt.
The film industry has long borrowed tech from the computer games industry.
Games production processes like mapping out scenes in 3D through photography, storyboards and animatics run on similar systems to those used to build games.
But the challenges of producing films and television under lockdown restrictions is likely to accelerate the use of games technology.
Dodger Stadium was recreated for Rocketman
For example game engines are particularly good for generating big crowd scenes.
“An extra has to be fed and clothed and housed on a film set. We can replicate huge crowd scenes with computer technology now and just have our main actors in the foreground,” says Mr Hunt.
A good example of this a scene in Rocketman, the musical fantasy about Elton John’s breakthrough years.
One scene was set in Dodger Stadium, but in reality actor Taron Egerton performed the scene at Shepperton Studios. The crowds used were 3D graphics designed to be parts of a larger scene.
Computer-generated scenery can be combined with LED walls – giant screens made up of individual displays.
“LED walls are becoming more popular in film and television production because they allow filmmakers to capture visual effects in camera, and manipulate digital objects in a scene in real time,” says Mr Libreri.
“LED panels as those we’ve seen with Disney’s The Mandalorian (a science fiction TV series and part of the Star Wars family) will be a huge benefit to on-set production once restrictions start to lift, as filmmakers can reduce travel by creating photorealistic digital backgrounds to mimic any location,” he continues.
Disney’s Mandalorian used a games engine and LED walls
“When the physical filming restrictions are lifted, directors will have a clear plan of what and how to shoot their film/TV series because they will have done so in the computer already,” says Mr Hunt.
What will all this mean to the actors, crew and support staff who work in the industry?
“There is a possibility that production crew needs will alter somewhat. Some roles may change with the ongoing use of CGI, perhaps other roles will be created,” Daniel Green, director of the Master of Entertainment Industry Management Program at Carnegie Mellon University says.
When production returns to normal, the industry will need to be unified and make changes to assure that safety is paramount says Dr Green.
Studio executives and producers will no longer be making decisions solely based on artistic vision and production costs.
“They will now be in close consultation with public health officials to make sure that crews, actors, and staff can have a work environment that is safe and secure,” he says.
Mr Green also points out that CGI creates realistic worlds which are fun, but good story-telling is still the key.
“The use of CGI and virtual production as in The Lion King remake can augment what can be shown on the screen, but at the end of the day, audiences are still going to want to be entertained by a story that captivates them.”
James Dean on the set of Giant
Travis Cloyd, chief technology officer of CMG Worldwide, a company that represents 1,700 celebrities, athletes, musicians, brands and historical figures, sees another use for games engines.
A great business opportunity lies in “resurrecting” historical figures and famous people of the past like James Dean or Rosa Parks via digital twins, according to Mr Cloyd.
“These famous people have a prebuilt awareness,” he says. “They will always resonate with the audiences and the expansion of new platforms amidst the pandemic will bring new opportunities to use them in a plethora of media,” Mr Cloyd says.
Technology like the Unreal Engine is likely to be used in that process.
Movie making is of course a business, so the hope will be that new technology will save time and money.
“Fewer things will need to be worked out on set and in post-production, and there will be less need for travel and expensive location shoots,” says Mr Libreri.
“Once people experience this new way of working,” Mr Libreri continues, “they will find that it is much more efficient and cost-effective”.
“It is safe to say that virtual production is here to stay,” he says.