“It was important for us to make every movement and reaction as realistic as possible, to create an immersive experience where players truly feel the weight of their decisions and connect to the characters,” said Will Byles, game director for The Quarry. “We wanted to get as close to the likeness of the actors as possible, both in terms of the super high detail scans and the subtleties and nuances of their performances. With its history of award-winning VFX and pioneering digital human work, Digital Domain was the best choice for this.”

Expanding on the idea of The Quarry as an interactive cinematic experience, there is even a “Movie Mode” option that allows users to tweak the personality traits and actions of each character, and then sit back and watch it all play out. Additional filters can further enhance the experience by adding an 8mm-style “Indie Horror” film grain, a retro VHS “80s Horror” look and a black-and-white Classic Horror filter.

Building a Better Slasher

To help bring photorealistic facial animation to a new level, the VFX studio famous for its work on Spider-Man: No Way Home, Stranger Things, WandaVision and countless others, began by conducting a series of facial scans of every cast member. That gave the team at Digital Domain an accurate template to work from, as well as a comprehensive library of facial shapes and expressions for each performer. The data was then sent on to the artists at Supermassive Games, who created the looks for the in-game characters.

Next, it was time for the performances. Each cast member reported to Digital Domain’s performance capture stage in LA, where they slipped on a full mocap suit and a facial capture rig. They then performed their roles individually or in small groups (in adherence with COVID safety protocols), acting out several versions of the story to create multiple branching narratives – including dozens of death scenes for each character.

Game developers are no stranger to using mocap and facial rigs to help create digital characters, but with help from Digital Domain, The Quarry went several steps further than most. In total, the cast underwent an incredible 42 days of motion capture, resulting in 32 hours of captured footage.

From Thanos to The Quarry

With the digital characters from Supermassive Games ready and the live-action performances recorded, the combined data was then fed through Digital Domain’s Masquerade facial capture system – a tool initially developed for Thanos in Avengers: Infinity War. Since its critically acclaimed debut, the system has evolved for use in projects beyond just feature films.

Masquerade 2.0 can analyze live-action footage, combine that with the individual and unique library of that performer’s facial shapes and then blend it all together with the CG character. Utilizing machine learning, it can then create a photorealistic CG version of the performance and fully animate it, all in record time. For The Quarry, that ensured that each performance seen in the game includes every facial movement, every emotive reaction and every bit of nuance from the talented cast.

“Masquerade is an incredibly powerful system, but ultimately it’s the performances that really make The Quarry stand out,” said Paul “Pizza” Pianezza, Digital Domain senior producer. “We wanted to ensure that our tools amplified what the cast brought, so each performance down to the subtlest movement is as true to the live-action performance as possible.”

But for a game like The Quarry that is built around interactivity, simply having film-quality digital characters wasn’t enough. It needed to be able to edit those digital characters in real time, and have those edited results be instantly ready to go without the need for additional touch ups from artists. The only problem was that the tools Digital Domain needed didn’t exist… yet.

Solving Common Problems

In order to create digital characters that could be edited in real time, Digital Domain first needed to address two common challenges with facial capture that typically require post-capture clean up by artists: eye tracking and helmet stabilization. First, Digital Domain started with an open-source eye-tracking solution known as “Gaze ML,” and heavily modified it over the course of three years. By using machine learning to identify the unique nature of each cast member’s eyes – and pairing it with a high frame rate head-mounted camera – Gaze ML was able to produce better accuracy in tracking, and improve the look of the digital eyes.

Next up, Digital Domain created a new way to stabilize the data from the headcam. Facial-tracking solutions typically require the wearer to remain relatively immobile to avoid shaking and blurring. But using a new, proprietary technique developed in-house, machine learning algorithms in Masquerade 2.0 were able to analyze the capture footage and compensate for any jostling. That gave the cast the freedom to move around normally and unrestricted, even run and jump while wearing a head rig to elevate their performances.

Introducing Chatterbox

Tying it all together, Digital Domain went one step further and introduced a new technology called “Chatterbox,” a tool initially developed by its internal Digital Human Group to help lay the groundwork for the future of autonomous humans. Powered by machine learning algorithms, Chatterbox can analyze live-action facial expressions unique to each performer, then determine the best possible options to seamlessly alter facial expressions, while still maintaining quality and lifelike movements. By streamlining the amount of data and focusing on only the best facial options, the digital characters can then be uploaded and edited directly in a game engine.

When paired with Masquerade 2.0 for The Quarry, Chatterbox was able to access the library of facial expressions from each cast member, opening up a host of possibilities for Supermassive Games. Once the data was running in Unreal Engine, the devs were able to see the characters in polished digital environments and make real-time adjustments to the performances as needed, all based on each performer’s actual unique facial movements and mannerisms. That eliminated the need for costly reshoots, while still ensuring the most accurate and faithful performances possible.

Overall, less than 0.5% – only 27 out of a staggering 4,500 shots deemed game-quality – required minor post-production alterations by an artist. In total, Digital Domain rendered over 250 million frames for The Quarry, helping to create a new highwater mark for high-fidelity, interactive cinematics.

“Creating high-quality, photorealistic visual effects used to be unique to feature films, but the technology has evolved, and given our decades of history and ongoing success, we are in a unique position to bring professional-quality VFX to any screen,” said John Fragomeni, global president of Digital Domain. “We have some of the best artists in the world, so there’s really no limit to where and how we employ our tools, and we were honored to work with Supermassive Games to help bring The Quarry to market.”

The Quarry will be available on June 10 in both physical and digital formats for PlayStation 4, PlayStation 5, Xbox One, Xbox Series X|S and digitally on Windows PC via Steam.