Join us at SIGGRAPH 2019 (July 28-31)

RYOT

  • Client Ryot
  • Year 2018

How Cubic Motion's facial capture tech elevates RYOT's immersive storytelling

RYOT sits at the forefront of immersive storytelling, creating interactive experiences that are timely, compelling, and truly unlike anything else out there. Initially focused on linear documentary films, including the Academy Award-nominated Body Team 12 and Watani: My Homeland, RYOT has since embraced virtual and augmented reality technology to tell stories in unexpected ways.

One of those AR projects, LA Louvre, transported the iconic sights of Paris’ Louvre Museum into a real warehouse in Los Angeles, allowing visitors to experience a truly unique experience via their smartphones. The project earned RYOT an Emmy nomination in the process.

As part of Verizon, RYOT also benefits from access to cutting-edge technology, and plans to utilize Verizon’s 5G network in a major way. High-speed cellular connectivity will be the key to delivering content very quickly after it is created and without the need for large downloads, streaming without delay directly to VR and AR devices for rapid, interactive consumption. RYOT is currently opening up a Los Angeles innovation lab to further that development.

Cubic Motion’s technology is key to realizing RYOT’s ambitious vision, as the company’s high-end facial capture solution allows partners to quickly capture lifelike animation data and seamlessly transfer it onto characters into a game engine. That allows RYOT’s artists and storytellers to retain photorealism despite rapid content turnarounds, and ensure that the company is poised to be a breakout creator on emerging platforms. RYOT is so confident in this new wave of facial animation technology that they will be licensing Cubic Motion product “Persona”, prior to its official release.

Speedy storytelling

“The mission here at RYOT is to couple thought-provoking and provocative storytelling—super long-form documentaries on the linear level—with what the lab team does, which is develop immersive platforms,” explains Adnan Ghani, director of animation capture.

“When we set out to create this studio, we wanted it to be the studio of the future,” he continues. “We open our doors to storytellers, filmmakers, and thought leaders in the immersive space, bringing in the best equipment and working with the best partners—all of which is going to be powered by Verizon’s 5G network.”

That incoming 5G network is exactly why Verizon owns a boundary-pushing content studio like RYOT. It’s not just about creating content that can be viewed on any device, but rather building immersive, one-of-a-kind experiences that are designed for virtual and augmented reality and empowered by the incredible, go-anywhere speed of a next-generation cellular network.

Delivering original content that is timely and built around the ongoing and ever-changing events of the world doesn’t require precognition (or even a very good guess)—it requires a talented team and top-of-the-line technology to enable rapid iteration, turnarounds, and delivery.

Cubic Motion sits on the production side of that equation. In under 200 milliseconds, its computer vision capture technology translates human facial expressions into lifelike animation for apps, games, and other experiences.

“How do we tell a story as fast as we need to, at a humanistic quality?” asks Ghani. “RYOT, Unreal Engine, Verizon, and Cubic Motion will all be at the crux of that, coming together to produce a high-quality and highly entertaining show.

“It will be up-to-date, relevant, and fresh. We don’t have to start six months ago. We can just take the news of that day, mocap it, animate it, and the next day it’s out into the ether,” Ghani explains. “This is a good example how we’re using performance capture and Cubic Motion to stay topical, with fast turnaround times, and the proof will be in the pudding once this is broadcast to a national audience.”

Vibrant, lifelike facial animation is key to RYOT’s work. Otherwise, users quickly become distracted by low-fidelity characters and focus on nitpicking the animation rather than being immersed in the story. Clunky character models simply aren’t an option for the immersive nature of RYOT’s content. The company needed a cutting-edge level of ultra-realism that only Cubic Motion’s technology can deliver.

“You can tell stories with Minecraft-like characters that are like 8-bit with no facial animation. Or you can tell stories with tech like Cubic Motion’s Siren, which is a whole new level of photorealism,” says Ghani. “It’s a creative style, but we want to the option to dial up or down the quality as needed. We want to have the ability to deliver on the highest possible performance, and then at the same time give our story-makers the tools they need to maneuver the story in the way they need to.”

Always cutting-edge

Using the Unreal Engine allows RYOT to achieve results at the speed they require, saving an incredible amount of time over classic animation methods.

“In a traditional animation house, every frame takes hours to render, but here it takes two milliseconds,” Ghani says. “Everything is all about quick, quick, quick. We do that so we can get ahead of the story, create new experiences, tinker with shots, and throw away things that don’t work. We never let a topic get cold.”

That’s why Cubic Motion’s role is absolutely critical. Animated characters may have hundreds of different control points, which an animator would otherwise need to individually manipulate to achieve the desired result. Instead, Cubic Motion’s facial capture technology and software allows RYOT to skip over that process entirely, instead importing the captured performance into Unreal Engine for use in the content.

RYOT has relied on Cubic Motion for its facial animation in almost every instance since adopting its technology, but there was one rare—and hilarious—exception.

“We were brainstorming ideas for a bald eagle, trying to design the creature digitally. What would its feathers look like in Unreal? And do we still use facial capture through Cubic Motion for a beak? Then we literally just brought in an eagle puppet, shot it with a regular video camera, and it was the perfect comedic element,” recalls Ghani. “The contrast between high-quality animation and an old school prop – it just worked. You’ve got to know when to go high-tech, and when to go low-tech.”

In addition to the quality of Cubic Motion’s technology, RYOT has been impressed with the team’s ongoing level of service and assistance.

“The engineers came in and were strictly here to make sure that the implementation is as easy as possible. We couldn’t ask for a better partner, because we don’t have the overhead to dedicate a team of engineers to figure this out,” says Ghani. “Cubic Motion is like a white glove service, if you will. They’re an amazing partner. Outside of their product being the talk of the gaming industry, the service that they provide us has been immensely helpful. Having them as a resource to help essentially guide our storytelling has been instrumental.”

It’s a relationship that RYOT sees growing and expanding over the years, and Ghani is confident that Cubic Motion’s technology and capabilities will continue to match the studio’s evolving ambitions and requirements over the years to come.

“What I’m most excited about is building an enduring partnership with Cubic Motion. There’s not just technology going back and forth, but a human relationship as well,” he says. “Two years from now, five years from now, or 10 years from now, I know that the team at Cubic Motion can step up to the plate for new technology needs.”

More Case Studies