Powering up performance capture with Synchronicity

When Side, Cubic Motion and 3Lateral teamed up to produce the recent Synchronicity demo, they had something special in mind.

There are many tech demos out there, but these often show conceptual glances deep into the games development crystal ball, or the likes of floating heads, showcasing a single specialist discipline in isolation, abstracted from its contribution within a wider game.

Synchronicity, meanwhile, is an attempt to demonstrate the cutting edge of production-ready facial animation and rendering, along with performance capture and game acting at its most nuanced and liberated. Watch the two-and-a-half-minute short, and it’s apparent that it is a work from the forefront of triple-A, but it is also a sample of what devs can implement in their games today, rather than in months’ or years’ time.

The emotionally charged, thematically dark composition is also a showcase of the increasing potential of – and necessity for – collaboration in the world of performance capture and delivery.

Recognising this, Side, 3Lateral and Cubic Motion decided it was time to make such a demo. The nature of their work on commercially released games is so often draped in embargoes, all three had become frustrated by an inability to show their creations until anything produced was already outdated.

“We’ve had it, as three outsourcing companies, where we’ve worked on various big projects, but been frustrated in the past by the fact that we can’t showcase anything we’ve worked on publicly until a point where everything had moved on,” explains Side MD Andy Emery.

“The very nature of our work means we’re very restricted on what we can do. Even if there’s a game with a making of video, our companies’ names typically wouldn’t be attached to what was being showcased.”

Subsequently, developers too suffer from not seeing the most contemporary production work themselves. Quite simply, Side, 3Lateral and Cubic Motion needed a way to show more teams what could be done using contemporary, practical techniques and tech.

CAPTURING THE IMAGINATION

And so it was they embarked on producing Synchronicity. Side handled the capture, voice recording, casting, writing and direction, head rigging specialist 3Lateral made the demo’s digital doubles, while Cubic Motion took charge of the animating itself, using its proprietary technology.

“We wanted to create something we could share, and that was completely in our control, where there was no sacrifice made to compression, or many of the other restrictions we sometimes face,” confirms Cubic Motion’s Simon Elms, who stands as chairman and CFO. “It’s a demonstration of what we can do if we work together closely, and goes a way to prove why we should.”

It’s certainly a striking piece of work, offering a performance by actress Lydia Leonard that would do much in a game to convey character, emotion and a sense of humanity. But what is important, says 3Lateral’s founder, owner and CEO Vladimir 3Lateral CEO Vladimir Mastilovic, is the way in which it is applicable to games development pipelines and processes.

“Compared to something like L.A. Noire the approach we used has many advantages,” he insists. “The technological approach of having 4D scanning output in games comes with huge drawbacks. The shoot on set restricts interactions between actors. Those actors usually have to sit in a room in isolation to deliver their performances, and they usually can’t even move too much, instead acting with restrictions. And the amount of data 4D scanning takes up inside the engine is enormous. It’s just not very practical and editors are not able to edit the performances or retarget it to make characters look different. And reuse of the animation is really impossible.”

The approach and pipeline used to make Synchronicity, starting with head-mounted cameras in the sound booth and concluding with CubicMotion’s work applying 3Lateral’s efforts to its rigs, is according to those behind it more efficient, flexible, and readily applicable to existing pipelines. It’s also argued to offer less time-consuming editing, considerably more practical retargeting, and let studios apply changes across a project.

“What we were able to do was take the inputs from Side, which are greatly advanced compared to just a few years ago, analyse the data – of which there’s a great deal more of today – and get more from it than ever before,” explains Elms of the process. “Then we worked very closely with Vladimir and his team in terms of getting the CG assets and translating that data.”

Synchronicity itself is in fact not a demo of a packaged service, or a set-in-stone pipeline; rather it is a taste of the potential of Side, Cubic Motion and 3Lateral working in unison. Each remains dedicated to providing its services separately, together or with other outfits, and the methods used to create Synchronicity are but one way the trio of specialists can work together.

LEARNING THROUGH COLABORATION

According to Emery, the approach particular to the case of Synchronicity was also one that allowed them to maximise each of their own disciplines. Side, for example, were able to give their actors far more freedom as a result of understanding the requirements of their partners more readily, and reap a great deal more from recording and capture sessions. In effect, producing Synchronicity served as a learning process for all involved.

“Thanks to the technology provided by Cubic Motion and 3Lateral, and the way we can work so closely together, we able to do far more with our actors. The three of us, by working closer together, have a greater understanding of both the creativity needed and the pipeline, as well as the implications of that pipeline.
Indeed, the same methods would have allowed for Side to start with full performance capture, work on a sound stage, or even using multiple actors in a large capture volume.

COME TOGETHER

Synchronicity additionally demonstrates something beyond technology and technique. It also sets an example of why collaboration matters; something already providing those involved with tangible benefits. Previously, the three outfits have worked on the same products, but only found out that is the case months after their work starts, as clients fail to realise the potential of connecting those providing performance and animation services. With Synchronicity setting an example of what unified collaboration can achieve, clients can better understand why connecting service providers on a project can yield superior results.

“This demo has already changed the way we speak to clients, because they are seeing Synchronicity, and they like what they see,” explains Mastilovic. “They ask us how we achieved it, and we can talk about working together, and so help them make the right decisions to get the best results. Of course, they have other considerations around the bigger picture, like budget and time, so there may always need to be compromises. But when they are deciding on the compromises, if we can give them all the information about the entire process, they can make the decisions in the right way.”

Working on the project, effectively positioning themselves as their own clients, Side, 3Lateral and Cubic motion have also learned much about how to better serve one another. They have, of course, previously been well aware of what each does, but are today feeling increasingly empowered by the fringe benefits of collaboration.
“We all got a greater understanding of all the processes involved by functioning as a single force, which has been great,” offers Emery. “As a result we’ve all learned more about one another’s disciplines. That’s hugely important in a meeting with clients.”

With Synchronicity complete and making its way to the monitors of ever more developers, it is back to business as usual for Side, 3Lateral and Cubic Motion. Each continues to work together – and separately – on a wealth of high-profile projects they are not at liberty to discuss. And each has been pushed by the experience, and is optimistic that they can deliver similar or better results if a client is added to the mix, even with the realities of deadlines and budgets remaining an important consideration.

Potentially, Synchronicity is a significant leap forward for each of the outfits, along with advancement of the disciplines they practice, and, ultimately, the games their future clients will produce. This is great news for both developers and players, because in the end, it is about making games more engaging through more believable characters.

You can watch Synchronicity at www.s3c.tv.