Computer Graphics World

April-May-June 2021

Issue link:

Contents of this Issue


Page 5 of 35

4 cgw a p r i l • m ay • j u n e 2 0 2 1 V I E W P O I N T T he rise of machine learning and AI are taking graphics and cre- ativity to a new level, helping transform an industry that relies on convincing an audience that what they're seeing on screen is as real – or as close as it gets – to what they see with their eyes. Content creators and studios are using machine learning and AI to complete repetitive tasks faster and allow for more creative iteration to perfect the finished product. They can also speed time from the set to screen, and increase production for in-home entertainment subscription services. Instead of coding soware with specific instructions to complete a task, a machine is "trained" using large amounts of data, giving it the ability to learn how to perform the task. With machine learning, pat- terns can be found from any kind of data, from images to numbers. Relieving pressure on the technical workflow frees actors to work on their performances, and editors to focus more on other areas. The Fountain of Youth and More De-aging is just the tip of what machine learning can do. With AI-powered denoising, creators can remove grains from their renders and create noise-free, interactive images in real time. Users get immediate visual feedback, so they can see and in- teract with their latest designs and even experiment with different elements such as materials, textures, lights, and shadows before finalizing a scene. Creators can also remaster content using AI-enhanced solutions. Integrating GPU-accelerated machine learning into super-resolu- tion techniques allows studios to achieve higher visual fidelity and increased productivity. Nvidia Deep Learning Super Sampling (DLSS) opens doors for real-time productions in virtual production. With DLSS, studios and artists can turn low-resolution rendered images into high-resolution ones. The technology helps accelerate workflows in animation and virtual production. By adding DLSS to creative pipelines, studios also are enhancing rendering performance and image quality with real-time ray tracing. Strike a Pose Pose estimation technology is making motion capture much easier. Previously, studios would need multiple cameras, bodysuits with sensors, and calibrations to capture an actor's body movements and produce 3D re-creations. With machine learning and AI, pose estimation technology detects and matches a person's movements from a single video feed, without the extra bodysuits and cameras. Pose estimation soware has led to advanced breakthroughs like Omniverse Audio2Face, an AI-powered application that generates facial animations from audio sources. Based on Nvidia research, Audio2Face takes an animation of a 3D character and matches it to any voice-over track. It works by feeding the audio to a deep neural network, and the output of the network creates the facial animation in real time. Breakthrough technologies like what is in Audio2Face are paving the way for new forms of interaction between humans and computers, with more realistic conversations via digital avatars. The Cutting Edge is Going Mainstream Today, more studios are accelerating creative workflows by using AI to enhance storytelling. Visual effects are being powered by AI, creating villains like Thanos in Disney's The Avengers, to the de-aging of actors in The Irishman. Machine learning makes the creation of digital humans possible. Award-winning VFX studio Digital Domain has worked on characters and visual effects for blockbuster movies, but they are also known for DigiDoug, the first digital human to give a TED talk in real time. Digital Domain's digital human technology is driven by inertial mo- tion capture and a single camera capture for facial animation. Then machine learning is employed to capture and showcase emotions in real time. The team starts the process by taking thousands of images of a person's face, using different angles and lighting to capture as much data as possible. Then deep neural networks put the data puzzle pieces together, outputting in a virtual human that acts like a real person. Machine learning is also beginning to play a part in new features from streaming services. For example, these services are using AI to power recommendation engines that provide audiences with per- sonalized content based on a user's viewing history. Recommenda- tion engines are possible with technology like Nvidia Jarvis, which is an application framework for conversational AI services, which could allow you to simply ask the service for a show you might like. Com- panies also use AI to optimize streaming quality at lower bandwidths BY RICHARD KERRIS MACHINE LEARNING: TRANSFORMING THE FILM AND TV INDUSTRIES AI-powered VFX helped ILM de-age De Niro in The Irishman. Image courtesy ILM.

Articles in this issue

Archives of this issue

view archives of Computer Graphics World - April-May-June 2021