Advanced Rendering For Augmented Reality Applications

Virtual Reality (VR) can be defined as the method to display synthetic images along with real ones, explains Miguel Churruca, Marketing and Communications Director, Brainstorm.

Some have started to talk about Mixed Reality, understood as the combination of virtual and real environments which, in essence, is yet another way to define Virtual Reality or Augmented Reality. Others use "Enhanced Reality" to define any of the above "realities", where real footage or live videos are mixed with virtual backgrounds or scenes, chroma keyed talents and additional broadcast graphics or data-driven objects.

But regardless how we call it, when facing an Augmented/Mixed Reality project, the integration between the real and virtual objects and environments becomes essential, so the next step in virtual set production and Augmented Reality applications is to increase the realism of the content. This not only involves high quality rendering, but also requires perfect integration between the different elements of the scene to provide a sense of realism, with precise perspective matching and correct broadcast continuity.

Photorealistic results can be achieved using advanced imaging and rendering techniques such as HDR (High Dynamic range), PBR (Physically Based Rendering) or using specialised render engines like Unreal Engine. Gaming engines achieve excellent rendering results for backgrounds and scenes, allowing for realistic results, but Augmented Reality broadcast applications also require the interaction between sets, talents and virtual objects, many of them created out of external data sources such as statistics, charts, bars, and many others. These allow for visually engaging representations of the data, which can then be better explained by the presenters placed on the set.

All Brainstorm products run on the eStudio render engine, considered the fastest in the industry. Additionally, Brainstorm supports gaming and architectural engines such as Unreal Engine. These highly complex engines provide realistic scenes with great possibilities, and played along with the eStudio render engine allows InfinitySet not only to show excellently rendered realistic background scenes, but also to integrate graphics elements in the final scene, like 3D motion graphics, lower-thirds, tickers, CG and many other elements. With the inclusion of these graphic elements, the scene can result in a highly complex composition, seamlessly integrating in real-time different render engines, virtual 3D backgrounds, real characters and synthetic graphics elements.

Brainstorm's approach to external render engine support is unique in the industry, as it allows control of Unreal engine's parameters from InfinitySet in real-time, and allows merging of both engines' render parameters and buffers to make them work as a single render engine (Combined Render Engine). This greatly simplifies the setup, as a single engine can handle all requirements and elements in the scene: chroma keyed talent, virtual set background, real footage and broadcast graphics. InfinitySet can control, from its own GUI, many Unreal Engine's Scenegraph parameters, including cameras, geometry or object visibility. This integration allows Unreal Engine's scene objects to be managed directly from InfinitySet, which can then add the chroma keyed talent and the data-driven graphics to complete the real-time output.

In addition, new InfinitySet features like VirtualGate allow for integrating the presenter not only in the virtual set but also inside additional content, so the talent in the virtual world to be "tele-transported" to any video with full broadcast continuity. This means that a presenter in the virtual set can walk into a virtual screen or into the featured news (which can be shown on a virtual videowall or similar) and become part of the video itself with full realism. The talent enters and exits the video with full precise and accurately matched perspective, and once inside the clip it behaves correctly in terms of spatial reference while adding true shadows, selective defocus and bokeh, etc.

VirtualGate extends the virtual scenario beyond the virtual set and creates an infinite world for the presenters to be in, allowing for better real-time content possibilities and interaction. So, the presenter on the set, for instance, can walk into a sports event with full broadcast continuity (without editing, cuts or multi-camera production), remotely walk on the field, explain whatever is required while being perfectly matched in the video regardless the movement difference between shots, and finally return to the set as if the camera was shooting a sequence shot.

www.brainstorm3d.com

This article is also available to read in the June edition of Broadcast Film & Video.

(JP/LM)