New Immersive Audio Concepts

Anyone with even half an eye on today's audio press has surely noticed the buzz around immersive experiences, says Jon Schorah, Founder and Creative Director of NUGEN Audio.

While it's easy to assume that most of this is to do with VR and computer gaming, there's also a revolution underway in movie theatres that could come to redefine the way we think about audio production for both film and television.

Since the invention of sound recording, audio has been largely a channel-based concept. As audience demand for greater immersion has increased, so have the number of channels in movie theatres, from stereo to 5.1 and now 7.1. More recently, theatres have started adding a vertical element as a height layer with 9.1 (and greater) channel counts.

Increasing the audio resolution allows the sound engineer to effectively design sound in 3D and employ more addressable speakers for specific localisation and realism. In a traditional sound mixing environment, this means lots more channels – lots more. The problem with this approach is that as channel counts increase, so does the complexity, leading to an unwieldy mix that does not translate well to different system configurations. Whilst this is manageable for a 5.1 soundtrack that can be effectively downmixed to stereo, the same cannot be said for a 22.2 cinematic release!

Object audio is an all-new paradigm in sound mixing which bypasses the concept of audio tracks entirely. Instead, the audio engineer positions sound at a theoretical point in a three-dimensional space, building a virtual audio model. There are no prescribed audio channels and translation to the available speaker configuration is handled automatically in software, meaning that the "perfect" model is maintained as the reference mix at all times.

Of course, swapping multiple channels for vast numbers of objects, whilst increasing translatability, does not necessarily lead to a reduction in complexity. Therefore, the object audio model also includes a 9.1 channel-based "bed track" (7.1.2 configuration) for use in combination with specific objects.

This is a dramatic conceptual step forward, but it will require new tools in NLE/DAW systems for placing and controlling objects and generating the complex associated metadata. Potentially, object audio technologies such as Dolby Atmos and its newly announced rival, DTS:X, mean that a single mix can be universally translated from the cinema right down to the most basic domestic environment. This preserves as much of the original artistic intent as possible without the need for individual cinema, 5.1, LCR sound-bar, or stereo mixes.

In addition to providing a single unifying mix from the theatre to the home, object audio offers another potential benefit to the home consumer: personalisation. Naturally, a personal auditory experience is more satisfying and immersive than a one-size-fits-all approach. Mixes using objects rather than fixed channels can allow elements to be exposed to user control. Dialog, for instance, can be made available as an individual object, allowing the user to raise or lower the level for personalised levels of intelligibility and comfort. Different objects can be activated and deactivated to allow multiple languages or viewpoints to be delivered within the same mix.

However, these new possibilities are not without their complications. Take, for example, a solution that gives the user control over the crowd volume level, position in the stadium, and commenter/dialogue levels for a football game. How will these elements be controlled? Will the user get lost in the interface and ultimately miss key moments in the game while fiddling with the controls in an attempt to get something that "just works?" Clearly these technologies allow for an immense level of customisation, but there is much work to be done so that the consumer can enjoy the benefits with no more effort than turning on the TV and sitting back to enjoy the show.

There are also technical issues to consider. We have only just settled on international standards for loudness measurement and control of channel-based audio. How does this relate to object audio? If the user has control over the elements included in the sound track, how is loudness taken into account? How will we stream all these new audio objects within the current bandwidth limits of existing distribution models? Already research is underway to examine these issues and many others, but perhaps this is the purview of a more technical discussion.

One thing however is sure: immersive, object audio is one of those rare conceptual game changers that has the potential to re-define how we create and enjoy sound. With their scalability and potential for new levels of creativity and realism, these concepts are likely here to stay. How this will pan out for audio engineers, broadcasters, and consumers remains to be seen. But for those of us involved at the creative end, now is the time to skill up and immerse ourselves in this exciting and rapidly developing technology.

Jon Schorah is founder and creative director of NUGEN Audio.

This article is also available to read at BFV online as part of this issue's Audio feature here, page 38.

(JP/LM)