Understanding spatialised and immersive audio

Spatial audio pioneers Robin Whittaker and Dave Haydon of TiMax developer Out Board give us insight into the trends of spatialised and immersive audio, including in theatre audio and show control
Author:
Publish date:
Updated on
Life of Galileo at the Young Vic. Photo by Leon Puplett and projections by 59 Productions.

Life of Galileo at the Young Vic. Photo by Leon Puplett and projections by 59 Productions.

Spatial audio pioneers Robin Whittaker and Dave Haydon of TiMax developer Out Board have penned an exclusive piece for PSNEurope, offering some insights into ongoing and future trends in theatre audio and show control, with a focus on spatialised and immersive audio.

The trends we are now seeing in spatial reinforcement and immersive audio differ only in subtlety to those we’ve always been interested in: realistic panoramas created by effectively rendered and managed localisation and spatialisation. And this is, no less, now appreciated more for orchestral rather than vocal reinforcement. A wider range of people are showing a greater awareness of what this process is and how it can enhance the listening experience. As a result, we are seeing globally greater interest for shows and events to move into the experiential, immersive audio space. 

Traditionally, the interest has come from experienced sound designers expanding their boundaries, creating soundscapes for their audiences that add more realism and dramatic impact. Nowadays, however, increasingly more mid-level customers are looking to create something experiential for their audiences. 

People have been sold on the concept of using immersive audio for its added wow factor, but often cannot describe what it is they want or expect. We find we need to dissect this new immersive audio paradigm in discussions with creatives, as it means different things to different people with variations that can be nuanced. 

However, once spatialised audio and localisation are experienced, the realisation of their benefits for an individual situation or a project is fast. Beyond the laudable development of ever more accurate and great sounding speaker systems over the years, localisation is the obvious triumph. A major part of this current immersion discussion, in a stage environment, is that multiple localisations can be made effective for an entire audience.

When you start looking closely at why spatialised audio sounds better, you can start to see why more people are interested in these types of sound designs. There are multiple mechanisms at work here. In particular, you can avoid all sorts of masking by keeping sources separated in space, which is more in harmony with and closer to how we naturally hear. 

This mixing of sources in the acoustic domain is demonstrably superior in terms of imaging and spatial unmasking than combining them electronically. Two instruments playing more or less the same note at the same amplitude will combine properly if they are absolutely in phase, but if you mix even just two out of phase, there will be destructive interference. Spatially reinforced audio, using strategic delay matrix management in particular, amplifies audio sympathetically by separating individual elements into a more familiar and intelligible spatial panorama. Placing and exposing sources individually in space minimises phasing and masking, which is central to the conversation as to why audio sounds better spatially balanced as opposed to mixed electronically. 

In the old days, when PAs were smaller and cruder, that’s pretty much what used to happen anyway. You would come to hear each instrument amplified by its own amplification system. When PAs started getting bigger and bigger and more powerful, and when monitor engineers converted to in-ear systems with minimal on-stage sound, everything the audience started to hear was then mixed to a pair of stereo loudspeakers.

There’s also been a growth in interest about a hybrid spatial and immersive experience. For Alan Ayckbourn’s The Divide at the Old Vic, sound designed by Bobby Aitken, the emphasis was on strong but unobtrusive spatial reinforcement for the actors’ mics to communicate the wide range of pathos and bathos in the piece. Complementary to this was the use of live instrumentals with woods, strings, piano, and electronics mixed with a 40-piece choir, all located upstage, which was then balanced with the acting voices’ variable dynamics. The dystopian fantasy nature of the piece lent itself considerably to immersive creativity, and a series of enhanced wide surround imaging objects were created to dynamically spread separate choir sections and spatial reverbs, making enough space to ‘lift the cast slightly and hear the nuances of their voices’ without having to use gain to help them cut through. 

Further into the immersive domain, The Young Vic’s Life of Galileo was dominated by a massive video mapped projection dome above the stage and audience, with high energy electronica from the Chemical Brothers’ Tom Rowlands, and the performance spread around amongst the audience. All the seating areas were mapped with their own matching surround zones so the music and effects could be intimately experienced by everyone, as well as larger spatial zones for the huge universe and sunburst tableaux that played out on the overhead dome. Not something you could readily throw a spatialisation algorithm at, but not untypical of the sort of stagings you’re seeing these days. 

A strong trend now is tracking where interest seems to have snowballed. Fortunately, we’re now able to build on the experience and successes of our already well established system, with a brand new tracker system coming out this year that provides a quantum leap in resolution, affordability, and application diversity. 

Related