Today's incarnation of stereoscopic 3D, currently sweeping cinemas and pubs, with the living room the next to succumb is all about the look, writes Kevin Hilton. Not just the extra dimensional images but polarized glasses, screens and now the new breed of plasma TV sets. But audio has not been left behind. Sound for 3D is being discussed and researched seriously, although quite what it will be is unclear right now.
What we do know is that 5.1 is linked to 3D, just as it is for 2D high definition. The Super Hi-Vision television system developed by NHK in Japan has something approaching multi-dimensional sound in the form of 22.2, which gives a sensation of height as well as width and depth, but the number of microphones and loudspeakers involved to create this would be impractical in most venues, let alone the home.
The ideal could be something between the two; a manageable number of channels but with a sense of immersion that matches the pictures. In US sports broadcasting sound engineers are experimenting with the conventional 5.1 mix as a foundation, augmented by feeds from microphones placed closer to the action.
Audio for 3D involves a shift in perception - and not just because of the visual third dimension. Viewers today are used to, and expect, numerous camera angles, covering sports and entertainment alike from as many aspects as possible. This point took some time to reach in 2D and the same will be true for stereoscopic production.
The 2008 Six Nations rugby match between England and Scotland was covered with just three camera rigs, each containing two cameras. Broadcasters and producers now have the relative luxury of eight to 12 rigs but fewer angles means that a 2D 5.1 mix cannot be used because the perspective is different. It also calls for a separate commentary.
This has been the experience of BSkyB, which launched a 3D channel at the beginning of April. "We're exploring audio for 3D, which is very much a visual experience," comments Keith Lane, operations manager for Sky Sports. "We want the audio to match the visuals, of course, so it is following the video, which means it will be slightly different to 2D coverage."
Lane says the aim is to have viewers "more engaged" with what they are watching, from both the visual and aural perspectives. There are technical problems to overcome, however, including compensating for the delay on the pictures introduced by 3D processing systems. Sky uses assignable delays on the Calrec Audio Apollo desk installed in Telegenic's 3D OB truck (shown in picture), which has been used for Premier League coverage.
Technicolor is also looking at synchronisation for its new 3D service, launched in west London during March. Right now the facility is working with stereo sound for the stereoscopic pictures but its operational and research teams are addressing the problem of delay and possibilities for future systems.
"There is a changing culture in how live broadcasts are done because there might be 10 to 12 seconds of delay," comments Will Berryman, global chief operating officer of Technicolor Digital Content Delivery. "We are looking at the signal paths but what this comes back to is old engineering skills, synchronisation, tuning and timing, in terms of what happens when."
Technicolor's technology department is looking into a new sound system to complement 3D pictures, both in cinemas and the home. "Because of the depth of the picture and particularly when images are moving quickly, the sound should follow the images," observes Gary Donnan, senior vice president of research and innovation at Technicolor.
This is being done by rendering sound, with the aim of producing a system that is standardised and open to all developers, as well as compatible with both 5.1 and stereo.
For Sky's football coverage raw B-format signals from SoundField surround mics already installed in stadia is being used to create a separate mix from the 2D 5.1 feed. Pieter Schillebeeckx, head of research and development at SoundField, observes that broadcasters are still experimenting with 3D in general. "They're finding out what works and what doesn't, particularly as there are fewer camera angles to match to and these are often lower than is usual in 2D," he says.
Manufacturers of surround sound coding systems and mixing consoles are also considering how 3D might call for changes in approach. Dolby Laboratories has expanded from audio into full d-cinema systems and at NAB 2010 launched an open 3D specification for broadcast. This includes a 3D coding format that is able to carry material through 2D production chains, rather as Dolby E did for surround over stereo connections.
"Surround sound is considered as standard for 3D, just as it is for HD," comments broadcast marketing manager James Caselton. "We are working with broadcast partners to help them create the right infrastructure to allow Dolby audio to be used for the future, whether streamed or as files."
At DTS Mark Seigle, vice president of business development, says the company is "excited" about consumers getting the full 3D experience: "Audio is an important aspect of that, to envelope them in what they're watching. We're seeing a large take-up in 3D in the US from broadcasters like ESPN. Sound, both 5.1 and 7.1, will play an important role for everyone working with the technology."
Dr John Emmett, technical director of Broadcast Project Research, says that in creating sound for 3D mixers are taking the general approach of conveying depth by using the surrounds to bring images back, "keeping the centre well out of it so only the
front pair sound deep".
The focus in sound can often be on the hardware and while observing that 3D will promote sales of the latest audio mixing technology, Henry Goodman, business development manager at Calrec, sees it as good for creativity in sound as well: "It pushes the boundaries of what can be achieved in surround mixing and encourages mixers to think about mixing in different ways."