When it comes to expertise in the recording of immersive sound, there are no greater experts in the world than Jim Anderson and Ulrike Schwarz. Husband and wife as well as production partners in Anderson Audio, New York, Anderson and Schwarz have made dozens of world-class, award-winning immersive recordings.
For about 50 years, Jim Anderson has won a dozen Grammy and Latin Grammy Awards, two Peabodys and a pair of Emmy nominations among countless accolades for his sound recordings. These include studio albums, live concerts and film scores with the likes of McCoy Tyner, Christian McBride, Arturo Sandoval, Petula Clark, Patricia Barber, John Zorn and hundreds more.
In 2003, Anderson became a professor at the Clive Davis Institute of Recorded Music in the Tisch School of the Arts at New York University. He is also a former president of the Audio Engineering Society.
Ulrike Schwarz, born and raised in Germany, was one of only three students accepted into the exclusive Tonmeister program at the Hochschule der Künste Berlin. She became a sound engineer with German public radio and spent more than a decade as the senior sound engineer for large orchestral and jazz productions at the famed Bayerischer Rundfunk in Munich.
Schwarz has created a groundbreaking body of work in the field of immersive audio, earning multiple Grammy nominations for her work with artists as diverse as American jazz stalwart Patricia Barber and contemporary Norwegian composer Gisle Kverndokk.
Broadcast Beat’s Frank Beacham, who first met Anderson in the 1990s while he was mixing Garrison Keillor’s A Prairie Home Companion radio show, spoke with the couple about immersive audio.
Frank Beacham: As partners, you record orchestras with 60 or more microphones. But today immersive audio is being marketed to far less experienced people with much lower budgets. Jim, in terms of how you teach immersive recording to your students at NYU, where do you begin?
Jim Anderson: Before we get to immersive sound, I try to teach kids the fundamentals first. I usually start them working in 5.1 surround, because they pretty much know how to record stereo. We determine whether to put a sound in the front, left or right. We start with the basics so the students are not like a deer in the headlights when presented with an entire array of speakers for a Dolby Atmos set-up.
Frank Beacham: You’ve said that in an immersive recording, the original sound source must be very good. Otherwise, the recording just sounds big and lame. I assume the importance of the musical performance is also very important.
Jim Anderson: The bigger a sound plays, the more the flaws tend to be revealed. We are very aware that if the music isn’t happening, immersive is not really going to work.
Frank Beacham: Yet, immersive is being pushed for live club recordings all over the world. Are these recordings or broadcasts really hit and miss?
Jim Anderson: Yes. I can think of about ten really good live recordings like that. It is so rare that you get a live performance that just has the edge and stands out musically.
Frank Beacham: Equipment manufacturers have been promoting a single microphone with four capsules designed to simplify immersive recording. How well do those mics work?
Ulrike Schwarz: Even Sennheiser told us that you should only use their VR microphone in connection with video, which is not a convincing argument when you’re recording audio. We are not fans of prefixed microphone arrays. I heard somebody who should know better say last week that they are waiting for the first immersive piano microphone to come out. To me, that kind of sounded like a joke.
We don’t create an immersive or surround field out of thin air. We use both very expensive microphones and not so expensive microphones. We don’t recommend a single microphone with four capsules. To do that, you have to imagine walking into a situation where everything is perfect, which never happens.
You have to understand what a microphone does. Maybe you’re trying to capture a live atmosphere. Some people mount a microphone on a camera to get the ambient sound. That doesn’t give you an immersive recording. I think Jim’s idea of starting his students off in stereo and then moving up is the right approach.
Jim Anderson: I tend to get students to read some of the some of the original literature that was done about the height of sound. You can’t just take an immersive mix and throw it up. You have to be selective and very careful, and also logical about what you put in the immersive height channels. Otherwise, it just turns into a big bland mix.
Another thing you need to know is the more speakers and the more directions that signals can come back from, the better. In the end, a mix has to make sense. If the bass is on the left behind you, that’s just disturbing and not going to get you anywhere.
Ulrike Schwarz: For a mix to make sense, you have to decide which instrument comes from which direction. The more choices available, the more nonsense some people come up with. That’s why I agree that starting beginners off with fewer channels and then expanding the possibilities as they learn is the right way to go. It is overwhelming once you sit in the chair for the first time and have all these things coming at you.
Frank Beacham: A confusing part of immersive sound is that manufacturers describe their various products and technologies only in unique marketing terms. An example is Sennheiser, with their AMBEO line. It covers everything from the Neumann KU 100 binaural dummy head, which came out 50 years ago, to their more recent VR mic for video. All these products are folded together under the same AMBEO branding. Can you offer a real-world definition of what immersive audio is and what it isn’t?
Jim Anderson: I would define immersive as surround with height. I’m surrounded by the mix in general, but the mix also has the third dimension of height.
Frank Beacham: It is clear that we are in a time when immersive technology is developing fast. What are your takes on what companies like Dolby and even Apple are doing with sound?
Jim Anderson: Immersive technology is getting better and better almost daily. It’s far better than five years ago. Now, with head tracking, and the new iOS on the phones and on the iPad, it’s getting darn impressive. The technology is constantly improving.
Frank Beacham: Jim, as a former president of the AES and from a standards perspective, what needs to be done to tame the confusion over immersive audio?
Jim Anderson: I’m not aware now of what the AES is up to on this front. But there have to be some standards. We’re kind of locked into whatever one company wants to do over another company. We have different technologies…binaural is quite good for headphones. But to me binaural is the Neumann KU 100 dummy head microphone, while for others it’s taking an immersive mix and then transferring it over to channels. Most people are not going to set up eleven speakers in their home. At our home, we have a soundbar and we listen to immersive off of it so we know how our work sounds in the real world.
We just finished a project called Gardenview that just came out a couple of weeks ago with Nataly Dawn from Pomplamoose. It’s available in true Auro3D and ADM. It’s on Apple Music and you can listen to it in binaural immersive. Another is Jane Ira Bloom’s Picturing the Invisible-Focus 1 in 9.1 Channel Auro3D 96k FLAC Immersive Audio. It was inspired by the science photography of legendary NYC photographer Berenice Abbott. These projects show what one can do today with this technology.
Frank Beacham: I remember at an AES show in the mid-80s hearing Hugo Zuccarelli’s Holophonic sound, an early form of immersive. Though it was on analog cassette, it was startling in its detail and caused quite a stir. I recall a headphone recording of the sound of barber shears cutting hair that was so realistic, you could actually feel your own hair being physically cut. This kind of realism seams to open the door to entirely new kinds of storytelling. Beyond music, what opportunities do you see ahead for using immersive audio?
Jim Anderson: There have been some explorations in this area already. The BBC has had some immersive storytelling that were sound portraits of various occupations. One was about a squad that defused bombs. When wearing headphones, you really feel like you were are the trenches with these guys and sweating it out as they work. ‘Do I clip the yellow wire or the red wire?’ Another one was a circus, where the recording gave you the very realistic sense of being a circus performer. If you can think of any kind of binaural situation that you can put people in, it has the potential to be good storytelling. As kids today experiment with this technology, I think the true storytellers will come out.
Thank you, Jim Anderson and Ulrike Schwarz!
Beacham has served as a staff reporter and editor for United Press International, the Miami Herald, Gannett Newspapers and Post-Newsweek. His articles have appeared in the Los Angeles Times, Washington Post, the Village Voice and The Oxford American.
Beacham’s books, Whitewash: A Southern Journey through Music, Mayhem & Murder and The Whole World Was Watching; My Life Under the Media
Microscope are currently in publication. Two of his stories are currently being developed for television.
In 1985, Beacham teamed with Orson Welles over a six month period to develop a one-man television special. Orson Welles Solo was canceled after Mr. Welles died on the day principal photography was to begin.
In 1999, Frank Beacham was executive producer of Tim Robbins’ Touchstone feature film, Cradle Will Rock. His play, Maverick, about video with Orson Welles, was staged off-Broadway in New York City in 2019.
- Dealing with Video Deepfakes in the Era of Artificial Intelligence - November 22, 2023
- Using Bluetooth Timecode Systems in the Video Workflow - November 15, 2023
- The Shape of Your Video Camera - November 15, 2023
