Normally, the annual Mobile World Conference (MWC) in Barcelona wouldn’t have much news of particular interest to AVS Forum. But this year, there is one announcement I’m quite excited about: the public debut of the THX Spatial Audio Platform. I learned about it at CES last January, but the information was under embargo until now. The system is designed to simulate the effect of sounds coming from various directions and distances using headphones or a pair of speakers, a process called spatialization or virtualization.
The THX Spatial Audio Platform is an end-to-end ecosystem that encompasses content creation, encoding and decoding, rendering, tuning and device optimization, and user customization. It supports immersive-audio formats, including object-based and ambisonic, as well as legacy formats on a wide range of devices, such as mobile, PC, and consumer electronics.
For those who might be unfamiliar with the terms, object-based formats like Dolby Atmos represent individual sound-emitting “objects” that are placed within the three-dimensional soundfield. Ambisonic formats also focus on the direction from which sounds appear to come. Ambisonic microphones are often spherical with mic elements pointing in many directions. Both formats are independent of the speaker layout.
Components of the platform include content-creation plug-ins that produce immersive audio in object-based or ambisonic formats and integrate into industry-standard audio-design tools. The encoding and transport format is MPEG-H, which was developed primarily by Fraunhofer IIS and is now part of the ATSC 3.0 broadcast system. It can also be used in streaming content.
The rendering engine spatializes object-based, ambisonic, and legacy channel-based content on headphones and even speakers using crosstalk cancellation. A tuning and optimization module is said to measure and calibrate the audio playback to deliver the highest-quality audio experience with the available equipment.
Perhaps most important is a customization function that lets users create a personal HRTF (head-related transfer function). Creating the illusion of sounds coming from outside your head while listening on headphones depends on simulating the effect of your head on those sounds. That effect is mathematically modeled with an HRTF.
Most headphone-spatialization systems use a single, generic HRTF. It works well for some listeners but poorly for others, depending on how closely their personal HRTF matches the one being used by the system. The THX Spatial Audio Platform addresses this problem by allowing users to measure their own HRTF.
According to THX, “The system uses the phone camera to take an image of the ear and send it to a cloud-based AI (artificial-intelligence) system. The AI system uses a complex ergonomic model to scale the head and make hundreds of measurements of the features of the pinna (outer-ear cartilage). From this, it can calculate the HRTF in less than a minute and return it to a user’s device. This removes the variability of transit time from the device to and from the cloud, authentication, etc. Other custom-HRTF services, such as IDA, need more info and can take hours or days to calculate the HRTF.”
What about head tracking to anchor the audio environment as you move your head? This is particularly important in gaming and virtual reality. THX Spatial Audio is fully compatible with head tracking, but developers can implement it using whatever technology is most appropriate. For example, the Samsung Gear VR would use the accelerometer in the phone.
At CES, I heard several examples of the THX Spatial Audio Platform in action. I started with a pair of Oppo PM-3 headphones; the source was a laptop using the HRTF of Patrick Flanagan, Audio Architect at THX. (The custom-HRTF function had not been implemented in the prototype I heard.)
First up was a 2-channel clip from Pink Floyd’s “Money.” Turning spatialization on, the effect was obvious; the sound seemed to come from outside the headphones with some of it appearing to come from above and in front of me. Turning it off collapsed the soundfield to a normal headphone presentation inside my head.
I also listened to a THX trailer in 5.1 on the Oppo headphones. With spatialization on, the sound seemed to come from speakers in the room. Turning it off, the soundfield collapsed as before.
Next, I listened to the 5.1 trailer for the movie Gravity on a pair of computer speakers. The spatialization was surprisingly effective from a pair of small speakers on the table in front of me.
Finally, I listened to a binaural recording of Wycliffe Gordon’s “Dreams of New Orleans” from Chesky Records on the tiny speakers of a Motorola mobile phone. Turning spatialization on, the effect was remarkable—the soundfield extended way beyond the speakers.
THX plans to license the technology—which includes hardware and software—to device makers as well as app and game developers. At MWC 2018, THX and Qualcomm are demonstrating the THX Spatial Audio Platform, which gives you a clue about who will be producing the chipsets for it.
This technology is quite intriguing. It’s especially appropriate for gaming, VR, and watching movies with headphones, not to mention music. I look forward to seeing it implemented in a commercial product.