Improving Context Awareness in Hearables

On-ear devices, also known as hearables, are one of the fastest-growing devices in the consumer electronics market today. Although at present their role only covers hearing aids and a tool for listening, the on-ear devices have progressed to being wireless. Now there is a brand-new way for engaging this technology to the wider world around us.

Qualcomm conducted a survey in 2019 and found that more than half the respondents were interested in hearables that are context-aware. One of the most useful capabilities the respondents were looking for in hearables was background noise reduction, and the other, dynamic volume adjustment.

The interest that users are showing for these features is for the next-generation of hearables. They are looking for a better, more intensive listening experience. With an increasing interest in hearable, users are now expecting these next-generation features that are currently not available.

For instance, traditional hearables may be wireless but controlled by the phone. While jogging or working out at the gym, it is inconvenient for the user to stare at their phones for adjusting the volume of their headphones. Even with buttons available on the headphones, they are likely to be tiny and not visible when the headphones are on the user’s ears. That makes it very difficult to locate and use the buttons.

One way of improving the user interface would be to add gesture control. Simple gestures and motion tracking can provide instructions for specific actions and controls. For instance, a simple tap on the earbud could mean an increase in volume. Tapping the entire headphone is much simpler than finding and pressing a specific button on it.

A gesture for detecting in-ear presence could automatically pause the audio as soon as the user removes the earbuds from their ears. The audio can resume the moment the user inserts the earbuds back in their ears.

As the range of movement of the human head and ears is relatively consistent compared to that of the pocket or the wrist, hearables can be ideal for tracking fitness. However, motion tracking needs to be precise to not generate false positives and negatives. Therefore, with proper fitness algorithms, it is possible for hearables to track whole body movement such as when running, biking, or standing in a queue. Accurate classification is necessary to convert step counts to calorie counts, providing a more complete picture of the user’s day.

Hearables with spatial audio can change the sound as the head turns, and linked with the accurate head tracking, can put the user right in the center of the orchestra, leading to a truly immersive listening experience. However, this also requires the latency to be low, so that the sound changes with the head movement without delay. This can help and elevate the user’s experience with XR or gaming as well.

Today’s headphones cannot provide the user with the above experience. Users may have to turn down the volume or remove at least one earbud when they want to listen to the external world. This is because the design of the hearables blocks most external sounds to enable the listener to focus on the audio.