Although we are used to traditional interfaces like touchscreens and keyboards, interfacing with computers has traversed a long distance over the years. Now, it is possible to turn any surface into an input device. DAWSense can do this by utilizing machine learning and taking advantage of surface acoustic wave technology. With different situations requiring varying methods of input, researchers are now exploring newer methods of human-computer interfacing. One of them is to embed the interface within everyday objects, thereby enhancing user experiences.
Human-machine interfaces may take many forms. For instance, the industry often uses microphones or cameras to control devices using methods like speech or gesture recognition. Although such systems may be of immense help in certain applications, they may not be practical for others. In a camera-based system, it is easy to obscure the arrangement by introducing objects in front of the camera. Similarly, microphone-based systems involving speech recognition may not function properly in noisy environments.
As an alternative, researchers were experimenting with transforming any arbitrary surface to act as an input device. For instance, for controlling a smart home, they have experimented with the arm of a couch acting as a TV remote, or an interactive wall. They have tried many methods for building such functionality so far, with accelerometers standing out as one of the most promising solutions, as they can sense touch gestures on various surfaces without any modifications on them.
However, the sampling bandwidth of accelerometers incorporated into a surface to act as a touch-sensing device is not enough to capture more than a few relatively coarse gestures. Now, a collaboration between researchers at the Meta Reality Labs and the University of Michigan has demonstrated another method that offers the necessary bandwidth for creating user interfaces that are more advanced.
The new method relies on SAWs or surface acoustic waves rather than mechanical vibrations for sensing touch inputs. The team has also fashioned a VPU or voice pick-up unit for detecting subtle touch gestures. They have designed the VPU to conduct the surface waves into a hermetically sealed chamber that contains the actual sensor. This practically removes any interference from background noise. As the team has fabricated each VPU using the MEMS process, the sensor has the necessary high bandwidth that is typically associated with a MEMS microphone.
Although the MEMS sensor was a high-performance one, the researchers still needed a method for converting the SAWs into swipes, taps, and other gestures. A hard-coded logic would fail to convert them satisfactorily, so the team had to design a machine-learning model with an algorithm to learn from the data.
VPUs typically collect a huge amount of data, and processing this data on an edge computing device in real-time would be a challenge. The researchers dealt with this problem by calculating Mel-Frequency Cepstral Coefficients, which helped in understanding the most informative features of the data. With this analysis, the researchers could reduce the number of features they needed to consider from 24,000 to just 128. They then fed the features into a Random Forest classifier for determining the exact representation of the surface waves.
