XR Turns Fingertips Into Microphones

⊲ Hackster.io ⊳ Carnegie Mellon University researchers have developed SoundBubble, a system that uses XR headset microphones and hand tracking to create virtual microphones that follow users’ fingertips. By isolating sounds from finger movements, it detects taps, swipes, subtle gestures and interactions with everyday objects without additional wearable sensors. Tests involving ten participants achieved a 94.7% overall detection rate across three applications, including noisy environments. The technology could enable more natural interaction with augmented reality interfaces and passive physical controls, with simulations suggesting compatibility with smart glasses.
◂ SoundBubble’s ability to wake passive 3D-printed knobs and sliders points towards cheap, electronics-free cockpit and console mock-ups for XR training rigs. ▸
⊲ Featured Image – SoundBubble can focus on sounds made by the hands – D. Kim et al. ⊳
⊲ www.simulation.news ⊳