Eye-Gaze Steered Beamforming for Hearing Aids

Student thesis: Master thesis (including HD thesis)

  • Simone Birk Bols Thomsen
4. semester, Mathematical Engineering, Master (Master Programme)
If multiple microphones are available in a hearing aid (HA) device, beamformers can be applied to enhance target speech signals in noisy environments. Many common beamformers require knowledge of the target sound source location relative to the HA user. Traditional beamforming methods are equipped with techniques that try to localize the target source acoustically, i.e., only using microphone signals. However, localizing the target source in presence of competing speakers remains an unsolvable problem. In this thesis, we study the use of an additional modality, apart from sound, to help enhancing the target signal. Specifically, we aim to use the HA user’s eye-gaze as an asset to efficiently identity the target direction. Initially, we examine the potential performance benefits of using eye-gaze steered beamforming under ideal conditions. Subsequently, we propose two eye-gaze based beamforming systems, namely a Bayesian beamformer with the posterior probability on the target direction estimated based on a prior probability derived from the user’s eye-gaze, and a Bayesian beamformer with the posterior jointly estimated from the HA microphone signals and the HA user’s eye-gaze signal. The performance of the proposed methods are compared with current audio-only methods. The main conclusion is that, under certain conditions, the proposed eye-gaze based beamformers are able to outperform audio-only methods in terms of estimated speech intelligibility and quality.
Publication date3 Jun 2022
Number of pages120
External collaboratorOticon Danmark AS
Jesper Jensen jesj@demant.com
ID: 472096049