Virtual reality in hearing research: Opportunities and future challenges

Virtual reality (VR) has been around for decades, but it’s only in the past few years that it moved beyond research labs and professionals, towards the consumer market. Considering the areas of audiology, hearing science and hearing aids technologies, it is easy to see how VR could very soon become a major player, both in research environments and clinical practice.

By Lorenzo Picinali (Imperial College London) and Debi Vickers (University of Cambridge)

Above: Figure 1: A teenage cochlear implant user, together with an audiologist from the research team, being involved in one of the BEARS participatory design sessions

One of the main issues with current clinical hearing assessments is that they are not representative of what happens in real life. For example, the procedures and signals employed during pure-tone and/or speech audiometric assessments, albeit being very controllable, repeatable and precise, are rather far from what individuals would experience in their everyday life, and from the situations in which their hearing impairment would cause problems. On the other hand, assessing hearing in typical everyday life settings would be rather problematic, as these are difficult to control and calibrate, and generally non-repeatable (or at least not with sufficient precision).

VR can easily allow the creation of very realistic scenarios using both immersive visual and audio techniques, and at the same time, being very controllable and 100% repeatable, especially when the rendering is done through a VR headset and a pair of headphones (i.e. very close to the eyes and ears). This means that new hearing tests could be designed which are much closer to what people experience in their lives, simulating for example those specific situations where difficulties are encountered, such as a conversation in a noisy restaurant, or watching a film at the cinema. Furthermore, the same technology could be used to demonstrate the functionality of hearing interventions (e.g. hearing aids), or to improve the fitting and personalisation of a hearing device, or again to improve hearing performances through perceptual training.

Above: Figure 2: A child using the BEARS VR apps during an outreach event

VR technologies
In terms of availability of VR technologies, we no longer need expensive computers and headsets worth thousands of pounds, but standalone devices costing a few hundred pounds can be used, allowing full position tracking without requiring a specific installation of antennas and other equipment in the surrounding environment. Furthermore, the performance of such devices is significantly higher if compared with what was available even just three years ago, allowing for complex interactions to be rendered very smoothly, without resulting in audiovisual glitches which could cause VR-based motion sickness.

An example of research going in this direction is the BEARS (Both Ears) project, recently featured in an article in The Observer . Evidence has been produced showing that we can learn perceptual hearing skills using a computer application, and that these skills might be generalisable to other domains. Using a participatory design approach, in which a patient group (eight to 16 year old bilateral cochlear implant users) collaborated with the developers to create a suite of VR games to help train spatial hearing skills (see also Figure 1). Three games categories have been created:

1. sound localisation (target practice);

2.speech-in-noise (ordering food at a diner and building a pizza); and

3. music (beating a rhythm and being the DJ).

All the games have different levels, provide feedback and for the speech-in-noise; incorrect sounds are repeated to help correct and learn sound contrasts. The BEARS games have also been adapted for the younger participants in our group and are being implemented on both a head-mounted display and an iPad so that children who don’t like wearing the headset or have balance issues can still get involved.

While a significant amount of work has already been done in designing, developing and evaluating VR tools and applications to facilitate hearing research, it is clear that there is still a lot to be done before such techniques will be widely standardised and, possibly, also available in clinical settings. In preparation for the Virtual Conference of Computational Audiology (VCCA) that was held in 2022, we involved several researchers and clinicians in an exercise attempting to map the future challenges with VR applied to hearing research. Some 26 separate areas of focus were identified, and later grouped in four separate categories:

1. hardware and software development: designing and developing new equipment and tools;

2. validation and standardisation: assessing the benefits, comparing these with existing solutions, and contributing to the standardisation of the novel approaches and tools;

3. realism and control: better balancing the trade-off between realism of the simulation and control of what is being delivered to the participant/patient; and

4. applications and use in clinical settings: explore uptake in clinical routine through extensive clinical trials.

Any large-scale implementation of VR in clinical practice would require a support infrastructure to keep the technology current, it would need to be engaging and exciting and be able to respond to any technological challenges that arise.