NHS hospitals are trialling an AI lip-reading app to help patients who struggle to speak
“I am in pain,” says a digital voice as 33-year-old Nathan Armstrong stares into an iPad at Royal Preston Hospital in Lancashire, northwestern England.
He is using SRAVI, short for Speech Recognition For The Voice Impaired, a lip-reading app that is being trialled in UK public hospitals among patients who have difficulties speaking, whether it’s because of throat cancer, throat surgery, or a stroke.
Created by a Northern Irish company, it uses artificial intelligence (AI) to decipher the lip movements of patients, and help them communicate better with healthcare workers and their families.
Patients simply need to mouth a phrase in front of a smartphone camera, and the app will interpret it and say it out loud for them.
Armstrong, who experienced paralysis of his vocal cords ever since he was a baby, had a tracheostomy in 2019.
While the surgical opening in his windpipe helped him breathe better, it caused him to lose his voice. Because most of the air he breathes out passes through his tracheostomy tube rather than over his vocal cords, he can only talk by covering the breathing hole and forcing air out through his mouth.
The new lip-reading app has had a “big impact” on his life, he says – in the hospital and beyond.
‘A real godsend’
“Breathing is very hard and tiring. So it helps me to communicate with others, without having to have a carer or anyone with me,” he said. “I can do it all on my own, which gives me more freedom to do things with my children as well”.
The Belfast-based company behind the app, Liopa, says it can understand more than 40 common phrases – such as “I need the bathroom” or “call my family” – and the software keeps learning to improve its performance.
Also satisfied are the health workers at the hospital where it’s being tried out.
“SRAVI has been a real godsend,” said Linda Phillips, research nurse for the trauma and acute care team at Royal Preston hospital.
“I’m an absolute rubbish lip reader. So it has really, really helped. And I feel that I’m able to give better care because I know what the patient’s asking me for”.
While the UK’s National Tracheostomy Safety Project highlights the importance of communication with tracheostomy and laryngectomy patients, tools for this have been limited.
Communication boards and analogue and digital tablets have helped, but some patients are too weak to have that “fine finger dexterity,” said Shondipond Laha, a critical care consultant at Royal Preston Hospital.
“Those things are useful, but they’re very hard at expressing whole phrases, for example,” he added.
For more on this story, watch the video in the media player above.
Video editor • Roselyne Min
“I am in pain,” says a digital voice as 33-year-old Nathan Armstrong stares into an iPad at Royal Preston Hospital in Lancashire, northwestern England. He is using SRAVI, short for Speech Recognition For The Voice Impaired, a lip-reading app that is being trialled in UK public hospitals among patients who have difficulties speaking, whether it’s…