Loose Lips Could Sink Perps

Michael A. Greenwood

Even a whispered secret may no longer be safe.

A team of researchers at the University of East Anglia in Norwich, UK, has begun a three-year project to devise computer-driven lip-reading systems that could take a videotaped conversation and convert it into text.

Although an eavesdropping camera that recognizes every turn of the mouth might sound like George Orwell’s Big Brother, proponents say that the technology could be a valuable tool for crime fighting, security and a host of more mundane applications, such as mobile phone cameras and in-car speech recognition.

The futuristic system is being developed, in part, to replace the shrinking pool of those who can read lips. As more people with hearing problems learn sign language to communicate, fewer people are mastering the art of reading lips, said Richard Harvey, a senior lecturer at the university’s school of computing sciences and the project’s lead researcher. And even those who know how to read lips are not always precise.

There are a number of challenges to creating an accurate computer program for lip reading. Any system will have to be able to track the speaker’s head from a variety of poses, to extract features that describe the lips and then to match those movements with corresponding text. Harvey said an additional challenge will be developing a system with a large lexicon.

Systems are planned for languages other than just English, including modern standard Arabic, which is quite expressive on the lips and therefore somewhat easier to lip read. Languages such as Mandarin and Cantonese are difficult to lip read.

Harvey said lip readers are frequently used in the UK to decipher silent footage taken from a security camera. That testimony can be quite controversial, however, and its accuracy has been challenged. A computer would be dispassionate and bias-free.

“Whether we can make a computer as reliable as a skilled person is a big question,” he said.

One of the better known cases of reading silent video footage involves the tapes of Adolf Hitler’s that were found after the demise of the Nazis in 1945. Researchers have been using modern software to figure out what Hitler and his cohorts uttered decades ago.

The university is working on the computerized lip-reading project with the Centre for Vision, Speech and Signal Processing at the University of Surrey in Guildford, and with the Home Office Scientific Development Branch, both in the UK.