Showing posts with label speech to text. Show all posts
Showing posts with label speech to text. Show all posts

Monday, May 09, 2022

Why is Speech Signal Processing more complex than Text Processing


At best a speech signal can be best described as 

 indiaisthelargestdemocracywelcometoindia

 and in reality it is 

either 

indiaesthelarzestdemocracyvelcometwondia  or indiaes thelarzestdemo cracyvel cometw ondia

Those of you who have paid attention to the 40 character signal would be able to see that it is actually

India is the largest democracy. Welcome to India.

This essentially is the difference between the input's seen in a text processing pipeline versus a speech processing pipeline. For this reason several ask to the speech processing community is (Speech Recognition)

Can you give us "India is the largest democracy. Welcome to India." from the speech signal "indiaes thelarzestdemo cracyvel cometw ondia"? 

so that we can go on with out text processing pipeline and do all the glittery stuff in Natural Language Processing. 

A speech processing researcher is looking at  "indiaes thelarzestdemo cracyvel cometw ondia" or "indiaesthelarzestdemocracyvelcometwondia"  while a text processing researcher is looking at "India is the largest democracy. Welcome to India."

The differences "capitalized alphabets, the punctuation's, the word separators, ..." are a boon, served on a silver plate, to a text processing engineer. While they are to be deciphered, with significant effort, from a speech signal for a speech engineer.

As an example, just imagine how easy it is to identify the number of times "India" occurs in  a text signal versus trying to identify "India" in the speech signal "indiaes thelarzestdemo cracyvel cometw ondia" (Keyword spotting).

But a speech signal come with a lot more information than just the linguistic content. It has information about the speaker gender (male, female), speaker age (adult, child), speaker identity (Sunil, ...), speaker state (happy, neutral, stressed, ....), speaker accent (Indian, Australian, ...), speaker dialect, speaker health (dysarthric, ...) ... which most often does not accompany the text signal "India is the largest democracy. Welcome to India." 

However for any transaction to happen between a machine and a human the crucial step is one of trying to get the linguistic content in the speech. For example, a robot in an airport trying to answer questions posed by the travelers requires the robot to convert speech into text perfectly. And for this reason the priority and focus, in the speech community, has been on exploiting only the linguistic content in the speech signal (Speech Recognition) and not so much on any other information that is abundantly and uniquely present only in the speech signal.

A robot that tries to answer information about Bangalore airport in India. The first part is the ability to understand what the human is "asking", namely to convert speech signal into text.
 

For this reason it is natural that concepts and algorithms that pass the test or performance on text strings (signal) are adopted (given a try!) in speech signal processing.

For those of who who are curious. For a visual feel as to why speech signal is complex, please see complexity of speech.


Sunday, March 07, 2021

Why is it hard to recognize Pathological Speech?

 “All happy families are alike; each unhappy family is unhappy in its own way.” 

-- Leo Tolstoy , Anna Karenina

Automatic Speech Recognition (ASR) or Speech Transcription (ST) is the process of converting human speech into text. Thanks to the availability of abundant speech data and the powerful processing power in the form of GPS and  significant strides made in Deep Machine Learning the process of speech transcription seems to have been solved. The cloud biggies have made it a commodity and have greatly packaged it making it a desirable toy (read smart speakers) to have. If you are wondering which toy? Well it is the Echo's, Home's. 

Productization has had a free learning curve, understanding what people want by throwing free "search what you want" interfaces. These people behaviour on the web comes in handy to build reliable ASR's under the hood of smart speakers. The ASR performance get's even more enhanced when they know more about you (personal information) so that they hardly make mistakes when you say "Call my Mom" on a Saturday 1500 hrs because this is what you do as part of your weekly routine. This is not to take away the advanced in speech signal processing, but these kind of human behaviour understanding does help in the over all enhanced performance of ASR and hence user experience making us all believe that the ASR is a solved problem. 

The fact that most of us a clueless as to what to do when we are faced with a screen-less interface (as human we expect to be given a hint as to what a machine interface is capable of doing for you) and the hint come in the form of /You can ask ...../. Eight out of 10 people seem to pick the  top 2 hints, making it even more easier the smart speaker to seem smarter than they are!

The ASR is one of the important components of a smart speaker and is generally build by a process of what is called "training". Basically you throw in a lot of speech data and the corresponding text and using some well know techniques (Machine Learning) learn the association between the speech and the text. When you throw hours and hours of data, these techniques are capable of associating patterns in speech with text (generally called learning).

Now here is the catch. The ability to associate is dependent on the data that you feed. While there could be variations in the speech of a man and and women (gender) of accent or dialect the variations is not too much for the pattern of association to be arrived at by the machine learning process. The more the variations, the less chances are the ability of the process to find an audio pattern to associate with the text. This leads to the ASR mis-recognition. 

Most of the cloud biggies have access to normal speech (that is most of us) and for this reason the smart speakers (or the ASR under the hood) seem to recognize us. On the other hand if someone who has an abnormal | pathological voice were to make use of these smart speakers it would fail to recognize them. Why? Because the kind of speech data was never used to train the ASR.

So what is the big deal? Add pathological speech data and train the system to make it inclusive. Two problems. Getting pathological speech data is an issue but more than that there is no similarity (as is seen in normal speech) between pathological speech from two different people. Something along the lines of 

 “All happy families are alike; each unhappy family is unhappy in its own way.” -- Leo Tolstoy 

where normal speech is like happy families more or less alike; however pathological speech is different in its own way making it difficult to find common patterns across pathological speech from different people and associate it with text. Making it damn hard to build systems that can be used by people with defective | abnormal | pathological speech.


Wednesday, January 13, 2016

Speech to Text Using IBM Watson from Command Line

This is more like a personal notes for my reference. If it assists anyone it is a bonus.

Assumed that 

  • You have created an account on blue-mix
  • You have obtained the user-id and password (I will call them your_userid, your_password)
  • You have a speech file called speech.wav 
  • You desire to obtain the transcript in the file transcribed_speech.txt
  • You have installed curl on your machine
  • You are not behind a proxy.com

Then from the command (long) line using curl 


curl -u your_userid:your_password -X POST --header "Content-Type: audio/wav" --header "Transfer-Encoding: chunked" --data-binary @speech.wav https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true > transcribed_speech.txt

should create  the best according to watson speech to text engine transcription of the audio speech.wav in the file transcribed_speech.txt

if you are behind a firewall with p_user as the user-id and p_passwd is the password. Then the following should do the job!


curl --proxy http://p_user:p_passwd@proxy.com:8080 -u your_userid:your_password -X POST --header "Content-Type: audio/wav" --header "Transfer-Encoding: chunked" --data-binary @speech.wav https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true > transcribed_speech.txt

Happy speech to text-ing :-) 
Sample very rudimentary perl script

Also: See how to similarly use Google ASR 

A Lesson Hidden in Two Simple Words

Sometimes profound ideas are hidden in the most ordinary places. Recently, while looking at two common English words, I stumbled upon an int...