VB and Voice Recognition, Part 2

This part covers using the Microsoft Diction Control, and adding it to your previous project to make a full speech-to-text application. Adding the Diction control to the application is almost as easy as adding the Voice commands.

Just a few notes about the Diction control before I start. The diction control has a text buffer that has to be kept up to date with the text in the displayed text box. The current cursor position in the text box has to be updated in the control as well. When dictating, the diction control will update the internal copy and pass only the updated text to the application. You can use a standard text box to work with this method, but a Rich Text Box is recommended.

Now, you add the Diction control to your Project form, and call it 'VoiceDic'. To Initialise the voice diction and set the correct recognition mode, as with the Voice command control, you use the following:

Private Sub Init_Diction()
VoiceDic.Initialized = 1
'Set the Diction control mode to Diction only.
VoiceDic.Mode = VSRMODE_DCTONLY    ' (&H20)
VoiceDic.Activate
End Sub
Note: The Mode property sets how the diction control reacts to any speech. More options for the property, and what they do, are listed in Part 3.

This will start up the Voice diction; any speech that is not part of voice commands will now be passed through the diction. This in turn is passed to the application through an event called 'TextChanged'. The event will tell you only that the internal text of the diction has changed and the reason why. (In other words, New text, Changed text, Replaced text, Deleted text). Within this event, you now have to query the diction's internal text for the changes and apply them to your rich text box. The following is a simplified version of the code.

Private Sub VoiceDic_TextChanged(ByVal Reason As Long)
Dim newStart As Long
Dim newend As Long
Dim oldStart As Long
Dim oldEnd As Long
Dim selstart As Long
Dim sellen As Long
Dim Text As String
' Lock or pause the diction control temporarily.
VoiceDic.Lock
' Request the currently buffered changes.
VoiceDic.GetChanges newStart, newend, oldStart, oldEnd
' Was text deleted?
If (oldStart < oldEnd) Then
   RTBox.selstart = oldStart
   RTBox.SelLength = oldEnd - oldStart
   ' Delete the text from the Rich text box.
   RTBox.SelText = ""
End If
' Was text added?
If (newend > newStart) Then
   RTBox.selstart = newStart
   RTBox.SelLength = 0
   ' Get the new text from the control.
   VoiceDic.TextGet newStart, newend - newStart, Text
   ' Add the new text to the Rich text box.
   RTBox.SelText = Text
End If
' Release the Lock on the Diction Control.
VoiceDic.Unlock
End Sub

Don't be too worried about why the text has changed; simply ignore reason. First, you have to Lock the diction control to prevent it from reactivating and changing the text before you can query the current changes.

The Getchanges method queries the Diction control for the start and end of any changes. Oldstart and OldEnd are references to deleted text, so you select the text and remove it. NewStart and NewEnd are references to added text.

You now set your rich text box to the same start position to insert the new text. Using the TextGet method with the NewStart and NewEnd values returns the text that needs to be inserted to your Rich text box. After doing that, you now can unlock the diction control.

But now, what if the user has manually edited the text or clicked in a new position, to add text in the middle of the document. All these events need to be placed back into the diction control to tell it where to carry on. When the user uses the mouse to select text, you can use the click event to notify the diction control of the new selected text using the TextSelSet method, giving it the start position and length of the selection.

Private Sub RTBox_Click()
' Lock or pause the diction control temporarily.
VoiceDic.Lock
VoiceDic.TextSelSet RTBox.selstart, RTBox.SelLength
' Release the Lock on the Diction Control
VoiceDic.Unlock
End Sub

If the user edits the selected text, you can use the keypress event to pass the new text back into the diction control. The TextSet method updates the internal copy of the text. Its parameters are Text, Start position, Text length, and Reason.

Private Sub RTBox_KeyPress(KeyAscii As Integer)
Dim s As String
' Lock or pause the diction control temporarily.
VoiceDic.Lock
' Set the start of the dictions selected text to the
' same as our Rich Text Box.
VoiceDic.TextSelSet RTBox.selstart, 0
'Place the pressed key character in the correct place
'in the diction control.
VoiceDic.TextSet Chr$(KeyAscii), _     '(&H10000)
   RTBox.selstart, RTBox.SelLength, _ VDCT_TEXTCLEAN
' Release the Lock on the Diction Control
VoiceDic.Unlock
End Sub

By using some of the methods included with the Rich text box, you also can trap Cut, Copy, and Paste events and, by using code simular to the above, place the changes in the Diction Controls Text. Drag and Drop methods also can be added.

Then, you have the problem of some users trying to type and dictate at the same time. You might think this funny, but it does happen. So, what you looking for is a method to either stop the diction control when typing, or prevent typing while dictating. Yor Lock and Unlock methods in the Keypress event take care of pausing the diction control while typing. This still isn't perfect, so you have to look at preventing typing while dictating.

The Diction control gives you two events that you can use to prevent typing during diction—UtteranceBegin and UtteranceEnd. You can use these two events to Enable and Disable the Rich text box whenever there is diction taking place. UtteranceBegin is triggered as soon as the diction control detects a sound, and UtteranceEnd is triggered about .25 - .5 seconds after silence is detected. So, now you can add the following to your code:

Private Sub VoiceDic_UtteranceBegin()
   RTBox.Enabled = False
End Sub

Private Sub VoiceDic_UtteranceEnd()
   RTBox.Enabled = True
End Sub

The example code adds a simple but fully functional Voice diction to your application, but there are not many variations to this. The diction control has the option of returning commands, like the Voice command control does, but only in a single menu set. A few advanced options are available with the Diction control, including Changing and Adding new Speakers (Users) to the Recognition Engine, and calling up the Voice Training Dialogs and automatic User detection. Details of these methods are listed in the next article.

In the download file, there is a simple application that uses what you covered in this article as well as the Voice commands that you covered in Part 1.

The next article will look at some of the other Events and Fuctions available in the Voice Command and Diction controls, and you will add the Text to speech control to your simple application in the last article.

A small note: The diction control requires a minimum of three hours voice training to get an average 90% accuracy; even then, it will make what appears to be major mistakes. The Speech recognition engine uses a phonetic alphabet, where some phonemes are very similar, and the engine also uses a common word grouping system to try and increase the accuracy. If one word is miss-recognised somewhere in the middle of the sentence, the engine may change some of the words prior and those that follow to make an understandable sentence. This can be frustrating at times but, more often than not, it does make correct assumptions.

Just remember to talk a bit slower than normal and to mouth the words correctly. Slang words can cause havoc to the recognition. Use "can not" rather than "can't", "is not" or "am not" rather than "ain't", and "mortuary" rather than "morgue."

Happy Dictating.



About the Author

Richard Newcombe

Richard Newcombe has been involved in computers since the time of the Commodore 64. Today, he has excelled in programming, and designs. Richard is in his mid 30's and, if or when you looking for him look no further than his computer. Always willing to help and give advice where he can in regard to computer related subjects. At present he is working as a .NET 2008 Software Developer for Syncrony Web Services, South Africa.

Downloads

Comments

  • RTBox.selstart help

    Posted by goodevile on 05/22/2009 12:57pm

    RTBox.selstart RTBox.SelLength RTBox.SelText VoiceDic.TextSet Chr(KeyAscii),RTBox.selstart, RTBox.SelLength, _ VDCT_TEXTCLEAN i get error on abouve code can yiu explain what is selText,RTBox.SelLength,RTBox.selstart plz help

    Reply
  • Easy to read, easy to understand

    Posted by JonnyPoet on 01/27/2006 10:22am

    This article is a really very straightlined clear and easy to understand code, which brings a complex theme to everyone who needs.

    Reply
Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Live Event Date: November 20, 2014 @ 2:00 p.m. ET / 11:00 a.m. PT Are you wanting to target two or more platforms such as iOS, Android, and/or Windows? You are not alone. 90% of enterprises today are targeting two or more platforms. Attend this eSeminar to discover how mobile app developers can rely on one IDE to create applications across platforms and approaches (web, native, and/or hybrid), saving time, money, and effort and introducing apps to market faster. You'll learn the trade-offs for gaining long …

  • Live Event Date: October 29, 2014 @ 11:00 a.m. ET / 8:00 a.m. PT Are you interested in building a cognitive application using the power of IBM Watson? Need a platform that provides speed and ease for rapidly deploying this application? Join Chris Madison, Watson Solution Architect, as he walks through the process of building a Watson powered application on IBM Bluemix. Chris will talk about the new Watson Services just released on IBM bluemix, but more importantly he will do a step by step cognitive …

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds