Monday, August 23, 2010

Clicking with Windows Speech Macros - Update

My trip down speech recognition lane started with discovery of the free Camera Mouse program (cameramouse.org). I started using the Camera Mouse to control the mouse pointer, but you could not click, at least not easily. I experimented with various ways of accomplishing a click, and settled on using Windows 7 Speech Recognition and speech macros you can install. (This PDF and other downloads at: http://cameramouse.org/downloads.html.)

After several weeks I began to try different ways of improving how the macros work. Here is an update to the mouse click macros available from cameramouse.org. If you want to jump right into them, scroll to the bottom of this post.

Keywords Used

Because Windows Speech Recognition is trying its best to understand everything you say as a command, it was tough to find a unique word that means "click". This is because, if you use "click" as your word to listen for, Windows Speech Recognition will often confuse it with other commands of its own. In the end I chose "iclick" because it was different enough, but even then it wasn't 100% foolproof, and sometimes Windows thought I said "iright" instead of "iclick".

I've been a devourer of the MSDN speech documentation (http://code.msdn.microsoft.com/wsrmacros/) and Rob's Rhapsody (http://code.msdn.microsoft.com/wsrmacros/). I soon discovered a seemingly undocumented speech macro feature called "priority" which seems to tell Speech Recognition to make your macros more important than what Windows wants to do.

Because of the priority attribute, I'm able to use the words "click", "right-click" and "double-click" in my macros and they work almost flawlessly. Example left click:

<command priority="100">
<listenFor>?mouse click</listenFor> <mouse button="left" command="click" />
</command>





What's the question mark for?

You'll notice that the "listenFor" definition has "?mouse click". The question mark indicates that the word is optional. This is because when I'm "talking" to my computer, I sometimes say "mouse click" and sometimes just "click". You might want to adjust the "listenFor" to your own habits. For example, if you say "left-click" often, you could write:

<listenFor>?left click</listenFor>


What else is new?


I added a triple-click command. This was accomplished by clicking three times with a slight delay between each click.

I also added a Ctrl-Click command, which means you can open multiple windows in a browser (for example) without touching the keyboard. This is done by adding a modifier attribute to the mouse statement:

<mouse button="left" command="click" modifierKeys="^" /> 


My current mouse macros

Note: Be sure to replace "[YOURPATHTOAUTOHOTKEYMACROS]" with your path to the two autohotkey speech macros, which are separate macros used to perform the hold down and release mouse button actions.


<command priority="100"> <listenFor>?mouse click</listenFor> <mouse button="left" command="click" />
</command>

<command priority="100">
<listenFor>?mouse control-click</listenFor> <mouse button="left" command="click" modifierKeys="^" />
</command>

<command priority="100">
<listenFor>?mouse double-click</listenFor> <mouse button="left" command="dblclick" />
</command>

<command priority="100">
<listenFor>?mouse triple-click</listenFor> <mouse button="left" command="click" />  <waitFor seconds="0.05" /> <mouse button="left" command="click" />  <waitFor seconds="0.05" /> <mouse button="left" command="click" />
</command>

<command priority="100">
<listenFor>?mouse context ?menu</listenFor> <listenFor>?mouse right-click</listenFor> <mouse button="right" command="click" />
</command>

<command priority="100"> <listenFor>?mouse hold</listenFor> <run command="[YOURPATHTOAUTOHOTKEYMACROS]\mousedown.ahk" params=""/>
</command>



<command priority="100"> <listenFor>?mouse release</listenFor> <run command="[YOURPATHTOAUTOHOTKEYMACROS]\mouseup.ahk" params=""/>
</command>


As always feel free to send me questions.



10 comments:

  1. Here's a great book on using WSR Macros:

    http://www.emicrophones.com/p-1231-wsrmacros-the-users-guide.aspx

    This is a "must have" for anyone writing WSR Macros.

    -Russ

    ReplyDelete
  2. Hi,

    I'd like to make a Smart Agent using WSRM with a Microphone Array.

    But, because Windows is trying to recognise every sounds in the rooms it becomes mad.

    Is there a way to make it recognise ONLY our macros ?
    - The priority tips is great but not enough for me.
    - On the forum some people says to define macro that will override all windows commands but it seems "painfull" to do this.


    Regards,
    Jp

    ReplyDelete
  3. Thanks for the information. I've been struggling with RSI after years of programming and have been experimenting with similar tools. Camera Mouse was a great suggestion. I tried it a bit and it's really help to cut down on my keyboard use. I've also been trying out "Enable Viacam" works similar and is a bit smoother, but lacks shortcut keys, which might be a show-stopper for me. At any rate, thanks for introducing me to headtracking software!

    ReplyDelete
  4. what programming language is this?
    and the link MSDN windows codes for wsr isnt working:(

    ReplyDelete
  5. You can now download Windows Speech Recognition Macros from here:

    http://www.microsoft.com/en-us/download/details.aspx?id=13045

    The link for the great book on WSR Macros also changed. It is now available here:

    http://www.mymsspeech.com/p/51/wsrmacros-the-users-guide

    ReplyDelete
  6. This comment has been removed by the author.

    ReplyDelete