Monday, August 23, 2010

Clicking with Windows Speech Macros - Update

My trip down speech recognition lane started with discovery of the free Camera Mouse program ( I started using the Camera Mouse to control the mouse pointer, but you could not click, at least not easily. I experimented with various ways of accomplishing a click, and settled on using Windows 7 Speech Recognition and speech macros you can install. (This PDF and other downloads at:

After several weeks I began to try different ways of improving how the macros work. Here is an update to the mouse click macros available from If you want to jump right into them, scroll to the bottom of this post.

Keywords Used

Because Windows Speech Recognition is trying its best to understand everything you say as a command, it was tough to find a unique word that means "click". This is because, if you use "click" as your word to listen for, Windows Speech Recognition will often confuse it with other commands of its own. In the end I chose "iclick" because it was different enough, but even then it wasn't 100% foolproof, and sometimes Windows thought I said "iright" instead of "iclick".

I've been a devourer of the MSDN speech documentation ( and Rob's Rhapsody ( I soon discovered a seemingly undocumented speech macro feature called "priority" which seems to tell Speech Recognition to make your macros more important than what Windows wants to do.

Because of the priority attribute, I'm able to use the words "click", "right-click" and "double-click" in my macros and they work almost flawlessly. Example left click:

<command priority="100">
<listenFor>?mouse click</listenFor> <mouse button="left" command="click" />

What's the question mark for?

You'll notice that the "listenFor" definition has "?mouse click". The question mark indicates that the word is optional. This is because when I'm "talking" to my computer, I sometimes say "mouse click" and sometimes just "click". You might want to adjust the "listenFor" to your own habits. For example, if you say "left-click" often, you could write:

<listenFor>?left click</listenFor>

What else is new?

I added a triple-click command. This was accomplished by clicking three times with a slight delay between each click.

I also added a Ctrl-Click command, which means you can open multiple windows in a browser (for example) without touching the keyboard. This is done by adding a modifier attribute to the mouse statement:

<mouse button="left" command="click" modifierKeys="^" /> 

My current mouse macros

Note: Be sure to replace "[YOURPATHTOAUTOHOTKEYMACROS]" with your path to the two autohotkey speech macros, which are separate macros used to perform the hold down and release mouse button actions.

<command priority="100"> <listenFor>?mouse click</listenFor> <mouse button="left" command="click" />

<command priority="100">
<listenFor>?mouse control-click</listenFor> <mouse button="left" command="click" modifierKeys="^" />

<command priority="100">
<listenFor>?mouse double-click</listenFor> <mouse button="left" command="dblclick" />

<command priority="100">
<listenFor>?mouse triple-click</listenFor> <mouse button="left" command="click" />  <waitFor seconds="0.05" /> <mouse button="left" command="click" />  <waitFor seconds="0.05" /> <mouse button="left" command="click" />

<command priority="100">
<listenFor>?mouse context ?menu</listenFor> <listenFor>?mouse right-click</listenFor> <mouse button="right" command="click" />

<command priority="100"> <listenFor>?mouse hold</listenFor> <run command="[YOURPATHTOAUTOHOTKEYMACROS]\mousedown.ahk" params=""/>

<command priority="100"> <listenFor>?mouse release</listenFor> <run command="[YOURPATHTOAUTOHOTKEYMACROS]\mouseup.ahk" params=""/>

As always feel free to send me questions.

Monday, August 9, 2010

Using Windows 7 Speech Recognition - Introduction

Recently I've been trying to use Windows 7 Speech Recognition to control my computer. I can still type but constant soreness is forcing me to find alternate ways to perform actions. I'm a very efficient computer user so the new way of doing things has to be pretty good.

I've tried speech recognition over the years, starting with Macintosh, but nothing quite fit for me.

Since my current employer uses Windows 7, I decided to turn it on and see how well it works.

The short conclusion is, pretty well. Many people are saying that Windows 7 Speech Recognition is just as good as Dragon Naturally Speaking. I've never used Dragon so I can't say. My needs are also different in that I just want to control my computer at the moment; I don't want to try to use dictation (though it's a goal down the road).

The essential part of the solution for me was the installation of Windows Speech Macros. Why these aren't just included with Windows 7 I can't hazard a guess, but they go a long way toward tailoring speech control to your needs.

Basic Setup Instructions
These instructions work for Windows Vista and Windows 7.
  1. Connect a microphone to your computer.
  2. Turn on Speech Recognition (Control Panel >  Speech Recognition, click Start Speech Recognition) and follow the wizard.
  3. Follow the speech tutorial and try out some commands to get the hang of it. For example, you can say “Show Desktop” to minimize all windows, or “Start” to “click” the Start menu.

Commands I commonly use 'out of the box':
  • Show Desktop
  • Close Window (actually closes the entire application, even if multiple tabs are open)
  • Minimize Window
  • Maximize Window
  • Open Control Panel (or anything else in the Start menu)
  • Copy, Cut, Delete, Paste, Undo, OK, Cancel
  • Any command available in an Windows Explorer tab bar (such as New Folder, but it has to be visible)
  • Click [button] - example, if a dialog box is visible, I can say Click Save.

Tips and Gotchas
  • Lower your expectations. Don't try to control the entire UI right from the start. Get used to making sure a few commands work before you try to use a few more.
  • If you say something that has multiple targets given the current context, you will see numbers appear on screen. If you see these, say the number out loud. The number will change to OK, and then you say OK to activate the command. For example, if you are in Microsoft Word 2007, and the Home tab is visible, and you say Paste, there are actually two paste commands available.
  • Controlling your computer with verbal commands is a cognitive challenge once you get into it. You may have noticed that you find it easier to perform commands when given a list of options - you are relying on recognition of a command and you are not having to recall it. However, with voice commands, you have to recall everything, which means it gets harder as you try to do more. I keep a list of commands taped to my monitor, and when I've got one down pat I remove it and put something else in its place.

I'll post information on what I've done for Windows Speech Macros to make my life easier in a future blog. Have fun for now!