Now with Text-to-Speech and Speech Recognition

Adding text-to-speech was equally painless - I used the System.Speech.Synthesis library and added the below lines of code to wherever speech was required.  Now as you say a command it is repeated back via text-to-speech and displays it in the journal.

            // Initialize a new instance of the SpeechSynthesizer.
            SpeechSynthesizer synth = new SpeechSynthesizer();

            // Configure the audio output.
            synth.SetOutputToDefaultAudioDevice();

            // Speak the recognised text passed to the recogniser event.
            synth.Speak(e.Result.Text);

About these videos:

To watch them in synchronisation start Arm Cam at 9 seconds and Computer Cam at 3 seconds.

Arm Cam: Speech controlled robotic arm with text-to-speech feedback

Computer Cam: Speech controlled robotic arm with text-to-speech feedback (no sound)

To help the speech recognition out I was toggling the mic between mute/un-mute and accidentally cut off part of the "wrist up"command, so it thought I said wrist down, quickly toggling the mic on again it picked up part of the text-to-speech repeating the command - this is why you hear "I have no idea what you said" which is triggered on the recognizer.SpeechRecognitionRejected event.  See the blooper at at 1:05 (Arm Cam).

Some links for further reading:

• Click here for text-to-speech
• Click here for speech recognition

If you would like a copy of the full C# source code for my application let me know.
(Just in case you were wondering why the pink desk, its running on my daughters laptop!)

Speech recognition for the robotic arm

While waiting for the step motor to arrive I had the idea of adding speech recognition to the C# project.  It was incredibly simple using the in-build Speech Recognition capabilities of Windows.

In summary;

1. Create a grammar with the phrased command (utterances) such as "light on", "shoulder down"
2. Use the speech recogniser to pick up the spoken commands
3. Parse the plain text result and use a simple Switch statement to run the relevant arm command
4. It really was that simple!

Screenshot of the updated GUI which now accepts voice commands

A short video of the arm moving via voice commands

Some snippets of the code:

 Creating the grammar and running the recogniser

Parsing the recogniser results and calling the corresponding arm movement function

If at first you don't succeed, use YouTube and eBay!

There looks to be a feasible solution to the control precision issue which involves replacing the 5 DC motors with step motors and using a Raspberry Pi (which just happens to have been sitting in my cupboard since the summer) to control them.

Step motor and Pi (Raspberry)

The theory at least is to replace the USB I/O PCB which came with the arm and use the Raspberry Pi instead to translate the movement commands pushed to it from the C# app on the PC.  Michael Horne has created a project to control four step motors, albeit without PC input, in his blog.

Controlling the step motors individually (C)Michael Horne

Before getting too excited though I need to check I can actually get a step motor to fit within the robotic arm, they're currently £5.50 on eBay so it's worth a punt.

Happy New Year!

To start the New Year off with a bang I've discovered a fundamental problem with the Robotic Arm, but more about that in a moment.

The core components 'in a nutshell'

So far I have addressed each of the four core components of this project individually and started to pull together a encompassing C# program to get them to working together.  It was at this point I started to put the arm through it's paces with one of the kids Christmas presents - Marble Madness.

The issue I discovered is despite sending an identical set of commands to the arm the final position was not the same place each time, in fact some axis where hugely off!  The PC I/O is the same each time as verified when using the PC application which came with the arm, it's the actual motors themselves.

Now for the fundamental oversight; the DC motors are of course affected by the weight of the arm, battery condition and arm position i.e., asking a motor to spin for 1 second could result in 10 degrees movement in one instance and 20 degrees movement in another.

The view from the arms claw

The view of the whole arm

Running the tests in the above videos several times resulted in the arm completely missing the marble drop position.

Noooo, now what?!   Maybe I need to take a look at some servos or a step motor to increase accuracy.