Testcard and region masking ...

The colour detection looks like it will be quite tricky so lets tackle it first.

I've created a 'testcard' photo using the actual webcam suspended over the Lego bricks, I'll get the Robot arm to pick it up and hold it when the picture is taken to again try and get a fixed point of reference.  The natural / artificial light is going to be a problem however I think there is a light on the Robots arm so I'll use that to keep the light consistent.

Photo of the Lego bricks from the webcam 1280 x 960

Hmm even at maximum resolution with lots of light the picture quality is not great.

Next I had a go creating a 'region mask' over the top of the picture so that I can start to look at specific pixel areas where each Lego brick is, from 0 in the top left to 8 in the bottom right.

Masking out the regions on the picture where the colour detection needs to analyse

Using Google again I've found some example code that analyses a bitmap picture and detects if a selected colour is present.  What I'll do is try and modify it so that rather than searching the entire picture it searches specific regions i.e., 0 through to 8 to see what colour brick is present.

Calculating region 2's coordinates using Microsoft Paint

I'm going to use some example C# code by Hemant Srivastava which detects a colour within an entire image, it managed to find the blue colour so lets have a go detecting the colour in region number 2 by modifying the code to only scan within region 2's coordinates.

See this link for Hemant's example code: 

http://hemant-srivastava.blogspot.co.uk/2012/11/image-color-detector-in-c.html