Lego Project Update

I've almost got object finding completed.  There's a bug in the algorithm somewhere that is making some interesting results.

I start with this image as input:

I run my code and I get this:

The individual objects that the software finds is color coded.  So all the red pixels above are part of one 'object'.  Same with green and purple.  Notice, however, that there's some strange stuff going on.

One part of the definition of an 'object' is that the pixels have to all be connected.  The software counted three objects in the input image.  I'm sure you notice that there's some funny stuff going on with the red and green pixels on the right.  The reds, for instance, are all part of the same object -- but that stuff on the right clearly isn't connected to the red pixels on the left!  Not sure what's going on here.  I'm using a recursive algorithm to find those pixels that are part of an 'object', but clearly there's something going wrong.  It's hard to see, but there are some green pixels around the purple object, too.

Once I get the object locator code working, I can turn to object identification.  The first thing I'll try to do is identify color.  I'll try to do it by brute force at first (conditional statements in the code like if red and green are nearly equal, then the color is probably yellow), but I'm thinking that this'll be my first opportunity to use a neural network for the color identification.

UPDATE

LOL after writing the above, I realized what I'd done wrong on the code.  I fixed that with these results:

So here are three found objects.  The little black dots show the centers of each one, proving that the code is finding individual objects.

I've jumped forward to writing out centered found objects to individual images.  So I start with this different image:

... and end up with these four extracted images showing four found objects.  These images can be used as input to a trained neural net or whatever:

Object 1

Object 2

Object 3

Object 4

Kinda rough, but not too bad for a first try.

As you can see, it didn't pick up the white piece next to the red.  That's mainly because I'm picking out these objects by measuring their contrast to the background.  A white Lego on a white background makes it hard to pick out.  I'll either need to use a different background (a non-Lego color), or change how I find the individual objects.

So the next step is color identification.  As I said, I'm gonna try hard-coding with RGB thresholding to see if I can measure colors that way.  If not, I'll turn to a neural net solution.

The step after that is object identification.  This seems to be where everyone gets stuck.  I have no idea if I will, so I'm proceeding.

MORE Update

Ahhhh!  I can't stop!  Here's an image with a few more objects.  The code found 8 objects, which I'd say that five are useable for matching.  It's also 1000x1000 pixels -- the largest array sofar.

probably not usable

probably not usable

probably not usable

More more update:

I've taken a new image of some close-packed legos to see how many my software can find.  This is, again, a 1000x1000 array.  It found 41 objects in this image!  Not too bad!

Input Image

Found Objects

LOL looks like it missed all the gray pieces.  Again, because of their closeness to the background.  Each object is marked with a red dot.  Notice all the spaces?

Average colors (dot) of found objects:

Wow this next image is impressive.  Here's each found object with its average color.  Compare this to the input image and the similarity is pretty stricking!