Robot Lab Spring 2012. Worked on OpenCV and object tracking. Below is a description of the work I've done, how to run my code, and some points on future development. I would suggest running the code alongside reading the descriptions, as this is very visual in nature and seeing the demos will help in understanding.
I have worked on various methods of analyzing input from a single camera, such as a webcam. All of my implementations have been in C++ using OpenCV. The algorithms implemented are as follows:
- Background subtraction
- Optical flow
- Color tracking
- Blob tracking
- Histogram of Oriented Gradients (untested)
I have made a demo of each of the algorithms. Data from each of these demos can be pulled and used for various robot tasks, like movement, mapping, etc. Each demo can simply be run with no arguments. They will pop up one or more windows with a camera image and some other data. Hit the escape key to exit.
All of the source code can be found here: Media:OpenCV.tar
To run it you will need to have OpenCV installed. An extra library is needed for the blob tracking. Installing is explained below.
They can explain it better than me (and keep up to date). Just find your flavor of OS and OpenCV version on that page and follow the instructions.
cvBlob is a library needed for the blob tracking code.
- Project page: http://code.google.com/p/cvblob/
- Installation instructions: http://code.google.com/p/cvblob/wiki/HowToInstall
In a nutshell, this algorithm detects movement by looking at recent changes in the image.
When started, the demo will show a completely black window. When the image changes (meaning there was movement), the parts of the image that changed will show up as white. Over time, it will "learn" a new background, so if the camera turns and the whole screen goes white (because everything moved), then it will adjust to the new background and turn the window black again.
My implementation of this algorithm uses edge detection to find "points of interest" and then tracks these points frame to frame. It also keeps track of how far each point moved between each pair of frames, which is an indicator of perceived speed.
For each tracked point, the demo will draw a line from where the point was the previous frame to where the point is in the current frame. The color of the line depends on how far the point moved, red being not far, green being medium distance, and blue meaning very far.
Assuming all of the tracked points are the same distance from the camera, the length of the line drawn (meaning how far the point traveled) is an indicator of speed. More movement means faster. However, if the points are all different distances from the camera, the line length is a function of speed and distance because distant objects will travel less per frame than close ones moving at the same speed.
Using these properties, it is possible to do some image segmentation based on approximate distance. For example, assuming everything but the camera is stationary, if a set of ten points are moving quickly while the rest of the points are moving slowly, those ten points are likely part of the same object and that object is close to the camera. The same principal can be applied to moving objects. If a set of points are moving in the opposite direction than all of the other points, then they are part of an object moving in a different direction.
A very simple method of determining the location of an object of a certain color. A color range is given, and any pixel within that range is tracked. The location of the object is given as the center of all of the tracked pixels (the average of their x and y coordinates). This can lead to some wonky behavior, such as when there are two trackable objects. In that scenario, it will claim there is a single object with a location directly between the two objects.
The demo will display two windows of interest. One will display a black and white image where the white indicates a pixel within the tracked range and black is an untracked pixel. The second window will paint a line wherever the tracked object is located. The tracked object is the paintbrush.
NOTE: In the code, the color ranges are in HSV format, not RGB. 
Blob Tracking and Location Estimation
A more advanced color tracking algorithm. This one segments the tracked color into "blobs". It then tracks each blob individually. This algorithm also has some approximate location estimation, based on distance from the camera and angle from the camera.
The distance calculation is based on a few factors. Firstly, the size of the object to be tracked needs to be known, and this value is currently hard coded in the demo. Secondly, some calibration needs to be done. The calibration is done by holding the object a set distance from the camera. By knowing how large the object appears at a set distance from the camera, the focal length of the camera can be calculated. The focal length needs to be known in order to perform the distance calculation.
The angle estimation is much simpler, but also needs claibration. By holding the object at a set angle from the front of the camera, we can convert the location of the object to the angle. The program assumes there's a linear rate of change between angle and pixel location.
The demo displays each blob on the screen with a rectangle around it. To calibrate the camera for distance estimation, hold the object one meter away from the camera and press the d key. To calibrate for angle estimation, hold the object at a 20 degree angle to the side of the camera and press the a key.
Once both distance and angle are calibrated, object tracking will start. The tracking is connected to ROS_Interface.cpp. The ROS interface is currently just stubbed out with print statements. It will print how far the robot has to move forward to reach the object or how far it has to turn to be directly facing the object. These stubs can later be