User:Aps43 - Revision history

Aps43 at 00:31, 12 June 2012

2012-06-12T00:31:10Z

Aps43 at 00:24, 12 June 2012

2012-06-12T00:24:27Z

Aps43 at 22:13, 11 June 2012

2012-06-11T22:13:44Z

Aps43: Created page with "Robot Lab Spring 2012. Worked on OpenCV and object tracking. Below is a description of the work I've done, how to run my code, and some points on future development. I woul..."

2012-06-11T21:58:18Z

Created page with "Robot Lab Spring 2012. Worked on OpenCV and object tracking. Below is a description of the work I've done, how to run my code, and some points on future development. I woul..."

New page

Robot Lab Spring 2012. Worked on OpenCV and object tracking. Below is a description of the work I've done, how to run my code, and some points on future development. I would suggest running the code alongside reading the descriptions, as this is very visual in nature and seeing the demos will help in understanding.

=Summary=
I have worked on various methods of analyzing input from a single camera, such as a webcam. All of my implementations have been in C++ using OpenCV. The algorithms implemented are as follows:
* Background subtraction
* Optical flow
* Color tracking
* Blob tracking
* Histogram of Oriented Gradients (untested)

I have made a demo of each of the algorithms. Data from each of these demos can be pulled and used for various robot tasks, like movement, mapping, etc. Each demo can simply be run with no arguments. They will pop up one or more windows with a camera image and some other data. Hit the escape key to exit.

All of the source code can be found here: [[Media:OpenCV.tar]]

To run it you will need to have OpenCV installed. An extra library is needed for the blob tracking. Installing is explained below.

=Installing OpenCV=
Go here: http://opencv.willowgarage.com/wiki/InstallGuide

They can explain it better than me (and keep up to date). Just find your flavor of OS and OpenCV version on that page and follow the instructions.

=Installing cvBlob=
cvBlob is a library needed for the blob tracking code.

* Project page: http://code.google.com/p/cvblob/
* Installation instructions: http://code.google.com/p/cvblob/wiki/HowToInstall

=Background Subtraction=
In a nutshell, this algorithm detects movement by looking at recent changes in the image.

When started, the demo will show a completely black window. When the image changes (meaning there was movement), the parts of the image that changed will show up as white. Over time, it will "learn" a new background, so if the camera turns and the whole screen goes white (because everything moved), then it will adjust to the new background and turn the window black again.

=Optical Flow=
My implementation of this algorithm uses edge detection to find "points of interest" and then tracks these points frame to frame. It also keeps track of how far each point moved between each pair of frames, which is an indicator of perceived speed.

For each tracked point, the demo will draw a line from where the point was the previous frame to where the point is in the current frame. The color of the line depends on how far the point moved, red being not far, green being medium distance, and blue meaning very far.

Assuming all of the tracked points are the same distance from the camera, the length of the line drawn (meaning how far the point traveled) is an indicator of speed. More movement means faster. However, if the points are all different distances from the camera, the line length is a function of speed and distance because distant objects will travel less per frame than close ones moving at the same speed.

Using these properties, it is possible to do some image segmentation based on approximate distance. For example, assuming everything but the camera is stationary, if a set of ten points are moving quickly while the rest of the points are moving slowly, those ten points are likely part of the same object and that object is close to the camera. The same principal can be applied to moving objects. If a set of points are moving in the opposite direction than all of the other points, then they are part of an object moving in a different direction.

=Color Tracking=
A very simple

=Blob Tracking=

=Histogram of Oriented Gradients=

=Points of Robot Integration=

@@ Line 25: / Line 25: @@
 * Project page: http://code.google.com/p/cvblob/
 * Installation instructions: http://code.google.com/p/cvblob/wiki/HowToInstall
 =Optical Flow=
@@ Line 56: / Line 51: @@
 The demo displays each blob on the screen with a rectangle around it.  To calibrate the camera for distance estimation, hold the object one meter away from the camera and press the d key.  To calibrate for angle estimation, hold the object at a 20 degree angle to the side of the camera and press the a key.
-Once both distance and angle are calibrated, object tracking will start.  The tracking is connected to ROS_Interface.cpp.  The ROS interface is currently just stubbed out with print statements.  It will print how far the robot has to move forward to reach the object or how far it has to turn to be directly facing the object.  These stubs can later be
+Once both distance and angle are calibrated, object tracking will start.  The tracking is connected to ROS_Interface.cpp.  The ROS interface is currently just stubbed out with print statements.  It will print how far the robot has to move forward to reach the object or how far it has to turn to be directly facing the object.  These stubs can later be changed to fully integrate with ROS, robot simulation software, or some other code.
 =Histogram of Oriented Gradients=
-=Points of Robot Integration=
+Because it was not working for me, I cannot provide much documentation.  The code is provided as is and for reference purposes.

@@ Line 48: / Line 48: @@
 =Blob Tracking and Location Estimation=
-A more advanced color tracking algorithm.  This one segments the tracked color into "blobs".  It then tracks each blob individually.  This algorithm also has some estimation of...(to be continued)
+A more advanced color tracking algorithm.  This one segments the tracked color into "blobs".  It then tracks each blob individually.  This algorithm also has some approximate location estimation, based on distance from the camera and angle from the camera.
-The demo displays each blob on the screen.
+The distance calculation is based on a few factors.  Firstly, the size of the object to be tracked needs to be known, and this value is currently hard coded in the demo.  Secondly, some calibration needs to be done.  The calibration is done by holding the object a set distance from the camera.  By knowing how large the object appears at a set distance from the camera, the focal length of the camera can be calculated.  The focal length needs to be known in order to perform the distance calculation.
 =Histogram of Oriented Gradients=
 =Points of Robot Integration=

@@ Line 41: / Line 41: @@
 =Color Tracking=
-A very simple
+A very simple method of determining the location of an object of a certain color.  A color range is given, and any pixel within that range is tracked.  The location of the object is given as the center of all of the tracked pixels (the average of their x and y coordinates).  This can lead to some wonky behavior, such as when there are two trackable objects.  In that scenario, it will claim there is a single object with a location directly between the two objects.
-=Blob Tracking=
+The demo will display two windows of interest.  One will display a black and white image where the white indicates a pixel within the tracked range and black is an untracked pixel.  The second window will paint a line wherever the tracked object is located.  The tracked object is the paintbrush.
 =Histogram of Oriented Gradients=
 =Points of Robot Integration=