Skip to content

My Face Tracking Robot

April 26, 2012

I’ve been playing with the OpenCV (Computer Vision) library for Python in my spare time, and it is pretty fun. It opens up a whole new world of possibilities. Now I can work on motion detection, facial recognition, sentry guns, etc.

Previously, I rigged up a miniature motion-controlled turret using 9g micro servos, cardboard and duct tape.
This time, I got a little more serious and used MG-995 servos and better hardware. The arduino software works exactly the same however, so I could just plug it right in.

That’s cool and all, but I want it to track people or track faces and follow them automatically. So… I mounted a webcam on top of my device and I wrote up some quick arduino code that listens for signals through the serial/USB port and moves it in the appropriate direction.

All the face tracking goodness will happen on the laptop then send signals to the arduino which will then rotate the pair of servos accordingly. Sounds like a plan!

Now, I made 2 different versions of the arduino code and 2 different versions of the python code because I wasn’t sure what would work better.
Face detection requires Haar cascades which require training. For your information, I didn’t need to train my program, I just used cascade files that were available to me. (Already includes full body, frontal face, profile face, upper body, lower body, etc.) Without further training, it has difficulty detecting tilted faces, sideways faces, and upside-down faces.

So I wrote another version that only detects a face initially, then keeps running a template match with the first matched face. This finds the segment in the screen that most closely resembles the face. This now works with upside-down faces, but doesn’t work well with shrinking faces or expanding faces (when you move too close or too far from the camera).

As for the arduino code, I made one that just takes a simple 0 for reset and 1-4 to move x angles up,right,down,left. I also made another one that takes [x-coordinate]x[y-coordinate]y and jumps the servos to that destination.

What I find works best is the pure face detection + directional arduino code.

See it in action!

You can download the source code here:


About these ads

From → Hacks

  1. Do you have the include files for this as well? I assume it is the sentry python files that I would be using?

    • Hi, for this example, I was using:

      Is that what you were asking?

  2. Hey good work man, just curious How about using camshift instead of template matching and in case if face goes out of the screen do a face track again and then switch to camshift. Well i’m not that sure of the speed factors though.. !!

    • Thanks abelpaul. I haven’t tried camshift, but I’ll experiment with it. Thank you for the heads up. I’m looking for better ways to detect faces.

  3. Rob permalink

    What kind of equipment would you need to create one of these? And also, would it be possible to create some sort of robot arm for this face tracker to sit on top, so it can do heights?

Trackbacks & Pingbacks

  1. Artificial Intelligence Applied to Your Drone |

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 1,508 other followers

%d bloggers like this: