Search code examples
opencvwebcamtrackinggesture-recognitionhuman-interface

Determine skeleton joints with a webcam (not Kinect)


I'm trying to determine skeleton joints (or at the very least to be able to track a single palm) using a regular webcam. I've looked all over the web and can't seem to find a way to do so.

Every example I've found is using Kinect. I want to use a single webcam.

There's no need for me to calculate the depth of the joints - I just need to be able to recognize their X, Y position in the frame. Which is why I'm using a webcam, not a Kinect.

So far I've looked at:

  • OpenCV (the "skeleton" functionality in it is a process of simplifying graphical models, but it's not a detection and/or skeletonization of a human body).
  • OpenNI (with NiTE) - the only way to get the joints is to use the Kinect device, so this doesn't work with a webcam.

I'm looking for a C/C++ library (but at this point would look at any other language), preferably open source (but, again, will consider any license) that can do the following:

  • Given an image (a frame from a webcam) calculate the X, Y positions of the visible joints
  • [Optional] Given a video capture stream call back into my code with events for joints' positions
  • Doesn't have to be super accurate, but would prefer it to be very fast (sub-0.1 sec processing time per frame)

Would really appreciate it if someone can help me out with this. I've been stuck on this for a few days now with no clear path to proceed.

UPDATE

2 years later a solution was found: http://dlib.net/imaging.html#shape_predictor


Solution

  • At last I've found a solution. Turns out a dlib open-source project has a "shape predictor" that, once properly trained, does exactly what I need: it guesstimates (with a pretty satisfactory accuracy) the "pose". A "pose" is loosely defined as "whatever you train it to recognize as a pose" by training it with a set of images, annotated with the shapes to extract from them.

    The shape predictor is described in here on dlib's website