Title Hand Gesture Recognition Using Kinect

 

Communicating between human and computer (or human and smart devices) is a natural and intuitive interaction thank to vision-based hand gesture recognition. To archive this goal, computers should be able to precisely recognize hand gestures from an input video, but also is able to process such stream data in real time. However, addressing these requirements are still very challenging problem due to the complexity of hand gestures, and high computational costs in vision algorithms, which make the most current gesture recognition systems unfeasible and inefficient.

 

Since recent advent of the depth camera (MS Kinect, SoftKinect, PrimeSense), some related works show that Kinect sensors can improve the performance of hand detection as well as recognition by combining the color (or intensity) information with the information from the depth camera. This work exploits method to extract the candidate hand regions from the depth image and select the best candidate based on the color and shape feature of each candidate regions.  Then the contour of the selected candidate is determined in the higher resolution RGB image to improve the positional accuracy. Since some heuristics parameters are always pre-determined in above steps, we also argue that a learning scheme, which is to decide correctly parameters, plays an important role to make a robust hand gesture detection/recognition system. We take into account proposing a learning scheme which is to adapt conditions in the environments as well as appearance features such as hand skin, or distance from human to device. The main purpose of the proposed system is that it should be feasible to deploy practical applications, particularly, to control devices in a smart-home such as Television, game console, or smart lighting system.

 Fig. 1: An example of depth and color combination for extracting precisely hand

Work description:

Theory:

  • Study KINECT SDK, and depth/color sensors calibration techniques
  • Study combination of  features of depth and color
  • Study learning scheme to control heuristic parameters
  • Study classifier to decide label of the hand gestures

Practice:

  • A method to take advantages of depth and color features
  • A method embed the results to applications on a smart device (like a android device: smart TV)

 

Student prerequisites

This subject is dedicated to Vietnamese students as well as foreigner students at Master degree of Signal and Image processing option. The students who have a fairly good knowledge about image processing and C++ programming are privileged. 

   
Supervisors/contacts

Dr. Vu Hai: This email address is being protected from spambots. You need JavaScript enabled to view it.