Title Visual Odometry based on RGB-D Images

 

In the past years, we have developed approaches to visual odometry using only RGB image sequences. One result is that such approaches that estimate the camera motion directly from RGB images through matching intermediate features (such as SIFT or SURF). However, matching results of the current approaches suffered from large camera motion, lighting conditions, noises in indoor environments, that causes a large drift in the reconstructed trajectories [1]. Recently, novel RGB-D cameras (e.g., Microsoft Kinect, Primisense, SoftKinect) that provides both color and dense depth images have been widely used. We are interested in using RGB-D cameras for 3D mapping and localization, path planning, navigation. In this research project, we will carefully examine approaches that serve a major role in combining RGB and Depth for visual odometry algorithms.

 Visual odometry based on image sequence.
Left panel: original RGB image sequence; Top-right panel: features matching; Right-bottom panel: reconstructed trajectory

  

Visual odometry based on RGB-D sequence [2]. Left panel: original image sequence;
Right panel: the reconstructed trajectories including texture of scene

The research project will calculate visual odometry using Microsoft Kinect incorporating depth information into RGB color information to generate 3D feature points based on conventional descriptors, such as SURF, SIFT descriptors. In particular, the generated 3D feature points are used for calculating the iterative closest point (ICP) algorithm between successive images from the sensor. The ICP algorithm works based on image information of features differently from previous approaches. This research project also interesting in visual odometry method wichout feature matching, such as works in [3]. We also propose the modified versions for a state-of-the-art implementation of the ICP algorithm (such as generative ICP). Such an approach makes accurate calculation of the rigid transformation matrix for visual odometry in a dynamic environment. From this calculation step, dynamically moving features can be separated into outliers. Then, the outliers are filtered with random sample consensus (RANSAC) algorithm for accurate calculation of the rigid transformation matrix. There are great expectations that proposed system will lead to creating new applications in the field of 3D perception which is particularly useful for robots operating in indoor environments or under ambiguous conditions.


Work description:

Theory:

  • A survey on Visual Odometry based on RGB and RGB-D image sequences
  • Study on ICP (Iterative closest point) and G-ICP (generative ICP)
  • Study on Point cloud registrations
  • RANSAC and bundle adjustment algrothims

Practice:

  • Image sequence registration based on ICP
  • 3-D reconstruction
  • Kinect MS SDK, libfreenect, openNI library

 Requirements:

The students who have a fairly good knowledge about image processing and C++ programming are privileged. 

 

Student prerequisites

This subject is dedicated to Vietnamese students as well as foreigner students at Master degree of Signal and Image processing option. 

   
Supervisors/contacts

Hai Vu, Computer Vision Department, MICA. Email: hai.vu at mica.edu.vn

 

References:

[1] David Van Hamme et al, “Robust Visual Odometry Using Uncertainty Models” ACIVS 2011

[2] http://airobots.ing.unibo.it/

[3] Frank Steinbrücker et al, “Real-Time Visual Odometry from Dense RGB-D Images”, ICCV 2011