PhD Research

My Research

The reconstruction of 3D objects is an active topic of study for the computer vision community and recently there has been a growing interest in the subject, particularly in the development of 'dense multi-view stereo' reconstruction techniques [Seitz et al. 2006]. These techniques focus on producing 3D models from a sequence of calibrated images of an object. Figure 1 provides an overview of the pipeline of such processes.

Many existing reconstruction algorithms share one or more of the following constraints:

• Images are acquired in calibrated rigs or using other specialist equipment such as turntables.
• Large numbers of images are required.
• Images are taken against simple or known backgrounds to provide object silhouettes.
• A degree of specialist user interaction is required, either to control the algorithm during reconstruction or to perform preparatory work on the system input.

So far, the goal of our research has been to to extend existing algorithms and develop new approaches to overcome these limitations which, in turn, should serve to increase the applications of reconstruction techniques as well as making them more accessible to wider audiences. Towards this end we offer the following contributions:

1. An undemanding and automatic method for camera calibration.
   The calibration of a set of images involves determining a common set of parameter values as well as the position and orientation of the camera for each of the photos taken. Our contribution to this topic is to present an algorithm that requires no input from the user other than the images themselves and a planar surface, such as the newspaper in Figure 2, which presents a very easy process for the user.

Figure 2: Automatic calibration. Only a textured plane (such as a newspaper) is required.
2. An automatic segmentation system for image sequences.
   Many of the existing reconstruction techniques require large numbers of calibrated images and the silhouette of the object in each image to obtain high accuracy reconstructions. Unfortunately, images taken outside of the laboratory will often observe the object against a cluttered background, for example compare the images in Figures 3(a) and 3(b). In this situation, the object must be separated from the background (segmented) manually, representing a sizeable task. Our contribution is an automatic segmentation algorithm for whole image sequences [Campbell et al. 2007, 2010]. Figure 3(c) provides some typical results.
(a) (b)
(c)
Figure 3: The segmentation task. (a) The sequence of images taken from the museum presents a realistic setting for segmentation and reconstruction. This is opposed to the images of (b), taken on a turntable and against a fixed backdrop, used by current state-of-the-art algorithms. (c) Some results of our automatic segmentation algorithm that operates without any user input.
3. An automatic algorithm for accurate reconstruction from a sparse set of images.
   Current state-of-the-art algorithms will recovery accurate 3D shape when provided with large numbers of images (50-100) but under sparse data-sets (10-20 images) they lose accuracy. In order to address this we have introduced a new depth estimation process [Campbell et al. 2008] and shown that, by removing erroneous depth estimates at the start of the algorithm, good performance can be maintained. Figure 4 provides an example of the improvement offered for a single depth-map computed from 3 images.

(a)

(b)

(c)

(d)
Figure 4: Single depth-map result. (a) The 3 images used to produce the depth-map. (b) The result used by existing MVS algorithms. (c) Our result has removed the vast majority of depth outliers, also shown with texture in (d). Images from the fountain dataset in [Strecha et al. 2008].

All these algorithms are intended to operate on images captured on standard digital cameras and the processing is performed automatically. This means that the system may be used without any need for computer vision training or knowledge.

References

[Seitz et al. 2006]   S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski.   A comparison and evaluation of multi-view stereo reconstruction algorithms.   In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2006.

[Hernández 2004]   C. Hernández.   Stereo and silhouette fusion for 3D object modeling from uncalibrated images under circular motion.   PhD thesis, École Nationale Supérieure des Télécommunications, 2004.

[Hernández and Schmitt 2004]   C. Hernández and F. Schmitt.   Silhouette and stereo fusion for 3D object modelling.   Computer Vision and Image Understanding, 2004.

[Strecha et al. 2008]   C. Strecha, W. Von Hansen, L. Van Gool, P. Fua, and U. Thoennessen.   On benchmarking camera calibration and multi-view stereo for high resolution imagery.   In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2008.