Master thesis proposal 2010-2011
Multi-View Image Segmentation


Multiple View Segmentation
Multiple View Segmentation [LFP08]

Advisors

Edmond Boyer Jean-Sébastien Franco
LJK/PERCEPTION LJK/PERCEPTION
E-mail: Edmond.Boyer@inrialpes.fr E-mail: Jean-Sebastien.Franco@inrialpes.fr
Tel. : 04 76 61 53 54 Tel. : 04 76 61 52 00

Context

Image segmentation is a well known problem in computer vision which consists basically in grouping pixels in the image that share similar properties, usually visual properties such as colors. This principle is employed with several approaches, Kmeans or mean shift for instance [FP02], to produce image regions with homogeneous properties. The interest arises in numerous applications that consider regions instead of pixels to identify objects in the image or for image size reduction, among other purposes. While the monocular case has been extensively studied over the last decades, few works [MD03, ZMPM03, GFP08] consider the case where several images of the same scene are available and for which image segmentation should not be applied independently in the images since they should exhibit regions with similar properties. The purpose of this work is to investigate this problem and to experiment new solutions as described below.

Objectives

Assuming that a set of images describe the same scene, there are 2 types of properties that can be shared by regions among different viewpoints: appearance and geometry. Regions corresponding to the same object in different images could have similar photometric properties and they should define the same 3D region in space where the object lies. This yield additional consistency constraints (photometric and spatial) that can be used when grouping pixels in the images. Of particular interest in our context is the situation where 2 regions only are considered: background and foreground, a typical situation in multiple camera environments and 4D modeling [LWB07]. We will consider however the general multi-view segmentation problem which is of broad interest in the computer vision field. Since few approaches have been proposed to solve for it our objective is to propose innovative models for both spatial and photometric consistency and to experiment them.

The work will include the following tasks:

  1. Review works on image segmentation, feature detection, 3D geometry.
  2. Investigate a first naive solutions that consists in using features matched between pairs of images to propagate spatial consistency constraints over regions using matched points as {\it anchors}. Such spatial consistency constraint can be first purely 2D and second account for 3D cues in the case of calibrated cameras.
  3. Based on the results obtained with the previous solutions propose investigate how to combine photometric and spatial constraints when segmenting over several images.

Pre-requisites

Required skills include computer vision, linear algebra, numerical analysis. Good programming skills are mandatory.

Keywords: computer vision, segmention, image segmentation, multiple view segmentation.

References

  1. [FP02] Computer Vision: A Modern Approach, David A. Forsyth, Jean Ponce, Prentice Hall, 2002.
  2. [MD03] M2tracker: A multi-view approach to segmenting and tracking people in a cluttered scene. A. Mittal and L. S. Davis. International Journal of Computer Vision 2003.
  3. [ZMPM03] 3d reconstruction using labeled image regions. R. Ziegler, W. Matusik, H. Pfister, and L. McMillan. In EG symposium on Geometry processing, 2003.
  4. [LWB07] Identifying Foreground from Multiple Images, W. Lee, W. Woo and E. Boyer, in Asian Conference on Computer Vision (ACCV'07)
  5. [GFP08] Multi-Object Shape Estimation and Tracking from Silhouette Cues, Guan, Li and Franco, Jean-Sebastien and Pollefeys, Marc, in IEEE Conference on Computer Vision and Pattern Recognition 2008.