Multi-View Reconstruction and Camera Recovery using a Real or Virtual Reference Plane

Reconstructing a 3-dimensional scene from a set of 2-dimensional images is a fundamental problem in computer vision. A systemcapable of performing this task can be used in many applications in robotics, architecture, archaeology, biometrics, human computer interaction and the movie and entertainment industry. Most existing reconstruction approaches exploit one source of information to tackle the problem. This is the motion of the camera, the 2D images are taken from different viewpoints.We exploit an additional information source, the reference plane,whichmakes it possible to reconstruct difficult scenes where other methods fail. A real scene plane may serve as the reference plane. Furthermore, there are many alternative techniques to obtain virtual reference planes. For instance, orthogonal directions in the scene provide a virtual reference plane, the plane at infinity, or images taken with a parallel projection camera. A collection of known and novel reference plane scenarios is presented in this thesis. The main contribution of the thesis is a novel multi-view reconstruction approach us- ing a reference plane. The technique is applicable to three different feature types, points, lines and planes. The novelty of our approach is that all cameras and all features (off the reference plane) are reconstructed simultaneously from a single linear system of im- age measurements. It is based on the novel observation that cameras and features have a linear relationship if a reference plane is known. In the absence of a reference plane, this relationship is non-linear. Thus many previous methods must reconstruct features and cameras sequentially. Another class of methods, popular in the literature, is factorization, but, in contrast to our approach, this has the serious practical drawback that all features are required to be visible in all views. Extensive experiments show that our approach is superior to all previously suggested reference plane and non-reference plane methods for difficult reference plane scenarios. Furthermore, the thesis studies scenes which do not have a unique reconstruction, so- called critical configurations. It is proven that in the presence of a reference plane the set of critical configurations is small. Finally, the thesis introduces a complete, automatic multi-view reconstruction system based on the reference plane approach. The input data is a set of images and the output a 3D point reconstruction together with the corresponding cameras.