Harmonic Shape Images: A 3D Free-form Surface Representation and Its Applications in Surface Matching

Abstract

Surface matching is the process of determining whether two surfaces are equivalent in terms of shape. The usual steps in surface matching include finding the correspondences on the two surfaces being compared and computing the rigid transformation between the two surfaces. The issue of surface matching not only has theoretical importance but also has wide applications in the real world. Unlike the matching of 2D images, the matching of 3D free-form surfaces is a much more difficult problem due to the shape and topology complexity of 3D surfaces. In general, there is no simple representation such as a matrix for 2D images that can be used to compare 3D surfaces. Issues such as surface sampling resolution, occlusion and high dimension of the pose space further com-plicate the problem. In this thesis, a novel representation called Harmonic Shape Images is proposed for representing the shape of 3D free-form surfaces that are originally represented by triangular meshes. This representation is created based on the mathematical theory on harmonic maps. Given a 3D surface patch with disc topology and a selected 2D planar domain, a harmonic map is constructed by a two-step process which includes boundary mapping and interior mapping. Under this mapping, there is one to one correspondence between the points on the 3D surface patch and the resultant harmonic image. Using this correspondence relationship, Harmonic Shape Images are created by associating shape descriptors computed at each point of the surface patch at the corresponding point in the harmonic image. As a result, Harmonic Shape Images are 2D shape representations of the 3D surface patch. Harmonic Shape Images have some properties that are important for surface matching. They are unique; their existence is guaranteed for any valid surface patches. More importantly, those images preserve both the shape and the continuity of the underlying surfaces. Furthermore, Harmonic Shape Images are not designed specifically for representing surface shape. Instead, they provide a general framework to represent surface attributes such as surface normal, color, texture and material. Harmonic Shape Images are discriminative and stable, and they are robust with respect to surface sampling resolution and occlusion. Extensive experiments have been conducted to analyze and demonstrate the properties of Harmonic Shape Images. The usefulness of Harmonic Shape Images in surface matching is demonstrated using application examples in the real world such as face modeling and mesh watermarking. In addition, the experimental results on recognizing objects in scenes with occlusion demonstrate its successful application in object recognition. Experiments to measure the accuracy of surface registration using Harmonic Shape Images are also described.