The goal of this paper is to learn dense 3D shape correspondence for topology-varying objects in an unsupervised manner. Conventional implicit functions estimate the occupancy of a 3D point given a shape latent code. Instead, our novel implicit function produces a part embedding vector for each 3D point, which is assumed to be similar to its densely corresponded point in another 3D shape of the same object category. Furthermore, we implement dense correspondence through an inverse function mapping from the part embedding to a corresponded 3D point. Both functions are jointly learned with several effective loss functions to realize our assumption, together with the encoder generating the shape latent code. During inference, if a user selects an arbitrary point on the source shape, our algorithm can automatically generate a confidence score indicating whether there is a correspondence on the target shape, as well as the corresponding semantic point if there is one. Such a mechanism inherently benefits man-made objects with different part constitutions. The effectiveness of our approach is demonstrated through unsupervised 3D semantic correspondence and shape segmentation.
Figure 1. Given a set of 3D shapes, our category-specific unsupervised method learns pair-wise dense correspondence (a) between any source and target shape (red box), and shape segmentation (b). Give an arbitrary point on the source shape (red box), our method predicts its corresponding point on any target shape, and a score measuring the correspondence confidence (c). For each target, we show the confidence scores of red/green points, and score maps around corresponded points. A score less than a threshold (e.g., 0.2) deems the correspondence as "non-existing"-- a desirable property for topology-varying shapes with missing parts, e.g., chair's arm.
Figure 2. Overview. (a) Given a shape S, PointNet E is used to extract the shape feature code z. Then a part embedding o is produced via a deep implicit function f. We implement dense correspondence through an inverse function mapping from o to recover the 3D shape. (b) To further make the learned part embedding consistent across all the shapes, we randomly select two shapes SA and SB. By swapping the part embedding vectors, a cross reconstruction loss is used to enforce the inverse function to recover to each other.
Figure 3. Correspondence accuracy for 4 categories in the BHCP benchmark. The dashed lines indicate the methods are rotation-invariant and for the unaligned setting.
Figure 4. Dense correspondences in 4 categories. Each row shows one target shape SB (red box) and its pair-wise corresponded 6 source shapes SA. Red regions show non-existing correspondences.
Table 1. Unsupervised segmentation on ShapeNet part.
Figure 5. Qualitative results of our unsupervised segmentation.
Figure 6. (a) 3D semantic correspondence reflecting the contribution of our loss terms. (b) 10 semantic points overlaid with the shape. (c) The t-SNE of the estimated PEVs over 3 training stages. Points of the same color are the PEVs of ground-truth corresponding points in 100 chairs. 10 colors refer to the 10 points in (b). In 3/k, only the 3 elements of PEVs max-pooled for 3-part chair segmentation are fed to t-SNE; 8/k uses extra 5 elements.