Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency - ANR - Agence nationale de la recherche Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency

Résumé

Approaches for single-view reconstruction typically rely on viewpoint annotations, silhouettes, the absence of background, multiple views of the same instance, a template shape, or symmetry. We avoid all such supervision and assumptions by explicitly leveraging the consistency between images of different object instances. As a result, our method can learn from large collections of unlabelled images depicting the same object category. Our main contributions are two ways for leveraging cross-instance consistency: (i) progressive conditioning, a training strategy to gradually specialize the model from category to instances in a curriculum learning fashion; and (ii) neighbor reconstruction, a loss enforcing consistency between instances having similar shape or texture. Also critical to the success of our method are: our structured autoencoding architecture decomposing an image into explicit shape, texture, pose, and background; an adapted formulation of differential rendering; and a new optimization scheme alternating between 3D and pose learning. We compare our approach, UNICORN, both on the diverse synthetic ShapeNet dataset - the classical benchmark for methods requiring multiple views as supervision - and on standard real-image benchmarks (Pascal3D+ Car, CUB) for which most methods require known templates and silhouette annotations. We also showcase applicability to more challenging real-world collections (CompCars, LSUN), where silhouettes are not available and images are not cropped around the object.
Fichier principal
Vignette du fichier
unsupsvr_eccv2022.pdf (6.89 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03793867 , version 1 (23-02-2024)

Identifiants

Citer

Tom Monnier, Matthew Fisher, Alexei A. Efros, Mathieu Aubry. Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency. Proceedings of the European Conference on Computer Vision (ECCV), Oct 2022, Tel-Aviv, Israel. 26 p. ⟨hal-03793867⟩
42 Consultations
1 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More