Learning Geometry of Pose Image Manifolds in Latent Spaces Using Geometry-Preserving GANs

Shenyuan Liang, Benjamin Beaudett, Pavan Turaga, Saket Anand, Anuj Srivastava

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The goal of this paper is to learn the differential geometry of pose image manifolds for 3D objects. Indexed by the rotation group SO(3), a pose manifold constitutes images of a 3D object from all viewing angles. Learning geometry implies computing geodesics, intrinsic statistics (means, etc), and curvatures on estimated manifolds. As these goals are unattainable in the huge image space, we perform dimension reduction that is geometry preserving and invertible. This paper introduces two distinct concepts: (1) A Geometry-Preserving StyleGAN (GP-StyleGAN2) that maps training images to a low-dimensional latent space with two novel geometry-preserving terms. These terms penalize changes in pairwise distances between points and pairwise angles between tangent spaces under the map. (2) Densifying the estimated manifold in latent space using Euler’s Elasticae-based nonlinear interpolations between sparse data points. In contrast to the past findings, the latent pose manifolds are found to be distinctly nonlinear and similar in shape across objects. Incorporating these features results in superior performance in image interpolation, denoising, and computing image summaries when compared to state-of-the-art GANs and VAEs.

Original languageEnglish (US)
Title of host publicationPattern Recognition - 27th International Conference, ICPR 2024, Proceedings
EditorsApostolos Antonacopoulos, Subhasis Chaudhuri, Rama Chellappa, Cheng-Lin Liu, Saumik Bhattacharya, Umapada Pal
PublisherSpringer Science and Business Media Deutschland GmbH
Pages56-72
Number of pages17
ISBN (Print)9783031783975
DOIs
StatePublished - 2025
Event27th International Conference on Pattern Recognition, ICPR 2024 - Kolkata, India
Duration: Dec 1 2024Dec 5 2024

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume15327 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference27th International Conference on Pattern Recognition, ICPR 2024
Country/TerritoryIndia
CityKolkata
Period12/1/2412/5/24

Keywords

  • Elasticae
  • Geodesics
  • Geometric GAN
  • Latent Space Geometry
  • Manifold Learning
  • Pose Image Manifold

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Learning Geometry of Pose Image Manifolds in Latent Spaces Using Geometry-Preserving GANs'. Together they form a unique fingerprint.

Cite this