TY - GEN
T1 - Learning Geometry of Pose Image Manifolds in Latent Spaces Using Geometry-Preserving GANs
AU - Liang, Shenyuan
AU - Beaudett, Benjamin
AU - Turaga, Pavan
AU - Anand, Saket
AU - Srivastava, Anuj
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
PY - 2025
Y1 - 2025
N2 - The goal of this paper is to learn the differential geometry of pose image manifolds for 3D objects. Indexed by the rotation group SO(3), a pose manifold constitutes images of a 3D object from all viewing angles. Learning geometry implies computing geodesics, intrinsic statistics (means, etc), and curvatures on estimated manifolds. As these goals are unattainable in the huge image space, we perform dimension reduction that is geometry preserving and invertible. This paper introduces two distinct concepts: (1) A Geometry-Preserving StyleGAN (GP-StyleGAN2) that maps training images to a low-dimensional latent space with two novel geometry-preserving terms. These terms penalize changes in pairwise distances between points and pairwise angles between tangent spaces under the map. (2) Densifying the estimated manifold in latent space using Euler’s Elasticae-based nonlinear interpolations between sparse data points. In contrast to the past findings, the latent pose manifolds are found to be distinctly nonlinear and similar in shape across objects. Incorporating these features results in superior performance in image interpolation, denoising, and computing image summaries when compared to state-of-the-art GANs and VAEs.
AB - The goal of this paper is to learn the differential geometry of pose image manifolds for 3D objects. Indexed by the rotation group SO(3), a pose manifold constitutes images of a 3D object from all viewing angles. Learning geometry implies computing geodesics, intrinsic statistics (means, etc), and curvatures on estimated manifolds. As these goals are unattainable in the huge image space, we perform dimension reduction that is geometry preserving and invertible. This paper introduces two distinct concepts: (1) A Geometry-Preserving StyleGAN (GP-StyleGAN2) that maps training images to a low-dimensional latent space with two novel geometry-preserving terms. These terms penalize changes in pairwise distances between points and pairwise angles between tangent spaces under the map. (2) Densifying the estimated manifold in latent space using Euler’s Elasticae-based nonlinear interpolations between sparse data points. In contrast to the past findings, the latent pose manifolds are found to be distinctly nonlinear and similar in shape across objects. Incorporating these features results in superior performance in image interpolation, denoising, and computing image summaries when compared to state-of-the-art GANs and VAEs.
KW - Elasticae
KW - Geodesics
KW - Geometric GAN
KW - Latent Space Geometry
KW - Manifold Learning
KW - Pose Image Manifold
UR - http://www.scopus.com/inward/record.url?scp=85211781465&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85211781465&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-78398-2_4
DO - 10.1007/978-3-031-78398-2_4
M3 - Conference contribution
AN - SCOPUS:85211781465
SN - 9783031783975
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 56
EP - 72
BT - Pattern Recognition - 27th International Conference, ICPR 2024, Proceedings
A2 - Antonacopoulos, Apostolos
A2 - Chaudhuri, Subhasis
A2 - Chellappa, Rama
A2 - Liu, Cheng-Lin
A2 - Bhattacharya, Saumik
A2 - Pal, Umapada
PB - Springer Science and Business Media Deutschland GmbH
T2 - 27th International Conference on Pattern Recognition, ICPR 2024
Y2 - 1 December 2024 through 5 December 2024
ER -