Equivariant Neural Networks and Geometric Priors in Temporal Non-linear ICA

Abstract:

Many real-world processes, such as human motion, molecular dynamics, and neural activity, exhibit inherent geometric structures that standard nonlinear ICA models ignore. Current approaches, including Delta-SNICA [1], rely on unconstrained latent representations, leading to suboptimal generalization and poor interpretability in structured data. Geometric priors and equivariant neural networks [2] provide a principled way to enforce structured transformations, ensuring learned representations respect underlying physical and geometric constraints.

Existing temporal structured nonlinear ICA models primarily capture temporal dependencies but fail to enforce geometric constraints, leading to poor generalization, unrealistic latent trajectories, and nonphysical representations in structured domains like biomechanics and video analysis. By incorporating geometric priors, we can align latent variables with intrinsic smoothness and curvature constraints, stabilize representations over time, improve disentanglement, and ensure temporal coherence—all crucial for structured, data-efficient models.

Recent advances in geometric deep learning [3] and equivariant modeling for motion forecasting [4] demonstrate that Lie group priors (SO(3), SE(3)), low-rank constraints, and symmetry-aware transformations improve model robustness and interpretability. By integrating these techniques into structured nonlinear ICA models, we can address existing limitations and enhance applications such as biomechanical motion tracking, physics-informed machine learning, and video data analysis, where spatial-temporal consistency is critical. While current methods show promising results, incorporating additional geometric structure has the potential to significantly improve model performance and interpretability.

Research Questions:

• How do geometric priors and equivariant constraints affect latent representations in nonlinear ICA?
• Can symmetry-preserving transformations improve generalization and sample efficiency in Delta-SNICA?
• How do different geometric constraints (e.g., Lie groups, low-rank priors) affect temporal modeling?
• Can geometric priors enable disentanglement of independent latent factors in structured data?

Prerequisites:

• Knowledge of geometric deep learning (manifolds, Lie groups, equivariant networks).
• Experience with latent variable models and probabilistic inference.
• Familiarity with time-series analysis and structured ICA methods.
• Experience with deep learning frameworks (PyTorch, TensorFlow).

Contact:

• Mila Coetzee

References:

[1] Halva & Hyvärinen (2021). “Nonlinear Independent Component Analysis is Solved via Nonstationarity.” ICLR.

[2] Cohen & Welling (2016). “Group Equivariant Convolutional Networks.” ICML.

[3] Bronstein et al. (2021). “Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges.” Nature Machine Intelligence.

[4] Zhou et al. (2020). “Learning SE(3)-Equivariant Representations for Motion Forecasting.” NeurIPS.