A disentangled recognition and nonlinear dynamics model for unsupervised learning

Authors: Marco Fraccaro, Simon Kamronn, Ulrich Paquet, Ole Winther

Authors: Marco Fraccaro, Simon Kamronn, Ulrich Paquet, Ole Winther

Publication date: 2017

Conference: Advances in Neural Information Processing Systems

This paper takes a step towards temporal reasoning in a dynamically changing video, not in the pixel space that constitutes its frames, but in a latent space that describes the non-linear dynamics of the objects in its world.

We introduce the Kalman variational auto-encoder, a framework for unsupervised learning of sequential data that disentangles two latent representations: an object's representation, coming from a recognition model, and a latent state describing its dynamics.

As a result, the evolution of the world can be imagined and missing data imputed, both without the need to generate high dimensional frames at each time step. The model is trained end-to-end on videos of a variety of simulated physical systems, and outperforms competing methods in generative and missing data imputation tasks.


Similar posts

Sign up for insights from raffle

Get the latest resources, events and webinars sent straight to your inbox. You'll learn about AI, customer service, employee engagement and much more with our knowledge-filled newsletter.