A machine that can dream

Note: This takes 20-30s to load, supported browsers are Chrome,  Safari 4.0+, 
Firefox 4+, Opera 10.0+. It requires heavy computations.

This is not a video. This is a live Boltzmann machine. On top, the flickering lights are the firing neurons of the machine, and at the bottom you see what it is thinking. This particular machine has been shown thousands of faces, and now it imagines faces when it dreams.

The Boltzmann machine has the remarkable ability of dreaming. It can imagine things that it's never seen before. They were first introduced by Geoff Hinton and Terry Sejnowski as a model of the brain in 1983. However, it was a recent breakthrough in 2006 that finally showed their true potential for real world problems. Nowadays, they are becoming a key component in some state of the art systems for speech recognition and computer vision.

How do they work? Here's a quick explanation focusing on the Restricted Boltzmann Machine (RBM) for simplicity. It is defined by its so-called "energy" function

$$E({\bf v}, {\bf h}) = - \sum\limits_{i,j} v_i h_j w_{ij}$$

This function measures the energy between a sensory input vector \({\bf v}\) and the state of each neuron \({\bf h}\). The parameters \(w_{ij}\) weight correlations in the data. This is used to define the probability

$$p({\bf v}, {\bf h}) = \frac{e^{-E({\bf v}, {\bf h})}}{\sum\limits_{{\bf v}',{\bf h}'} e^{-E({\bf v}', {\bf h'})}}$$

where the denominator is the summation of the energy of all possible configurations of inputs and brain states.

Learning consists of adjusting \(w_{ij}\) to maximize the probability the RBM assigns to what you show it. This will make the neurons detect patterns in the sensory input. Dreaming consists of traveling in probable sensory inputs and brain states using Markov Chain Monte Carlo (MCMC).

If you want to know more about Boltzmann machines and Deep Learning, you should checkout this excellent talk by Geoff Hinton, or you can read this introductory paper by Yoshua Bengio and Yann LeCun.

You can also find here a pythonic implementation of the binary Restricted Boltzmann Machine (RBM) that I wrote.