A machine that can dream

Note: This takes 20-30s to load, supported browsers are Chrome,  Safari 4.0+, 
Firefox 4+, Opera 10.0+. It requires heavy computations.

This is a live demo of a type of Boltzmann machine in Javascript. The evolving image at the bottom is what the machine is thinking and the flickering lights are the state of its neurons. This particular machine has been shown thousands of faces, and now it imagines faces when it dreams.

The Boltzmann machines have a remarkable ability similar to dreaming. They were first introduced by Geoff Hinton and Terry Sejnowski as a model of the brain in 1983. They can discover patterns when they are learning from data. And when run in a closed loop they can generate or dream new examples based on what is has learned.




How do they work? The full answer is beyond the scope of this post, but for motivated readers here's a quick explanation focusing on the restricted Boltzmann machine (RBM). It is defined by its so-called energy function

$$E({\bf v}, {\bf h}) = - \sum\limits_{i,j} v_i h_j w_{ij}$$

This function measures the energy between a sensory input vector ${\bf v}$ and the state of each neuron ${\bf h}$. The parameters $w_{ij}$ weight correlations in the data. This is used to define the probability

$$p({\bf v}, {\bf h}) = \frac{e^{-E({\bf v}, {\bf h})}}{\sum\limits_{{\bf v}',{\bf h}'} e^{-E({\bf v}', {\bf h'})}}$$

where the denominator is the summation of the energy of all possible configurations of inputs and brain states.

Learning consists of adjusting $w_{ij}$ to maximize the probability the RBM assigns to what you show it. This will make the neurons detect patterns in the sensory input. Dreaming consists of traveling in probable sensory inputs and brain states using Markov Chain Monte Carlo (MCMC).

If you want to know more about Boltzmann machines and Deep Learning, you should checkout this excellent talk by Geoff Hinton, or you can read this introductory paper by Yoshua Bengio and Yann LeCun.

You can also find here a pythonic implementation of the binary restricted Boltzmann machine (RBM) that I wrote.