Thursday, February 16, 2012

A machine that can dream










Note: This takes 20-30s to load, supported browsers are Chrome,  Safari 4.0+, 
Firefox 4+, Opera 10.0+. It requires heavy computations.

This is a live demo of a type of Boltzmann machine in Javascript. The evolving image at the bottom is what the machine is thinking and the flickering lights are the state of its neurons. This particular machine has been shown thousands of faces, and now it imagines faces when it dreams.

The Boltzmann machines have a remarkable ability similar to dreaming. They were first introduced by Geoff Hinton and Terry Sejnowski as a model of the brain in 1983. They can discover patterns when they are learning from data. And when run in a closed loop they can generate or dream new examples based on what is has learned.




How do they work? The full answer is beyond the scope of this post, but for motivated readers here's a quick explanation focusing on the restricted Boltzmann machine (RBM). It is defined by its so-called energy function
$$E({\bf v}, {\bf h}) = - \sum\limits_{i,j} v_i h_j w_{ij}$$
This function measures the energy between a sensory input vector \({\bf v}\) and the state of each neuron \({\bf h}\). The parameters \(w_{ij}\) weight correlations in the data. This is used to define the probability
$$p({\bf v}, {\bf h}) = \frac{e^{-E({\bf v}, {\bf h})}}{\sum\limits_{{\bf v}',{\bf h}'} e^{-E({\bf v}', {\bf h'})}}$$
where the denominator is the summation of the energy of all possible configurations of inputs and brain states.

Learning consists of adjusting \(w_{ij}\) to maximize the probability the RBM assigns to what you show it. This will make the neurons detect patterns in the sensory input. Dreaming consists of traveling in probable sensory inputs and brain states using Markov Chain Monte Carlo (MCMC).

If you want to know more about Boltzmann machines and Deep Learning, you should checkout this excellent talk by Geoff Hinton, or you can read this introductory paper by Yoshua Bengio and Yann LeCun.

You can also find here a pythonic implementation of the binary restricted Boltzmann machine (RBM) that I wrote.

46 comments:

Mentifex said...
This comment has been removed by a blog administrator.
Benedito Ventura said...

Muito bem yann, novo e já com um futuro excepcional só posso parabenisa-lo e desejar-lhe um futuro promissor, pois o mais difícil você já fez. Ventura!!!.

Benedito Ventura said...

Pleasy. Ventura!!!.

Unknown said...

The material of the essay is also extremely crucial. For essays, content that is informative is a need to. Filling the webpages with fluff words merely to meet up with the web page prerequisite will land you a poor evaluation and some severe criticism. Consequently, when you really feel like you have operate out of items to say, attempt carrying out a little bit much more custom research papers study. There is often something else that you forgot to mention, or possibly go back and elaborate on one thing you spent little time on. Filler terms are never an excellent selection.

Simone Bertolucci said...

Amazing and inspiring work, at least to my uninitiated eye. My area of studies is far from machine learning, but I am fascinated by this field and I'm trying to wrap my head around the main concepts in my spare time. Projects like this are my main source of motivation!

kongponleu88 said...

i really thanks for sharing this blog..
goldenslot

Sean Parker said...

Would be a great step towards artificial intelligence.
airport parking gatwick

ganga besant said...

Excellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
Block Chain Training in Bangalore

Block Chain Training in tambaram

Block Chain Training in chennai

Block Chain Training in annanagar

Block Chain Training in pune

Block Chain Training in velachery

sweety ganga said...
This comment has been removed by the author.
amala jst said...

Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.

rpa training in velachery| rpa training in tambaram |rpa training in sholinganallur | rpa training in annanagar| rpa training in kalyannagar

lenin christ said...

I found this informative and interesting blog so i think so its very useful and knowledge able.I would like to thank you for the efforts you have made in writing this article.
Data Science course in Chennai | Data science course in bangalore
Data science course in pune | Data science online course
Python course in Kalyan nagar

Sugi Bala said...

I found your blog while searching for the updates, I am happy to be here. Very useful content and also easily understandable providing.. Believe me I did wrote an post about tutorials for beginners with reference of your blog. 
java training in jayanagar | java training in electronic city

java training in chennai | java training in USA

mouni yoga said...

Thank you a lot for providing individuals with a very spectacular possibility to read critical reviews from this site.
python training in tambaram | python training in annanagar | python training in jayanagar

gowthunan said...

Your very own commitment to getting the message throughout came to be rather powerful and have consistently enabled employees just like me to arrive at their desired goals.
safety course in chennai

gowsalya said...

Thanks for the good words! Really appreciated. Great post. I’ve been commenting a lot on a few blogs recently, but I hadn’t thought about my approach until you brought it up. 
Online DevOps Certification Course - Gangboard
Best Devops Training institute in Chennai

devipriya chitra said...

A very nice guide. I will definitely follow these tips. Thank you for sharing such detailed article. I am learning a lot from you.

rpa training in electronic-city | rpa training in btm | rpa training in marathahalli | rpa training in pune

gowthunan said...

I’d like to write some material for your blog in exchange for a link back to mine. Please shoot me an email if interested. Thanks.
safety course institute in chennai

cynthia williams said...

Learned a lot from your blog. Good creation and hats off to the creativity of your mind. Share more like this.
Blockchain Training Institutes in Chennai
Blockchain course in Chennai
AWS course in Chennai
AWS Certification in Chennai
Python Training in Chennai
Python Classes in Chennai

Anbarasan14 said...

I believe that your blog will surely help the readers who are really in need of this vital piece of information. Waiting for your updates.

French Class in Mulund
French Coaching in Mulund
French Classes in Mulund East
French Language Classes in Mulund
French Training in Mulund
French Coaching Classes in Mulund
French Classes in Mulund West

Sadhana Rathore said...

Thanks for taking time to share this valuable information admin. Really informative, keep sharing like this.
Appium Training in Chennai
Best Appium Training institute in Chennai
Appium Certification in Chennai
Mobile Appium Training in Chennai
Mobile Appium course in Chennai

Aruna Ram said...

Very creativity blog!!! I learned a lot of new things from your post. It is really a good work and your post is the knowledgeable. Waiting for your more updates...
Blue Prism Training Institute in Bangalore
Blue Prism Course in Bangalore
Blue Prism Training Bangalore
Blue Prism Classes in Bangalore
Blue Prism Course in Adyar
Blue Prism Training in Mogappair

LindaJasmine said...

Amazing Post . Thanks for sharing. Your style of writing is very unique. Pls keep on updating.
Spoken English Classes in Chennai
Best Spoken English Classes in Chennai
Spoken English Class in Chennai
Spoken English in Chennai
Best Spoken English Class in Chennai
English Coaching Classes in Chennai

sathyaramesh said...

Selenium training in Chennai
Selenium Courses in Chennai
best ios training in chennai
Digital Marketing Training in Chennai
JAVA J2EE Training Institutes in Chennai
German Classes in Adyar
German Classes in Velachery
German Classes in Tambaram

Vicky Ram said...

Thanks for your contribution in sharing such a useful information. Waiting for your further updates.

Article submission sites
Education

Gautam krish said...

The blog is well written and Thanks for your information.
Advanced JAVA Training
JAVA Training Classes
Core JAVA Certification
JAVA Language Course
Core JAVA Course

pavithra dass said...

I wanted to thank you for this great blog! I really enjoying every little bit of it and I have you bookmarked to check out new stuff you post.
Angularjs course in Chennai
Angularjs Training institute in Chennai
Angular 2 Training in Chennai
DevOps Training
DevOps course in Chennai
DevOps course

Aruna Ram said...

Really it was an awesome article!!! It was so good to read and used to improve my knowledge as updated one, keep blogging.....
Data Science Course in Mogappair
Data Science Training in Annanagar
Data Science Training in Ambattur
Data Science Course in Tnagar
Data Science Training in Saidapet
Data Science Course in Vadapalani

Ananya Krishnan said...

Good job in presenting the correct content with the clear explanation. The content looks real with valid information. Good Work

DevOps is currently a popular model currently organizations all over the world moving towards to it. Your post gave a clear idea about knowing the DevOps model and its importance.

Good to learn about DevOps at this time.


devops training in chennai | devops training in chennai with placement | devops training in chennai omr | devops training in velachery | devops training in chennai tambaram | devops institutes in chennai | devops certification in chennai | trending technologies list 2018

Riya Raj said...

Wonderful blog!!! Thanks for your information… Waiting for your upcoming data.
Ethical Hacking Course in Coimbatore
Hacking Course in Coimbatore
Ethical Hacking Training in Coimbatore
Ethical Hacking Training Institute in Coimbatore
Ethical Hacking Training
Ethical Hacking Course

mercyroy said...

I really thank you for your innovative post.I have never read a creative ideas like your posts.here after i will follow your posts which is very much help for my career.
Selenium Training in Ashok Nagar
Selenium Training in Nungambakkam
Selenium Training in Kelambakkam
Selenium Training in Padur

Safety Professionals said...

I and my friends were going through the nice, helpful tips from the blog then the sudden came up with an awful suspicion I never expressed respect to the website owner for those secrets.
iosh course in chennai

Kayal m said...

Great blog!!! It was very impressed to me. I like so much and keep sharing. Thank you.
Robotics Courses in Bangalore
Automation Courses in Bangalore
RPA Courses in Bangalore
Robotics Classes in Bangalore
Robotics Training in Bangalore
Automation Training in Bangalore

Aruna Ram said...

Great concept! I got more information from your post. Good work and continuing………
SEO Course in Velachery
SEO Training in Chennai Velachery
SEO Training in Tnagar
SEO Training in Omr
SEO Training in Tambaram
SEO Course in Kandanchavadi

sathyaramesh said...


Pretty blog, so many ideas in a single site, thanks for the informative article, keep updating more article.
Best institute for Cloud computing in Chennai
Cloud computing Training Chennai
Cloud computing Training centers in Chennai
Salesforce Course in Chennai
Salesforce developer training in chennai
Salesforce Course

Gautam krish said...

The blog is well written and Thanks for your information.
JAVA Training Coimbatore
JAVA Coaching Centers in Coimbatore
Best JAVA Training Institute in Coimbatore
JAVA Certification Course in Coimbatore
JAVA Training Institute in Coimbatore

Riya Raj said...

Outstanding information!!! Thanks for sharing your blog with us.
Spoken English Institute in Coimbatore
Spoken English Training in Coimbatore
English Training Institutes in Coimbatore
Spoken English Training
Spoken English Course

Kayal m said...

It was very great idea!!! Your post is too good and very helpful for improve myself. Keep going....
Digital Marketing Classes near me
Digital Marketing Training in Chennai Velachery
Digital Marketing Course in Tnagar
Digital Marketing Training in Sholinganallur
Digital Marketing Training in Navalur
Digital Marketing Training in Omr

asha said...

Keep on sharing this post
blockchain training in Bangalore

amsa leka said...

Thanks for your great and helpful presentation I like your good service. I always appreciate your post. That is very interesting I love reading and I am always searching for informative information like this. Well written article Thank You for Sharing with Us pmp training courses online | pmp training fee | | project management training in chennai | project management certification online

Gautam krish said...

Really great information!!! Thanks for your blog. Waiting for your upcoming posts.
German Classes in Coimbatore
German Language Course in Coimbatore
German language Classes in Coimbatore
German Language Course
German Courses in Coimbatore

Praylin S said...

Awesome post! Thanks for sharing. Regards.
Microsoft Dynamics CRM Training in Chennai | Microsoft Dynamics Training in Chennai | Microsoft Dynamics CRM Training | Microsoft Dynamics CRM Training institutes in Chennai | Microsoft Dynamics Training | Microsoft CRM Training | Microsoft Dynamics CRM Training Courses | CRM Training in Chennai

Ananya Krishnan said...

Good job in presenting the correct content with the clear explanation. The content looks real with valid information. Good Work

DevOps is currently a popular model currently organizations all over the world moving towards to it. Your post gave a clear idea about knowing the DevOps model and its importance.

Good to learn about DevOps at this time.


devops training in chennai | devops training in chennai with placement | devops training in chennai omr | devops training in velachery | devops training in chennai tambaram | devops institutes in chennai | devops certification in chennai | trending technologies list 2018

jefrin adams said...

Very impressive to read the post

Blockchain training in chennai

Ram Niwas said...

Nice article thank you for sharing valuable information...

Digital Marketing Training
Digital Marketing Courses

lekha mathan said...

I have gone through your blog, it was very much useful for me and because of your blog, and also I gained many unknown information, the way you have clearly explained is really fantastic. Kindly post more like this, Thank You.
best aviation academy in chennai
cabin crew course in chennai
diploma in airport management course in chennai
airport ground staff training in chennai
medical coding classes in Chennai
Fashion designing courses in Chennai
best interior design courses in chennai

amsa leka said...


Nice blog..! I really loved reading through this article. Thanks for sharing such a
amazing post with us and keep blogging... iot training in chennai | iot training in chennai quora | iot training and placement in chennai | iot training center in chennai | best iot training centre in chennai