|
I am currently working with Energy Models. The common characteristic of these models is that they represent the joint distribution over the variables at hand using an energy function. This might not sound familiar. The most famous instantiation of such models is the Restricted Boltzmann Machine, or as it was initially named the Harmonium. They have been gaining increasing popularity recently for two reasons. First, experimental results are in favor of an approximate learning method for such models, called Contrastive Divergence. Secondly, they allow us to build Deep Architectures which are necessary to produce human-like artificial intelligence. I am developing a C++ library that can implement any kind of Energy Model and learn its parameters given some data using the contrastive divergence learning scheme. I am working on this software along our excellent scientific programmer Bas Terwijn, aka Basman. If you would like to try our beta version please contact me! Good introductory readings and application papers are: Yoshua Bengio's review paper on learning algorithms for deep architectures Hinton, G. E., Osindero, S. and Teh, Y. (2006)
A fast learning algorithm for deep belief nets.
Neural Computation, 18, pp 1527-1554. [pdf]
Movies of the neural network generating and recognizing digits Modeling Human Motion Using Binary Latent Variables
Supplementary Material (including videos and source code)
Graham Taylor, Geoffrey Hinton, and Sam Roweis
Proc. of Advances in Neural Information Processing Systems (NIPS) 19 (2007)
|