Restricted Boltzmann Machine and Deep Belief Network Implementation in C#
based primarily on the papers:
"Learning Deep Architectures for AI" By Joshua Bengio
and "A Practical Guide to Training Restricted Boltzmann Machines" By Geoffrey Hinton.
Code insipred by Edwin Chen's python implementation
RBM.cs - Contains the core code for the Restricted boltzmann machine. The number of lines of code have been minimized to mimic a python implementation.
This gives us clean and readable code.
DeepBeliefNetwork.cs - Is a simple multilayer RBM where the hidden layer is connected to a lower level RBM's visible layer.
Matrix.cs and Vector.cs Classes that help in simplifying the code.
A note about Multidimentional arrays in C# (<=4.0) - These arrays are the most natural way to do matrix multiplications and operations.
However, due to a poor CLR implementation, they work much slower than traditional jagged arrays. Hence, we use Jagged arrays in our Matric.cs implementation, and apply a simple [x,y] indexer to mimic a multidimentional array.
Example:
The project consists of a simple console mode example, where we give our AI many images of handwritten digits ranging from 0 to 9.
We the try to reconstruct several images (encoding and decoding) and we get a relative good approximation of the source.
The next step in the program is "Daydreaming" where we imput random samples and reconstruction yields various random images that depict strong features learned by the machine.