How to implement a Restricted Boltzmann Machine manually?

Question

I am learning about Restricted Boltzmann Machines and I'm so excited by the ability it gives us for unsupervised learning. The problem is that I do not know how to implement it using one of the programming languages I know without using libraries. I want to implement it manually, which means that I want to use native functionalities of a language as much as possible.

The programming languages I know are Java, C, PHP (my preferred language), JavaScript, R and Python. I am not familiar with TensorFlow or Scikit-Learn or similar stuff.

Thanks in advance

Answer 1

If your interest is to follow the hype and become an "expert" in machine learning so that we can automate the world the rest of the way and our grandchildren can just play golf and video games, then the fast path is to learn TensorFlow, Scikit-Learn, or Keras while making money writing PHP apps for fortune 500 companies.

You can begin by finding some RBM example code that looks like it executes and has some example data or a link to some. Then download whatever of those common frameworks they use, and then study by following the path of least resistance as if you were a machine learning.

No joke. Many will make money from big corporations that way and probably already are. Also if the reason you want to do the RBMs manually is you want to see how they work, this is the way to go. Take apart what others wrote like you would an old lawnmower to learn about internal combustion engines.

If your interest is to learn about the nature of intelligence and consciousness because you like to discover new things, then take the trendy frameworks less seriously and learn what RBMs are and why they are being superseded by other architectures. In that case, learning about probability, calculus, and searching algorighms in 2D, 3D, and higher dimensions (and reading books that most of the trendy people would think were dated) is the best direction.

If you want to see if, in your lifetime, you can create a digital brain that you can teach to fix the lawnmower and then go mow your lawn (also no joke) then learn to code in C and C++ and learn sockets, other systems level programming, and how to access the DSP in a video card because your going to need some blinding speed and parallelism.

Answer 2

If you already know Java, you may wish to investigage Java-ML, Mahout, MLib, and DL4J for AI experimentation and research. Once you have visualizations to publish from your investigation, you can couch them in PHP for web publication. Also, use of RBMs has been giving way to convolution layering and other types of auto-encoding and feature extraction designs.

Answer 3

If you are only in need of implementing for a presentation/learning experience and you aren't doing this in an enterprise capacity you can essentially use any of these languages. Python would be best in my opinion for several reasons, its fast to prototype in, its easy to create visualizations of graphs in python, and if you want to continue in ai development it has great libraries for handling data and most neural net libraries have python api's.

A high level overview of doing this for scratch would be, to first create your node and connection classes as you will be creating a bipartite graph, then create your rbm class which facilitates the construction of the machine by utilizing your node and connection objects.