As a researcher, I am getting interested in deep learning (as everyone else:)), and I decided to start with the variational eutoencoders, since I am more interested in unsupervised than supervised learning. I have already read tutorials on the general idea of the mechanism of VAE, and what people usually suggest is to use TensorFlow, Keros, etc. to implement them. However, I do not just want to use machine learning, but also want to develop novel algorithms on top of the existing ones, so I want to deeply understand the mathematical details. And in my experience, in order to understand them, reading is not enough, you need to implement. Any suggestions on where/how to start? I have a general machine learning and programming background, but no experience on deep learning. I even don't know what exactly back-propagation is used for, if this would help.


I even don't know what exactly back-propagation is used for, if this would help.

Programming backprop manually is one of the best exercises in machine learning. In fact, to do this you don't need a special package, like Tensorflor or Keras, just a general computing library like NumPy (and a whiteboard).

Though I dare to say that VAE is not the best thing to implement, if you are not 100% familiar with backprop. It's pretty advanced technique and you risk to mix up fundamental (e.g. backprop) and specific ideas (e.g. deliberately crafted loss function) and slow down your understanding.

I really think that writing a simple feed-forward neural net (including forward and backward passes) is a very good starting point, after which you can move on to VAE details, if you'd like.

Any suggestions on where/how to start?

If you feel confident with algebra and calculus, I highly recommend this Stanford course, especially its assignments. The course itself is about computer vision and convolutional neural networks, but the first assignments are very general and will give a lot of insight of what and how neural networks are doing.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.