I found a paper about using an Unscented Kalman Filter(UKF) for traning a neural network.

The UKF filter is modified so it works for parameter estimation. Assume that we have a neural network model $\hat d_k = G(x_k, W_k)$ where $G$ is the neural network, $x_k$ is the input vector, $W_k$ is the parameter gain matrix and $\hat d_k$ is the output vector.

This paper counts servral methods to train a neural network.

  • Gradient descent
  • Quasi-Newton
  • UKF parameter estimation

So my questions for you are:

  1. What's the benefit for using a kalman filter for training a neural network compared to other optimization algorithms?

  2. I remember that I have been using gradient descent methods and they work, but I have always used regularization. Is that due to the noise in the data?

  3. If I'm using a UKF-filter as parameter estimation, I can avoid regularization then?

  4. Is this UKF parameter estimation algorithm only made for 1 layer neural network, or can it be used for deep neural networks as well?

  • 2
    $\begingroup$ Hello, asking multiple questions in a single post is not recommended. Please try to split the questions into multiple posts. $\endgroup$
    – hanugm
    Sep 14 at 21:53
  • 1
    $\begingroup$ @hanugm Yes! I created a new question with only 1 question. ai.stackexchange.com/questions/31701/… $\endgroup$
    – MrYui
    Sep 15 at 22:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.