I found a paper about using an Unscented Kalman Filter(UKF) for traning a neural network.
The UKF filter is modified so it works for parameter estimation. Assume that we have a neural network model $\hat d_k = G(x_k, W_k)$ where $G$ is the neural network, $x_k$ is the input vector, $W_k$ is the parameter gain matrix and $\hat d_k$ is the output vector.
This paper counts servral methods to train a neural network.
- Gradient descent
- UKF parameter estimation
So my questions for you are:
What's the benefit for using a kalman filter for training a neural network compared to other optimization algorithms?
I remember that I have been using gradient descent methods and they work, but I have always used regularization. Is that due to the noise in the data?
If I'm using a UKF-filter as parameter estimation, I can avoid regularization then?
Is this UKF parameter estimation algorithm only made for 1 layer neural network, or can it be used for deep neural networks as well?