# Questions tagged [convergence]

For questions related to the convergence of AI algorithms.

71 questions
Filter by
Sorted by
Tagged with
50 views

### Is there a textbook for understanding the proof of convergence for TD learning?

I'm reading the article An Analysis of Temporal-Difference Learning with Function Approximation (1997) on the convergence of temporal difference learning with function approximation, but the ...
35 views

### Hand Landmark Detector Not Converging

I'm currently trying to train a custom model with TensorFlow to detect 17 landmarks/keypoints on each of 2 hands shown in an image (fingertips, first knuckles, bottom knuckles, wrist, and palm), for ...
89 views

### How does $\alpha$ affect the convergence of the TD algorithm?

In Temporal-Difference Learning, we update our value function by $V\left(S_{t}\right) \leftarrow V\left(S_{t}\right)+\alpha\left(R_{t+1}+\gamma V\left(S_{t+1}\right)-V\left(S_{t}\right)\right)$ If we ...
34 views

### Why does TD (0) converge to the MLE solution of the Markov model?

Why does TD (0) converge to the MLE solution of the Markov model? Let's take the Example 6.4 in Sutton and Barto's book as an example. Example 6.4: You are the Predictor Place yourself now in the ...
8 views

### References for the convergence of gradient-based algorithms for training neural networks

I'm looking for some good references that give convergence results of training neural networks. I'm decently familiar with works that analyze the convergence of SGD, and, in particular, I really like ...
17 views

### proof of convergence for the random forest algorithm

I am looking for the proof of convergence of the random forest algorithm. A cursory google search shows many, but I do not understand which version (original?) of the algorithm this is. Can you kindly ...
64 views

### How to check whether my loss function is convex or not?

Loss functions are useful in calculating loss and then we can update the weights of a neural network. The loss function is thus useful in training neural networks. Consider the following excerpt from ...
32 views

### Why is the proof of convergence in the GAN paper not applicable practically?

This question is about generative adversarial networks and restricted to the research paper titled Generative Adversarial Nets. If I select a particular architecture of MLP as a generator and trained ...
33 views

323 views

### When do SARSA and Q-Learning converge to optimal Q values?

Here's another interesting multiple-choice question that puzzles me a bit. In tabular MDPs, if using a decision policy that visits all states an infinite number of times, and in each state, randomly ...
35 views

### How can deep Q-learning converge if the targets may not be correct?

In deep Q-learning, $Q(s, a)$ and $Q'(s, a)$ are predicted or estimated by the neural network itself. In supervised learning, the target value is a true unbiased value. However, this isn't the case in ...
59 views

### If the performance of an RL agent in a partially observable environment is "good", is this likely only accidental?

In my research, I remember to have read that, in case of an environment which can be modeled by partially observable MDP, there are no convergence guarantees (unfortunately, I do not find the paper ...
171 views

34 views

### Is it possible to use deeplearning with spark (with a distributed databases as HDFS or Cassandra)?

If it is possible, will it be really useful or the model will end up converging very early(with a typical optimum learning rate) ? Any content on this topic will be helpful for me.
193 views

### Does TD(0) prediction require Robbins-Monro conditions to converge to the value function?

Does the learning rate parameter $\alpha$ require the Robbins-Monro conditions below for the TD(0) algorithm to converge to the true value function of a policy? \sum \alpha_t =\infty \quad \text{...
841 views

### Is there a simple proof of the convergence of TD(0)?

Does anybody know a simple proof of the convergence of the TD(0) value function prediction algorithm?
327 views

### Why does the error of my LSTM not decrease after 10 epochs?

Despite the problem being very simple, I was wondering why an LSTM network was not able to converge to a decent solution. ...
171 views

### Why does reinforcement learning using a non-linear function approximator diverge when using strongly correlated data as input?

While reading the DQN paper, I found that randomly selecting and learning samples reduced divergence in RL using a non-linear function approximator (e.g a neural network). So, why does Reinforcement ...