Questions tagged [convergence]

For questions related to the convergence of AI algorithms.

15 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
44 views

Convergence of a delayed policy update Q-learning

I thought about an algorithm that twists the standard Q-learning slightly, but I am not sure whether convergence to the optimal Q-value could be guaranteed. The algorithm starts with an initial ...
3
votes
1answer
57 views

REINFORCE algorithm for portfolio optimization - problem while training

I'm trying to implement the Reinforce algorithm (Monte Carlo policy gradient) in order to optimize a portfolio of 94 stocks on a daily basis (I have suitable historical data to achieve this). The idea ...
3
votes
0answers
29 views

How do we give a kick start to the Facenet network?

I read the Facenet paper and one thing I am not sure about (it might be trivial and I missed it) is how do we give the kick start to the network. The embeddings in the beginning are random, so ...
3
votes
0answers
36 views

How to know when a Environment will yield a deterministic model

Given enough experiment data on time taken for objects to fall to earth from different heights, one can create various models that will accurately predict the time it will take for an object falling ...
2
votes
0answers
29 views

If the minimum Q value is decreasing and the maximum Q value increasing, is this a sign that dueling double DQN is diverging?

I'm training a dueling double DQN agent with prioritized replay buffer and notice that the min Q values are decreasing, while the max Q values are increasing. Is this a sign that it is diverging? ...
2
votes
1answer
78 views

Why does the error of my LSTM not decrease after 10 epochs?

Despite the problem being very simple, I was wondering why an LSTM network was not able to converge to a decent solution. ...
2
votes
0answers
42 views

How to show Monte Carlo methods converge to an estimate which minimizes mean squared error?

In chapter six of Sutton and Barto (p.128), they claim Monte Carlo methods converge to an estimate minimizing the mean squared error. How can this be shown formally? Bump
2
votes
0answers
27 views

Are there reinforcement learning algorithms that ensure convergence for continuous state space problems?

The Q-learning does not guarantee convergence for continuous state space problems (Why doesn't Q-learning converge when using function approximation?). In that case, is there an algorithm which ...
1
vote
0answers
20 views

Why is it hard to prove the convergence of the deep Q-learning algorithm?

Why is it hard to prove the convergence of the DQN algorithm? We know that the tabular Q-learning algorithm converges to the optimal Q-values, and with a linear approximator convergence is proved. ...
1
vote
0answers
29 views

When does Monte Carlo linear function approximation converge?

In this Stanford lecture (minute 35:47 and 37:00), the professor says that Monte Carlo (MC) linear function approximation does not always converge, and she gives an example. In general, when does MC ...
1
vote
0answers
21 views

Does SARSA(0) converge to the optimal policy in expectation if the Robbins-Monro conditions are removed?

The conditions of convergence of SARSA(0) to the optimal policy are : The Robbins-Monro conditions above hold for $α_t$. Every state-action pair is visited infinitely often The policy is greedy with ...
1
vote
1answer
161 views

Is there a simple proof of the convergence of TD(0)?

Does anybody know a simple proof of the convergence of the TD(0) value function prediction algorithm?
1
vote
0answers
25 views

Imposing contraints on sequence of image classifications

Are there example implementations of networks that apply constraints across sequences of image classifications where class labels are ordinal numbers? For example, to cause the output of a CNN to ...
0
votes
1answer
32 views

Is it a good idea to overfit on a small part of your data for faster model convergence?

I working on a classification problem that needs to detect patterns on a time serie. Basically, there's a catch-all class that means "no pattern detected", the other are for the specific patterns. The ...
-2
votes
1answer
1k views

RNN LSTM not converging with Adam

I am trying to train a RNN with text from wikipedia but I having having trouble getting the RNN to converge. I have tried increasing the batch size but it doesn't seem to be helping. All data is one ...