4

The convergence and optimality proofs of (linear) temporal-difference methods (under batch training, so not online learning) can be found in the paper Learning to predict by the methods of temporal differences (1988) by Richard Sutton, specifically section 4 (p. 23). In this paper, Sutton uses a different notation than the notation used in the famous book ...


3

writing here my suggestion, because i haven't earned the right to comment yet. Your main "problem" could be your loss function. It converges, this is why your loss value is decreasing. So I suggest to let it maybe train longer. Alternatively you could change the loss function to fit your need. For example you could use: loss = tf.reduce_mean(tf.square(...


3

The inputs that you describe seem like they should be sufficient for a DQN-based agent to learn a good strategy for playing Minesweeper, regardless of whether or not the starting layout changes. The inputs contain all information that is necessary. However, the problem certainly becomes much easier (probably too easy) if the initial problem is always the ...


2

It looks like you have some common misconceptions about AI and neural networks. First, AI programs generally do not try to imitate the human behaviour of a human brain. Instead, they try to imitate some higher-level behaviour. For example, they might imitate the reasoning process that you go through when you make a plan. In this context, the building-blocks ...


2

Which is more important, doubt or reinforcement? The single-sentence answer to this would be: it depends. The core of this question seems to be very closely related to the well-known trade-off between exploration (similar to how you describe "doubt") and exploitation (similar to how you describe "reinforcement"). It is almost never the case that someone ...


2

Has this been done? Difficult to prove a negative, but I suspect although plenty of research has been done into finding ideal learning rate values (the need for learning rate at all is an annoyance), it has not been done to the level of suggesting a global function worth approximating. The problem is that learning rate tuning, like other hyperparameter ...


2

When formulating a problem in deep learning, we need to come up with a loss function, which uses model weights as parameters. Back-propagation starts at an arbitrary point on the error manifold defined by the loss function and with every iteration intends to move closer to a point that minimises error value by updating the weights. Essentially for every ...


1

See here for a potential way to do it: http://infinity77.net/global_optimization/#motivation-motivation http://infinity77.net/global_optimization/#rules-the-rules You basically test the two (or more) optimization algorithms against known objective functions, with several random (but repeatable) starting points and then analyze the outcome.


1

There are different actor-critic (AC) algorithms with different convergence guarantees. For example, AC algorithms where the critic is tabular have different convergence guarantees than AC algorithms where the critic is a neural network (function approximation). Most convergence proofs assume that the actor and the critic operate at different time scales, ...


1

See the paper On the Convergence Properties of the Hopfield Model (1990), by Jehoshua Bruck. In the first section of the paper, J. Bruck describes the Hopfield network (popularized by J. J. Hopfield in 1982 in his paper Neural networks and physical systems with emergent collective computational abilities, hence the name of the network), then he describes ...


1

I tried to play with your code and found changing loss function to the cross_entropy alternate of negative log-likelihood makes the difference between 2000th epoch's loss and 9000th epoch's loss is greater about 0.2 alternate of 0.09 I also tried to change optimizer and learning rate but no loss didn't improve. you can explore the modified code may help ...


1

Regression for models more complex than $y = a x + b$ is a convergence strategy. Surface fitting algorithms, such as Levenberg–Marquardt, are often successful at achieving regression using a damped version of least squares as an optimization criterion. The marriage of regression and the multilayer perceptron, an early model artificial network, led to the use ...


Only top voted, non community-wiki answers of a minimum length are eligible