Neil Slater
  • Member for 5 years, 4 months
  • Last seen this week
  • Durham, United Kingdom
Can non-differentiable layer be used in a neural network, if it's not learned?
Accepted answer
7 votes

It is not possible to backpropagate gradients through a layer with non-differentiable functions. However, the pooling layer function is differentiable*, and usually trivially so. For example: If an ...

View answer
What is the actual learning algorithm: back-propagation or gradient descent?
7 votes

You can run gradient descent without back propagation, in some cases: Simple structures such as linear or logistic regression, where the gradients can be calculated directly from the inputs and cost ...

View answer
If two perfect chess AI's played each other, would it always be a stalemate or would white win for an inherent first-move advantage?
7 votes

This relates to the concept of "solved games". In general, two player turn-based games with perfect information - of which chess is an example - can result in all three possible outcomes: a forced win ...

View answer
Why not teach to a NN not only what is true, but also what is not true?
7 votes

Yes this is done routinely. For example this is how the YOLO object detection and classifier system works, to give a real-world for example. In YOLO, the "non-object" classification is "background" i....

View answer
Why most imperfect information games usually use non machine learning AI?
Accepted answer
7 votes

A heuristic search using MCTS + minimax + alphabeta pruning is a highly efficient AI planning process. What the AI techniques of reinforcement learning (RL) plus neural networks (NNs) typically add to ...

View answer
Do we need automatic hyper-parameter tuning when we have a large enough dataset?
Accepted answer
6 votes

Unfortunately, even with large amounts of training data, hyperparameter choices can strongly influence the performance of a trained model. What you can usually drop when you have large amounts of ...

View answer
Can neural networks have continuous inputs and outputs, or do they have to be discrete?
Accepted answer
6 votes

Neural networks normally work in continuous spaces. A typical neural network function could be written as $f(\mathbf{x}, \mathbf{\theta}): \mathbb{R}^N \rightarrow \mathbb{R}^M$. That is, a function ...

View answer
What is the relation between an environment, a state and a model?
Accepted answer
6 votes

Environment This is the manifestation of the problem being solved. It might be a real physical situation (a road network and cars), or virtual on a computer (a board game on a computer). It includes ...

View answer
Can ML/DL solve my classification problem?
Accepted answer
6 votes

A simple sanity-check on whether an image classifier can perform a task in theory is: Can a human expert, using the same image plus a list of catgeories that they are familiar with, perform the same ...

View answer
Did Alphago zero actually beat Alphago 100 games to 0?
6 votes

Did AlphaGo and AlphaGo [Zero] play 100 repetitions of the same sequence of boards, or were there 100 different games? There were 100 different games. You can view some example games between AlphaGo [...

View answer
Why does TD Learning require Markovian domains?
Accepted answer
6 votes

The Markov assumption is used when deriving the Bellman equation for state values: $$v(s) = \sum_a \pi(a|s)\sum_{r,s'} p(r,s'|s,a)(r + \gamma v(s'))$$ One requirement for this equation to hold is that ...

View answer
What does the term $|\mathcal{A}(s)|$ mean in the $\epsilon$-greedy policy?
Accepted answer
6 votes

This expression: $|\mathcal{A}(s)|$ means $|\quad|$ the size of $\mathcal{A}(s)$ the set of actions in state $s$ or more simply the number of actions allowed in the state. This makes sense in the ...

View answer
What is the name of an AI whose primary goal is to create a better AI?
6 votes

I don't think there is a single standard word or phrase that covers just this concept. Perhaps recursive self-improvement matches the idea concisely - but that is not specific AI jargon. Very little ...

View answer
What is the "thing" which is trained in AI model training
Accepted answer
6 votes

This answer applies to Machine Learning (ML) part of AI, as that seems to be what you are asking about. Please bear in mind that AI is still a broad church, including many other techniques than ML. ML,...

View answer
After a model has been trained, how do I use it to address the real-world problems?
6 votes

Using a machine learning or AI-powered model once it has been built and tested, is not directly an AI issue, it is just a development issue. As such, you won't find many machine learning tutorials ...

View answer
What is the difference between DQN and AlphaGo Zero?
Accepted answer
6 votes

DQN and AlphaZero do not share much in terms of implementation. However, they are based on the same Reinforcement Learning (RL) theoretical framework. If you understand terms like MDP, reward, return, ...

View answer
How can a reinforcement learning agent generalize if it is trained against only one opponent?
Accepted answer
6 votes

Reinforcement Learning (RL) at its core does not have anything directly to say about adversarial environments, such as board games. That means in a purely RL set up, it is not really possible to talk ...

View answer
Can neural networks be used to prove conjectures?
6 votes

Your idea may be feasible in general, but a neural network is probably the wrong high level tool to use to explore this problem. A neural network's strength is in finding internal representations ...

View answer
How could I use reinforcement learning to solve a chess-like board game?
Accepted answer
6 votes

I would like to use reinforcement learning to make the engine improve by playing against itself. I have been reading about the topic but I am still quite confused. Be warned: Reinforcement learning ...

View answer
What is the difference between vanilla policy gradient with a baseline as value function and advantage actor-critic?
Accepted answer
5 votes

The difference between Vanilla Policy Gradient (VPG) with a baseline as value function and Advantage Actor-Critic (A2C) is very similar to the difference between Monte Carlo Control and SARSA: The ...

View answer
When to use the state value function $V(s)$ and when to use the state-action value function $Q(s, a)$?
Accepted answer
5 votes

The core differences between using $V(s)$ or $Q(s,a)$ are: $V(s)$ cannot be used stand-alone to decide a policy. You either need a separate policy function $\pi(a|s)$ that it is the value function for,...

View answer
How is the AI in 3d games implemented?
Accepted answer
5 votes

Overlap between AI and "Game AI" Nowadays, if you search for AI online, you will find a lot of material about machine learning, natural language processing, intelligent agents and neural ...

View answer
Why can we take the action $a$ from the next state $s'$ in the max part of the Q-learning update rule, if that action doesn't lead to any reward?
Accepted answer
5 votes

I'm using OpenAI's cartpole environment. First of all, is this environment not Markov? The OpenAI Gym CartPole environment is Markov. Whether or not you know the transition probabilities does not ...

View answer
What are the advantages of RL with actor-critic methods over actor-only methods?
5 votes

In general, what are the advantages of RL with actor-critic methods over actor-only (or policy-based) methods? One practical benefit is that critics can use TD learning to bootstrap, allowing them to ...

View answer
If $\alpha$ decreases over time, why is Q-learning guaranteed to converge?
Accepted answer
5 votes

Why is this a convergence criterion? It is because $R$ and $S'$ are stochastic. A large learning rate applied when these values have variance would not converge to mean, but would wander around ...

View answer
Can I apply DQN or policy gradient algorithms in the contextual bandit setting?
Accepted answer
5 votes

MDPs are strict generalisations of contextual bandits, adding time steps and state transitions, plus the concept of return as a measure of agent performance. Therefore, methods used in RL to solve ...

View answer
Is the LSTM component a neuron or a layer?
5 votes

The diagram you show works at least partially for describing both individual neurons and layers of those neurons. However, the "incoming" data lines on the left represent all inputs under ...

View answer
How powerful is OpenAI's Gym and Universe in board games area?
Accepted answer
5 votes

OpenAI's Gym is a standardised API, useful for reinforcement learning, applied to a range of interesting environments many of which you can then access for free with little effort. It is very simple ...

View answer
What is the mathematical definition of an activation function?
5 votes

There is no strict definition of suitability of an activation function for neural networks. Instead there are a number of desirable traits, and functions that don't meet them or come close enough may ...

View answer
Why cannot an AI agent adjust the reward function directly?
5 votes

Why do both approaches prevent the AI agent from changing its reward function at will? In RL for optimal control, the reward function is part of the problem formulation. That is, it describes the ...

View answer
1
2
3 4 5
20