20 votes
Accepted

Where can I find the proof of the universal approximation theorem?

There are multiple papers on the topic because there have been multiple attempts to prove that neural networks are universal (i.e. they can approximate any continuous function) from slightly different ...
user avatar
  • 33.8k
16 votes

Are there other approaches to deal with variable action spaces?

Does anyone know any paper regarding this subject? I'm not familiar with any off the top of my head. I do know that the vast majority of Reinforcement Learning literature focuses on settings with a ...
user avatar
  • 9,379
13 votes

Why doesn't Q-learning converge when using function approximation?

Here's an intuitive description answer: Function approximation can be done with any parameterizable function. Consider the problem of a $Q(s,a)$ space where $s$ is the positive reals, $a$ is $0$ or $...
user avatar
12 votes
Accepted

Can supervised learning be recast as reinforcement learning problem?

Any supervised learning (SL) problem can be cast as an equivalent reinforcement learning (RL) one. Suppose you have the training dataset $\mathcal{D} = \{ (x_i, y_i \}_{i=1}^N$, where $x_i$ is an ...
user avatar
  • 33.8k
11 votes

Is there a way to understand neural networks without using the concept of brain?

tl;dr I always like to think of Neural Networks as a generalization of logistic regression. I too don't like that, traditionally, when introducing Neural Networks, books start with biological ...
user avatar
  • 3,093
8 votes

Can a neural network with linear activation functions produce a connection of linear functions?

A linear activation would not be able to separate the data like you have shown, no matter how many layers you throw into the network. If we had multiple linearly activated layers, each feeding into ...
user avatar
7 votes

Why doesn't Q-learning converge when using function approximation?

As far as I'm aware, it is still somewhat of an open problem to get a really clear, formal understanding of exactly why / when we get a lack of convergence -- or, worse, sometimes a danger of ...
user avatar
  • 9,379
6 votes

Which functions can't neural networks learn efficiently?

One of the important qualifications of the Universal approximation theorem is that the neural network approximation may be computationally infeasible. "A feedforward network with a single layer is ...
user avatar
6 votes
Accepted

Why can't neural networks learn functions outside of the specified domains?

The problem you discuss extends past the machine but to the man behind the machine (or woman). ML can be broken down into 3 components, the model, the data, and the learning procedure. This by the way ...
user avatar
  • 2,249
6 votes
Accepted

Smallest possible network to approximate the $sin$ function

Before anything, the function you have wrote for the network lacks the bias variables (I'm sure you used bias to get those beautiful images, otherwise your tanh ...
user avatar
  • 430
5 votes

Is a multilayer perceptron a recursive function?

Inherently, no. The MLP is just a data structure. It represents a function, but a standard MLP is just representing an input-output mapping, and there's no recursive structure to it. On the other ...
user avatar
5 votes
Accepted

What makes multi-layer neural networks able to perform nonlinear operations?

Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.
user avatar
5 votes
Accepted

Why is there a sigmoid function in the hidden layer of a neural network?

Let us suppose we have a network without any functions in between. Each layer consists of a linear function. i.e layer_output = Weights.layer_input + bias ...
user avatar
4 votes

Where can I find the proof of the universal approximation theorem?

"Modern" Guarantees for Feed-Forward Neural Networks My answer will complement nbro's above, which gave a very nice overview of universal approximation theorems for different types of ...
user avatar
  • 495
3 votes

Why doesn't Q-learning converge when using function approximation?

There are three problems Limited capacity Neural Network (explained by John) Non-stationary Target Non-stationary distribution Non-stationary Target In tabular Q-learning, when we update a Q-value, ...
user avatar
3 votes
Accepted

Which machine learning models are universal function approximators?

Support vector machines In the paper A Note on the Universal Approximation Capability of Support Vector Machines (2002) B. Hammer and K. Gersmann investigate the universal function approximation ...
user avatar
  • 33.8k
3 votes

Are there other approaches to deal with variable action spaces?

Another approach that came across was to, assuming the number of different action set $n$ is quite small, have functions $f_{\theta_1}$, $f_{\theta_2}$, ..., $f_{\theta_n}$ that returns the action ...
user avatar
3 votes

Is a multilayer perceptron a recursive function?

Sure, you can define plenty of things we don't generally need to regard as recursive as so. An MLP is just a series of functions applied to its input. This can be loosely formulated as $$ o_n = f(o_{...
user avatar
  • 316
3 votes
Accepted

How can my Q-learning agent trained to solve a specific maze generalize to other mazes?

I'm going to assume here that you're using the standard, basic, simple variant of $Q$-learning that can be described as tabular $Q$-learning, where all of your state-action pairs for which you're ...
user avatar
  • 9,379
3 votes

Is there a way of converting a neural network to another one that represents the same function?

To answer this, it's helpful to consider the notion of a neural network architecture – in this context, we can think of the architecture as being the network depth (i.e. number of layers), width (i.e. ...
user avatar
3 votes
Accepted

Is there a possibility that there is no relationship between some inputs and outputs?

Of course, it's possible to define a problem where there is no relationship between input $x$ and output $y$. In general, if the mutual information between $x$ and $y$ is zero (i.e. $x$ and $y$ are ...
user avatar
3 votes
Accepted

Why are neural networks preferred to other classification functions optimized by gradient decent

You can indeed fit a polynomial to your labelled data, which is known as polynomial regression (which can e.g. be done with the function numpy.polyfit). One ...
user avatar
  • 33.8k
3 votes
Accepted

Why do all states appear identical under the function approximation in the Short Corridor task?

You can choose those states, but is the agent aware of the state it is in? From the text, it seems that the agent cannot distinguish between the three states. Its observation function is completely ...
user avatar
3 votes
Accepted

How do we derive the expression for average reward setting in continuing tasks?

We assume that our MDP is ergodic. Loosely speaking, this means that wherever the MDP starts (i.e. no matter which state we start in) or any actions the agent takes early on can only have a limited ...
user avatar
3 votes

Can stochastic gradient descent be properly used in any sample based learning algorithm in Reinforcement Learning?

First I will address the issue of Tabular methods. These do not use SGD at all. Although the updates are very similar to an SGD update there is no gradient here and so we are not using SGD. Many ...
user avatar
3 votes
Accepted

What is the relation between the context in contextual bandits and the state in reinforcement learning?

The notion of a state in reinforcement learning is (more or less) the same as the notion of a context in contextual bandits. The main difference is that, in reinforcement learning, an action $a_t$ in ...
user avatar
  • 33.8k
3 votes

What is the relation between the context in contextual bandits and the state in reinforcement learning?

Conceptually, in general, how is the context being handled in CB, compared to states in RL? In terms of its place in the description of Contextual Bandits and Reinforcement Learning, context in CB is ...
user avatar
  • 23.9k
3 votes
Accepted

Why are only neural networks (and not SVMs, for example) used for reinforcement learning?

The biggest problem with SVMs, random forests, gradient boosting and others for reinforcement learning (RL) is that they are not able to learn online, adjusting for new data as it arrives, and equally ...
user avatar
  • 23.9k
2 votes

Can a neural network with linear activation functions produce a connection of linear functions?

Since I can't comment, there are a few caveats to previous answers. For instance, if you knew beforehand what the expected boundary function for that variable was, then you could transform it first. ...
user avatar
2 votes
Accepted

Are ReLUs incapable of solving certain problems?

There are a variety of possible things that could be wrong, but let me give you some potentially useful information. Neural networks with ReLU activation functions are Turing complete for a ...
user avatar
  • 136

Only top scored, non community-wiki answers of a minimum length are eligible