# Questions tagged [function-approximation]

For questions related to the concept of function approximation. For example, questions that involve the use of a neural network (which is a function approximator) in the context of RL in order to approximate a value function or questions that are related to universal approximation theorems.

33 questions
Filter by
Sorted by
Tagged with
919 views

The tabular Q-learning algorithm is guaranteed to find the optimal $Q$ function, $Q^*$, provided the following conditions (the Robbins-Monro conditions) regarding the learning rate are satisfied $\... 2answers 2k views ### Where can I find the proof of the universal approximation theorem? The Wikipedia article for the universal approximation theorem cites a version of the universal approximation theorem for Lebesgue-measurable functions from this conference paper. However, the paper ... 1answer 78 views ### What makes multi-layer neural networks able to perform nonlinear operations? As I know, a single layer neural network can only do linear operations, but multilayered ones can. Also, I recently learned that finite matrices/tensors, which are used in many neural networks, can ... 2answers 569 views ### Is it possible to implement reinforcement learning using a neural network? I've implemented the reinforcement learning algorithm for an agent to play snappy bird (a shameless cheap ripoff of flappy bird) utilizing a q-table for storing the history for future lookups. It ... 1answer 114 views ### Can supervised learning be recast as reinforcement learning problem? Let's assume that there is a sequence of pairs$(x_i, y_i), (x_{i+1}, y_{i+1}), \dots$of observations and corresponding labels. Let's also assume that the$x$is considered as independent variable ... 2answers 272 views ### Why don't people use projected Bellman error with deep neural networks? Projected Bellman error has shown to be stable with linear function approximation. The technique is not at all new. I can only wonder why this technique is not adopted to use with non-linear function ... 1answer 120 views ### Is there a way of converting a neural network to another one that represents the same function? I have read the paper Neural Turing Machines and the paper On the Computational Power of Neural Nets about the computational power of neural networks. However, it isn't still clear to me one thing. ... 3answers 111 views ### Can we optimize an optimization algorithm? In this answer to the question Is an optimization algorithm equivalent to a neural network?, the author stated that, in theory, there is some recurrent neural network that implements a given ... 1answer 83 views ### Why can't neural networks learn functions outside of the specified domains? I understand that neural nets are fundamentally interpolative tools. Meaning, given a training dataset, a well trained neural net can approximate values within the domain of the training dataset. ... 1answer 96 views ### Convergence of semi-gradient TD(0) with non-linear function approximation I am looking for a result that shows the convergence of semi-gradient TD(0) algorithm with non-linear function approximation for on-policy prediction. Specifically, the update equation is given by (... 3answers 492 views ### Which functions can't neural networks learn efficiently? There are a lot of papers that show that neural networks can approximate a wide variety of functions. However, I can't find papers that show the limitations of NNs. What are the limitations of ... 1answer 42 views ### How do I represent a multi-dimensional state using a neural network? I have a set of 15 unique playing cards from a deck of 52 playing cards. A given state is represented by the respective card values in the set of 15 cards, where the card value is a prime number ... 3answers 398 views ### Which neural network should I use to approximate a specific function? We have convolutional neural networks and recurrent neural networks for analysing respectively images and sequential data. How do I determine which neural network architecture is more appropriate to ... 1answer 536 views ### Which machine learning models are universal function approximators? The universal approximation theorem states that a feed-forward neural network with a single hidden layer containing a finite number of neurons can approximate any continuous function (provided some ... 1answer 97 views ### What are the differences between artificial neural networks and other function approximators? Modern artificial neural networks use a lot more functions than just the classic sigmoid, to the point I'm having a hard time really seeing what classifies something as a "neural network" over other ... 0answers 45 views ### Is there any open source implementation of the SBEED learning algorithm? Are there are any openly available implementations of the SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation paper? 0answers 86 views ### What characteristics make it difficult for a Neural Network to approximate a function? What are the characteristics which make a function difficult for the Neural Network to approximate? Intuitively, one might think uneven functions might be difficult to approximate, but uneven ... 2answers 242 views ### Why is there a sigmoid function in the hidden layer of a neural network? [duplicate] I got this slide from CMU's lecture notes. The$x_i$s on the right are inputs and the$w_i$s are weights that get multiplied together then summed up at each hidden layer node. So I'm assuming this is ... 2answers 96 views ### Is there a possibility that there is no relationship between some inputs and outputs? I'm doing machine learning projects. I took a look at many datasets I worked with, mostly there are already famous datasets that everyone uses. Let's say I decided to make my own dataset. Is there a ... 2answers 64 views ### Returning function that a neural network estimated As stated in the Universal approximation theorem, a neural network can approximate almost any function. I was wondering, if there are methods to return the actual formula of the function that the NN ... 1answer 27 views ### Equivalence between expected parameter increments in “Off-Policy Temporal-Difference Learning with Function Approximation” I am having a hard time understanding the proof of theorem 1 presented in the "Off-Policy Temporal-Difference Learning with Function Approximation" paper. Let$\Delta \theta$and$\Delta \...
62 views

Consider some MDP with no terminal state. We can apply bootstrapping methods (like TD(0)) to learn in these cases no problem, but in policy gradient algorithms that have only a simple monte carlo ...
59 views

### Tweaking a CNN for large number of input channels

I am using a CNN for function approximation using geospatial data. The input of the function I am trying to approximate consists of all the spatial distances between N location on a grid and all the ...
31 views

### Changes in flow detection neural network?

Do you have any advice, what architecture of neural network is the best for following task? Let input be some (complex function), the neural network gains a flow of its values, so I guess there will ...
708 views

Semi-gradient methods work well in reinforcement learning, but what is there a reason of not using the true gradient if it can be computed? I tried it on the cart pole problem with a deep Q-network ...
107 views

### Hashed Tile Coding vs Regular Tile Coding

In the book "Reinforcement Learning: An Introduction" (2018) Sutton and Barto explain at page 221 a form of tile coding using hashing, to reduce memory consumption. I have two questions about that: ...
44 views

### Correct dimensionality of parameter vector for solving an MRP with linear function approximation?

I'm in the process of trying to learn more about RL by shadowing a course offered collaboratively by UCL and DeepMind that has been made available to the public. I'm most of the way through the course,...
33 views

### Why is it hard to prove the convergence of the deep Q-learning algorithm?

Why is it hard to prove the convergence of the DQN algorithm? We know that the tabular Q-learning algorithm converges to the optimal Q-values, and with a linear approximator convergence is proved. ...
39 views

### How can I build a model to approximate the function $f(n) = 2n$?

I made the following HTML nd javascript to predict $f(n) = 2n$. Basically, I am trying to design my first neural network which predicts 2 multiplied by a number. I know we don't need a neural network ...
316 views

### State aggregation methods

In Sutton's RL:An introduction 2nd edition it says the following(page 203): State aggregation is a simple form of generalizing function approximation in which states are grouped together, with ...
22 views

### Neural Network architecture for going from scalar input to time series outputs?

I have a problem where I know p features which are each scalar values and the output of 1 set of those features is a time history. Is there a specific neural network (NN) type architecture that can ...