# Questions tagged [function-approximation]

For questions related to the concept of function approximation. For example, questions that involve the use of a neural network (which is a function approximator) in the context of RL in order to approximate a value function or questions that are related to universal approximation theorems.

70 questions
Filter by
Sorted by
Tagged with
24 views

### Alternatives to neural networks for function approximation in Q learning?

I want to know if there is anything other than neural networks (or Deep NNs) that I can effectively use to perform function approximation? I am asking this w.r.t to the use of approximators in Q ...
54 views

### What do we actually 'approximate' when dealing with large state spaces in Q-learning?

I realized that my state space is very large in size. I had planned to use tabular Q-learning (Bellman equation to update the $Q(s, a)$ after each action taken). But this 'large space' realization has ...
21 views

### Derivation of weights for linear function approximation of Temporal difference method

https://web.stanford.edu/class/cs234/slides/lecture5.pdf I am reading the above material. On page 38, the update for the parameters for the linear function approximation of TD(0) is given. I have a ...
29 views

### Uniform representation of images for machine learning

I'm new to the field of ML so please bear with me while I try to explain what I'm looking for. In most machine learning pipelines that deal with images there is a requirement to "normalize" ...
28 views

### Why use sin/cos to give periodicity in time series prediction

In this tutorial https://www.tensorflow.org/tutorials/structured_data/time_series#feature_engineering (scroll down a bit to "Time" heading), they take the sin/cos of the time index, and give ...
33 views

### Is it true that real world data is highly discontinuous?

A function $f$ is said to be continuous at a point $c$ if it satisfies three properties: Should be defined at the point $c$ Left and right-hand limits at $c$ must be equal i.e., the limit must exist ...
32 views

### Is Reinforcement Learning capable of learning complex functions (such as producing a 3d model given an image)?

I want to build an AI that can convert an image of a subject into an anatomically accurate 3D model. To do this, I was thinking of adapting the following code for Deep Deterministic Policy Gradient: ...
40 views

### Can reinforcement learning be used to learn an unknown analytical function (for example, $y = x^2$ )?

Are there any examples for RL to learn analytical functions (for example, $y=x^2$)? What are the considerations when constructing the environment? Are there any literature that analyzes the difficulty/...
16 views

### Recursive Least squares (RLS) for mini batch

For my application I am considering a learning problem where I simulate a bunch of episodes say '$n$' first, and than carry out the recursive least squares update. Similar to $TD(1)$. I know that RLS ...
20 views

25 views

### Is there any thumb rule on the cardinality of state space in order to use the parameterized function to estimate value functions?

Value functions for a given MDP can be learned in at least two ways by experience. The first way (tabular calculation) is generally used in the case of state spaces that are small enough. The second ...
18 views

### Can transformers be used to improve regression?

I was recently reading a bit about transformers and I don't understand them very much but I was wondering if anyone knows if any of their techniques such as attention mechanism or anything has been ...
33 views

### Trying to understand why nonlinearity is important for neural networks by analogy

Is the reason why linear activation functions are usually pretty bad at approximating functions the same reason why combinations of hermitian polynomials or combinations of sines and cosines are ...
79 views

### Why can a neural network use more than one activation function?

From trying to understand neural networks better, I've come upon a tentative notion that an activation function aims to build a function it's approximating via linear combinations with biases and ...
155 views

### Why the optimal Bellman operator of a Q-function can be approximated by a single point

I am currently studying reinforcement learning, especially DQN. In DQN, learning proceeds in such a way as to minimize the norm (least-squares, Huber, etc.) of the optimal Bellman equation and the ...
36 views

### How can "any process you can imagine" be thought of as function computation?

I stumbled upon this passage when reading this guide. Universality theorems are a commonplace in computer science, so much so that we sometimes forget how astonishing they are. But it's worth ...
27 views

### How to find good features for a linear function approximation in RL with large discrete state set?

I've recently read much about feature engineering in continuous (uncountable) feature spaces. Now I am interested what methods exist in the setting of large discrete state spaces. For example consider ...
83 views

112 views

### Is there a possibility that there is no relationship between some inputs and outputs?

I'm doing machine learning projects. I took a look at many datasets I worked with, mostly there are already famous datasets that everyone uses. Let's say I decided to make my own dataset. Is there a ...
76 views

Consider some MDP with no terminal state. We can apply bootstrapping methods (like TD(0)) to learn in these cases no problem, but in policy gradient algorithms that have only a simple monte carlo ...
181 views

### Why does reinforcement learning using a non-linear function approximator diverge when using strongly correlated data as input?

While reading the DQN paper, I found that randomly selecting and learning samples reduced divergence in RL using a non-linear function approximator (e.g a neural network). So, why does Reinforcement ...
222 views

### Convergence of semi-gradient TD(0) with non-linear function approximation

I am looking for a result that shows the convergence of semi-gradient TD(0) algorithm with non-linear function approximation for on-policy prediction. Specifically, the update equation is given by (...
910 views

### Why is there a sigmoid function in the hidden layer of a neural network? [duplicate]

I got this slide from CMU's lecture notes. The $x_i$s on the right are inputs and the $w_i$s are weights that get multiplied together then summed up at each hidden layer node. So I'm assuming this is ...
56 views

### Is there any open source implementation of the SBEED learning algorithm?

Are there are any openly available implementations of the SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation paper?
223 views

### Can supervised learning be recast as reinforcement learning problem?

Let's assume that there is a sequence of pairs $(x_i, y_i), (x_{i+1}, y_{i+1}), \dots$ of observations and corresponding labels. Let's also assume that the $x$ is considered as independent variable ...
95 views

### Tweaking a CNN for large number of input channels

I am using a CNN for function approximation using geospatial data. The input of the function I am trying to approximate consists of all the spatial distances between N location on a grid and all the ...
107 views

### Remove drawbacks of Neural Network regressor as compared Polynomial Regressor

As far as my knowledge goes (might be a bit vague and not mathematical), a Neural Network can and should only be able to approximate a bounded function, which is not the case of a Polynomial Regressor....
135 views

### Can we optimize an optimization algorithm?

In this answer to the question Is an optimization algorithm equivalent to a neural network?, the author stated that, in theory, there is some recurrent neural network that implements a given ...