29
votes
Accepted
What is the Bellman operator in reinforcement learning?
The notation I'll be using is from two different lectures by David Silver and is also informed by these slides.
The expected Bellman equation is
$$v_\pi(s) = \sum_{a\in \cal{A}} \pi(a|s) \left(\cal{R}...
- 2,018
24
votes
Accepted
Is it possible to train the neural network to solve math equations?
Yes, it has been done!
However, the applications aren't to replace calculators or anything like that. The lab I'm associated with develops neural network models of equational reasoning to better ...
- 356
15
votes
Why is ChatGPT bad at math?
chatGPT is able to create well-formed sentences which contain phrases that are fitting for the input. It has rules extracted from its data, but those are not rules of understanding, but rules of '...
- 251
14
votes
Accepted
What sort of mathematical problems are there in AI that people are working on?
In artificial intelligence (sometimes called machine intelligence or computational intelligence), there are several problems that are based on mathematical topics, especially optimization, statistics, ...
- 37k
10
votes
How do we prove the n-step return error reduction property?
Let's start by looking at:
$$\max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert.$$
We can rewrite this by plugging in the definition of $G_{t:t+n}$:
\...
- 9,794
10
votes
Accepted
Why does the "reward to go" trick in policy gradient methods work?
An important thing we're going to need is what is called the "Expected Grad-Log-Prob Lemma here" (proof included on that page), which says that (for any $t$):
$$\mathbb{E}_{\tau \sim \pi_{\...
- 9,794
10
votes
Why is the derivative of the activation functions in neural networks important?
Consider a dataset $\mathcal{D}=\{x^{(i)},y^{(i)}:i=1,2,\ldots,N\}$ where $x^{(i)}\in\mathbb{R}^3$ and $y^{(i)}\in\mathbb{R}$ $\forall i$
The goal is to fit a function that best explains our dataset....
- 131
9
votes
Is it possible to train the neural network to solve math equations?
Not really.
Neural networks are good for determining non-linear relationships between inputs when there are hidden variables. In the examples above, the relationships are linear, and there are no ...
- 1,345
8
votes
Is it possible to train the neural network to solve math equations?
It is possible! In fact, it's an example of the popular deep learning framework Keras. Check out this link to see the source code.
This particular example uses a recurrent neural network (RNN) to ...
- 181
8
votes
Can neural networks be used to prove conjectures?
Your idea may be feasible in general, but a neural network is probably the wrong high level tool to use to explore this problem.
A neural network's strength is in finding internal representations ...
- 26.4k
7
votes
Can neural networks be used to prove conjectures?
Not in such straight forward way as described, but neural networks are successfully applied to guide the search of proof. There are automated theorem provers. What they do look roughly like this:
Get ...
- 171
7
votes
Accepted
Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?
The derivative of $\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$ with respect to $\hat{y}$, where $f$ is the model and $\hat{y} = f(x)$ is the output of the model, is
\begin{align}
\frac{d}{...
- 37k
7
votes
Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?
The MSE can be defined as $(\hat{y} - y)^2$, which should be equivalent to $(y - \hat{y})^2$
They are not just "equivalent". It is actually the exact same function, with two different ways to write ...
- 26.4k
7
votes
Why is ChatGPT bad at math?
(Check out my heavily related answer to a similar question here)
Why is ChatGPT bad at math, while it is very good at other stuff?
The problem comes down to the age-old problem of learning vs ...
- 808
6
votes
Why do activation functions need to be differentiable in the context of neural networks?
No, it is not necessary that an activation function is differentiable. In fact, one of the most popular activation functions, the rectifier, is non-differentiable at zero!
This can create problems ...
- 161
6
votes
Accepted
Is it ok to struggle with mathematics while learning AI as a beginner?
I think the key part of your question is "as a beginner". For all intents and purposes you can create a state of the art (SoTA) model in various fields with no knowledge of the mathematics what so ...
- 2,329
6
votes
Why is the derivative of the activation functions in neural networks important?
If what you are asking is what is the intuition for using the derivative in backpropagation learning, instead of an in-depth mathematical explanation:
Recall that the derivative tells you a function'...
- 171
6
votes
Accepted
What does the Markov assumption say about the history of state sequences?
A stochastic process has the Markov property if the probability distribution of future states conditioned on both the present and past states depends only on the present state or, more formally, the ...
- 37k
6
votes
How does ChatGPT know math?
Adding on to txopen's answer, it is interesting to note that for larger numbers with similar digits ChatGPT is unable to make any useful distinctions. For instance:
Me: Which number is bigger: 1234.12 ...
- 161
5
votes
Accepted
How can I start learning mathematics for machine learning?
You should begin from Dr Andrew Ng machine learning course on Coursera. It's probably the most popular course for newcomers in machine learning. It's a free course.
You should also grab "Elements of ...
- 1,411
5
votes
Accepted
What does "probabilistically" mean?
In a genetic algorithm, crossover (recombination) is the analogy to mating in the real world. For example, you have some genetic information from each parent. In the case of an optimization where you ...
- 474
5
votes
Accepted
How good is AI in math?
Nim was actually one of the first games ever played by an electronic machine. It was called the Nimatron and was displayed at the 1940 New York World's Fair.
It is also well known that neural networks ...
- 4,127
5
votes
Is the mean-squared error always convex in the context of neural networks?
Answer in short: MSE is convex on its input and parameters by itself. But on an arbitrary neural network it is not always convex due to the presence of non-linearities in the form of activation ...
- 562
5
votes
Accepted
Is recursion used in practice to improve performance of AI systems?
To my knowledge, recursion does not play a strong role in the definition of modern AI techniques, although it does feature used in Lovasz's definition of 'Local Search' and Kurzweil is certainly an ...
- 7,166
5
votes
Can neural networks be used to prove conjectures?
It's possible, but probably not a good idea.
Logical proof is one of the oldest areas of AI, and there are purpose-built techniques that don't need to be trained, and that are more reliable than a ...
- 9,037
5
votes
Accepted
What makes multi-layer neural networks able to perform nonlinear operations?
Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.
- 558
5
votes
What sort of mathematical problems are there in AI that people are working on?
Most of the math work being done in AI that I'm familiar with is already covered in nbro's answer. One thing that I do not believe is covered yet in that answer is proving algorithmic equivalence and/...
- 9,794
5
votes
Accepted
What is "conditioning" on a feature?
This is conditioning in the sense of conditional probability. The idea is that the authors have some "standard physically-inspired features". They are splitting the data up into bins based on the ...
- 9,037
5
votes
What is the mathematical definition of an activation function?
There is no strict definition of suitability of an activation function for neural networks. Instead there are a number of desirable traits, and functions that don't meet them or come close enough may ...
- 26.4k
5
votes
Accepted
What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$?
$\|x\| = |x|$ denotes the absolute value norm, which is a special case of the $L_1$ norm defined on the 1-D vector spaces formed by real or complex numbers.
$\|\textbf{x}\|_1 = \sum_{i=1}^n|x_i|$ ...
- 370
Only top scored, non community-wiki answers of a minimum length are eligible