Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Hot answers tagged

27 votes
Accepted

What is the Bellman operator in reinforcement learning?

The notation I'll be using is from two different lectures by David Silver and is also informed by these slides. The expected Bellman equation is $$v_\pi(s) = \sum_{a\in \cal{A}} \pi(a|s) \left(\cal{R}...
user avatar
24 votes
Accepted

Is it possible to train the neural network to solve math equations?

Yes, it has been done! However, the applications aren't to replace calculators or anything like that. The lab I'm associated with develops neural network models of equational reasoning to better ...
user avatar
  • 356
13 votes
Accepted

What sort of mathematical problems are there in AI that people are working on?

In artificial intelligence (sometimes called machine intelligence or computational intelligence), there are several problems that are based on mathematical topics, especially optimization, statistics, ...
user avatar
  • 33.2k
9 votes

Is it possible to train the neural network to solve math equations?

Not really. Neural networks are good for determining non-linear relationships between inputs when there are hidden variables. In the examples above, the relationships are linear, and there are no ...
user avatar
  • 1,335
9 votes

How do we prove the n-step return error reduction property?

Let's start by looking at: $$\max_s \Bigl\lvert \mathbb{E}_{\pi} \left[ G_{t:t+n} \mid S_t = s \right] - v_{\pi}(s) \Bigr\rvert.$$ We can rewrite this by plugging in the definition of $G_{t:t+n}$: \...
user avatar
  • 9,336
9 votes
Accepted

Why does the "reward to go" trick in policy gradient methods work?

An important thing we're going to need is what is called the "Expected Grad-Log-Prob Lemma here" (proof included on that page), which says that (for any $t$): $$\mathbb{E}_{\tau \sim \pi_{\...
user avatar
  • 9,336
7 votes

Is it possible to train the neural network to solve math equations?

It is possible! In fact, it's an example of the popular deep learning framework Keras. Check out this link to see the source code. This particular example uses a recurrent neural network (RNN) to ...
user avatar
7 votes
Accepted

Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?

The derivative of $\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$ with respect to $\hat{y}$, where $f$ is the model and $\hat{y} = f(x)$ is the output of the model, is \begin{align} \frac{d}{...
user avatar
  • 33.2k
7 votes

Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?

The MSE can be defined as $(\hat{y} - y)^2$, which should be equivalent to $(y - \hat{y})^2$ They are not just "equivalent". It is actually the exact same function, with two different ways to write ...
user avatar
  • 23.3k
7 votes

Why is the derivative of the activation functions in neural networks important?

Consider a dataset $\mathcal{D}=\{x^{(i)},y^{(i)}:i=1,2,\ldots,N\}$ where $x^{(i)}\in\mathbb{R}^3$ and $y^{(i)}\in\mathbb{R}$ $\forall i$ The goal is to fit a function that best explains our dataset....
user avatar
  • 71
6 votes

Can neural networks be used to prove conjectures?

Your idea may be feasible in general, but a neural network is probably the wrong high level tool to use to explore this problem. A neural network's strength is in finding internal representations ...
user avatar
  • 23.3k
6 votes
Accepted

Is it ok to struggle with mathematics while learning AI as a beginner?

I think the key part of your question is "as a beginner". For all intents and purposes you can create a state of the art (SoTA) model in various fields with no knowledge of the mathematics what so ...
user avatar
  • 2,239
6 votes

Why is the derivative of the activation functions in neural networks important?

If what you are asking is what is the intuition for using the derivative in backpropagation learning, instead of an in-depth mathematical explanation: Recall that the derivative tells you a function'...
user avatar
6 votes
Accepted

What does the Markov assumption say about the history of state sequences?

A stochastic process has the Markov property if the probability distribution of future states conditioned on both the present and past states depends only on the present state or, more formally, the ...
user avatar
  • 33.2k
5 votes
Accepted

How can I start learning mathematics for machine learning?

You should begin from Dr Andrew Ng machine learning course on Coursera. It's probably the most popular course for newcomers in machine learning. It's a free course. You should also grab "Elements of ...
user avatar
  • 1,368
5 votes
Accepted

What does "probabilistically" mean?

In a genetic algorithm, crossover (recombination) is the analogy to mating in the real world. For example, you have some genetic information from each parent. In the case of an optimization where you ...
user avatar
  • 474
5 votes
Accepted

How good is AI in math?

Nim was actually one of the first games ever played by an electronic machine. It was called the Nimatron and was displayed at the 1940 New York World's Fair. It is also well known that neural networks ...
user avatar
5 votes
Accepted

Is recursion used in practice to improve performance of AI systems?

To my knowledge, recursion does not play a strong role in the definition of modern AI techniques, although it does feature used in Lovasz's definition of 'Local Search' and Kurzweil is certainly an ...
user avatar
5 votes

Can neural networks be used to prove conjectures?

Not in such straight forward way as described, but neural networks are successfully applied to guide the search of proof. There are automated theorem provers. What they do look roughly like this: Get ...
user avatar
  • 151
5 votes
Accepted

What makes multi-layer neural networks able to perform nonlinear operations?

Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.
user avatar
5 votes

What sort of mathematical problems are there in AI that people are working on?

Most of the math work being done in AI that I'm familiar with is already covered in nbro's answer. One thing that I do not believe is covered yet in that answer is proving algorithmic equivalence and/...
user avatar
  • 9,336
5 votes
Accepted

What is "conditioning" on a feature?

This is conditioning in the sense of conditional probability. The idea is that the authors have some "standard physically-inspired features". They are splitting the data up into bins based on the ...
user avatar
5 votes

What is the mathematical definition of an activation function?

There is no strict definition of suitability of an activation function for neural networks. Instead there are a number of desirable traits, and functions that don't meet them or come close enough may ...
user avatar
  • 23.3k
5 votes
Accepted

What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$?

$\|x\| = |x|$ denotes the absolute value norm, which is a special case of the $L_1$ norm defined on the 1-D vector spaces formed by real or complex numbers. $\|\textbf{x}\|_1 = \sum_{i=1}^n|x_i|$ ...
user avatar
  • 360
5 votes
Accepted

Formal definition of the Object Detection problem

This is just an idea Given a set of pixels, the task is to decide: Which pixel is the center of an object? What is the size of the bounding boxes with the center is the pixel in part 1? Formula, ...
user avatar
  • 677
5 votes

What is the fundamental difference between an ML model and a function?

A model as a set of functions In some cases in machine learning, a model can be thought of as a set of functions, so here's the first difference. For example, a neural network with an arbitrary vector ...
user avatar
  • 33.2k
4 votes

Is the mean-squared error always convex in the context of neural networks?

Answer in short: MSE is convex on its input and parameters by itself. But on an arbitrary neural network it is not always convex due to the presence of non-linearities in the form of activation ...
user avatar
  • 532
4 votes

Why do activation functions need to be differentiable in the context of neural networks?

No, it is not necessary that an activation function is differentiable. In fact, one of the most popular activation functions, the rectifier, is non-differentiable at zero! This can create problems ...
user avatar
4 votes

Why does the cost function contain a 2 at the denominator?

To simplify the derivative, probably. Otherwise there will be constant 2 in it.
user avatar
  • 159
4 votes

Why do activation functions need to be differentiable in the context of neural networks?

Training While "running" a neural network can be done with any activation functions, we usually want to train it - i.e., adjust its parameters so that the result would be closer to what we desire. ...
user avatar
  • 853

Only top scored, non community-wiki answers of a minimum length are eligible