# Tag Info

## Hot answers tagged math

Accepted

• 9,519

### Why is the derivative of the activation functions in neural networks important?

Consider a dataset $\mathcal{D}=\{x^{(i)},y^{(i)}:i=1,2,\ldots,N\}$ where $x^{(i)}\in\mathbb{R}^3$ and $y^{(i)}\in\mathbb{R}$ $\forall i$ The goal is to fit a function that best explains our dataset....
• 101

### Is it possible to train the neural network to solve math equations?

It is possible! In fact, it's an example of the popular deep learning framework Keras. Check out this link to see the source code. This particular example uses a recurrent neural network (RNN) to ...
• 171

### Can neural networks be used to prove conjectures?

Your idea may be feasible in general, but a neural network is probably the wrong high level tool to use to explore this problem. A neural network's strength is in finding internal representations ...
• 24.7k
Accepted

### Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?

The derivative of $\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$ with respect to $\hat{y}$, where $f$ is the model and $\hat{y} = f(x)$ is the output of the model, is \begin{align} \frac{d}{...
• 35k

### Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?

The MSE can be defined as $(\hat{y} - y)^2$, which should be equivalent to $(y - \hat{y})^2$ They are not just "equivalent". It is actually the exact same function, with two different ways to write ...
• 24.7k
Accepted

### Is it ok to struggle with mathematics while learning AI as a beginner?

I think the key part of your question is "as a beginner". For all intents and purposes you can create a state of the art (SoTA) model in various fields with no knowledge of the mathematics what so ...
• 2,299

### Why is the derivative of the activation functions in neural networks important?

If what you are asking is what is the intuition for using the derivative in backpropagation learning, instead of an in-depth mathematical explanation: Recall that the derivative tells you a function'...
• 171
Accepted

### What does the Markov assumption say about the history of state sequences?

A stochastic process has the Markov property if the probability distribution of future states conditioned on both the present and past states depends only on the present state or, more formally, the ...
• 35k
Accepted

### How can I start learning mathematics for machine learning?

You should begin from Dr Andrew Ng machine learning course on Coursera. It's probably the most popular course for newcomers in machine learning. It's a free course. You should also grab "Elements of ...
• 1,390
Accepted

### What does "probabilistically" mean?

In a genetic algorithm, crossover (recombination) is the analogy to mating in the real world. For example, you have some genetic information from each parent. In the case of an optimization where you ...
• 474
Accepted

### How good is AI in math?

Nim was actually one of the first games ever played by an electronic machine. It was called the Nimatron and was displayed at the 1940 New York World's Fair. It is also well known that neural networks ...
• 4,097

### Is the mean-squared error always convex in the context of neural networks?

Answer in short: MSE is convex on its input and parameters by itself. But on an arbitrary neural network it is not always convex due to the presence of non-linearities in the form of activation ...
• 552

### Why do activation functions need to be differentiable in the context of neural networks?

No, it is not necessary that an activation function is differentiable. In fact, one of the most popular activation functions, the rectifier, is non-differentiable at zero! This can create problems ...
• 151
Accepted

### Is recursion used in practice to improve performance of AI systems?

To my knowledge, recursion does not play a strong role in the definition of modern AI techniques, although it does feature used in Lovasz's definition of 'Local Search' and Kurzweil is certainly an ...
• 7,106

### Can neural networks be used to prove conjectures?

Not in such straight forward way as described, but neural networks are successfully applied to guide the search of proof. There are automated theorem provers. What they do look roughly like this: Get ...
• 151
Accepted

### What makes multi-layer neural networks able to perform nonlinear operations?

Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.
• 548

### What sort of mathematical problems are there in AI that people are working on?

Most of the math work being done in AI that I'm familiar with is already covered in nbro's answer. One thing that I do not believe is covered yet in that answer is proving algorithmic equivalence and/...
• 9,519
Accepted

### What is "conditioning" on a feature?

This is conditioning in the sense of conditional probability. The idea is that the authors have some "standard physically-inspired features". They are splitting the data up into bins based on the ...
• 8,937

### What is the mathematical definition of an activation function?

There is no strict definition of suitability of an activation function for neural networks. Instead there are a number of desirable traits, and functions that don't meet them or come close enough may ...
• 24.7k
Accepted

### What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$?

$\|x\| = |x|$ denotes the absolute value norm, which is a special case of the $L_1$ norm defined on the 1-D vector spaces formed by real or complex numbers. $\|\textbf{x}\|_1 = \sum_{i=1}^n|x_i|$ ...
• 360
Accepted

### Formal definition of the Object Detection problem

This is just an idea Given a set of pixels, the task is to decide: Which pixel is the center of an object? What is the size of the bounding boxes with the center is the pixel in part 1? Formula, ...
• 857

### What is the fundamental difference between an ML model and a function?

A model as a set of functions In some cases in machine learning, a model can be thought of as a set of functions, so here's the first difference. For example, a neural network with an arbitrary vector ...
• 35k