Questions tagged [math]

For questions about mathematics related to artificial intelligence.

Filter by
Sorted by
Tagged with
1
vote
1answer
37 views

Mathematically speaking, Is it only the product operation used in the chain rule causing the vanishing or exploding gradient?

I am asking this question from the mathematical perspective of the vanishing and exploding gradient problems that we face generally during training deep neural networks. The chain rule of ...
21
votes
1answer
8k views

What is the Bellman operator in reinforcement learning?

In mathematics, the word operator can refer to several distinct but related concepts. An operator can be defined as a function between two vector spaces, it can be defined as a function where the ...
1
vote
1answer
54 views

What are the Calculus books recommended for beginner to advanced researchers in artificial intelligence?

Calculus is a branch of mathematics that primarily deals with the rate of change of outputs of a function w.r.t the inputs. It contains several concepts including limits, first-order derivatives, ...
0
votes
1answer
45 views

how to go from mathematical problem to neural network (and back)?

I am a little confused on how, you can find online papers that describe complex Machine Learning formulas in a mathematical/probabilistic way, and, in the other hands, easy tutorials that teach you ...
1
vote
1answer
80 views

Is it possible to know the distance objects are from camera based on only knowing one object's height?

I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
1
vote
1answer
17 views

Are the domains of objective functions in AI always equals to $\mathbb{R}^D$ or subset of it?

Consider the following paragraph from the chapter named Vector Calculus from the textbook titled Mathematics for Machine Learning by Marc Peter Deisenroth et al. Central to this chapter is the ...
2
votes
1answer
42 views

What is the analytical formula for "Kaiming He" probability density function?

A probability density function is a real-valued function that roughly gives the density of probability at a particular value of a random variable. For example, the probability density function of a ...
1
vote
1answer
100 views

In this paper, if region $R_{2}$ moves in a sliding window manner, won't the saliency map have a smaller size than the original image?

In the paper Salient Region Detection and Segmentation, I have a question pertaining to section 3 on the convolution-like operation being performed. I had already asked a few questions about the paper ...
1
vote
2answers
240 views

Formal proof that every purely reactive agent has behaviorally equivalent standard agent

It kind of makes sense intuitively but I'm not sure about a formal proof. I'll start with briefly listing definitions from Intro to Multiagent systems, Wooldridge, 2002 and then give you my reasoning ...
1
vote
1answer
33 views

Educational Resources and Programming Languages for AI & ML [closed]

I am a Mathematics graduate who is interested in AI and Machine Learning. I would like to dig deep into the maths of them but I do not know where to start. It seems Linear Algebra is the most crucial ...
0
votes
1answer
60 views

Why don't integrated gradients explain samples correctly?

I have a linear tabular dataset made of floats. The dataset follows a simple rule like: ...
32
votes
5answers
21k views

Is it possible to train the neural network to solve math equations?

I'm aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations? So given the ...
1
vote
1answer
86 views

Given an axis-angle rotation vector, how can I find the unit rotation axis and angle?

I have a robotics assignment, which I am unable to solve. Given the axis-angle rotation vector $\Theta = (2, 2, 0)$, how can I calculate the unit vector of the rotation axis $k$ and the angle $\theta$?...
2
votes
0answers
23 views

Is there any closed form analytical expression to represent fractional max pooling?

There are Nineteen types of pooling layers in PyTorch. Almost all of the layers are provided with corresponding analytical formulae. But analytical formulae are not provided for the fractional max-...
3
votes
4answers
2k views

Has the Fibonacci series or the golden ratio been applied in any way in AI?

I have been looking at the Fibonacci series, the golden ratio, and its uses in nature, like how flowers and animals grow based on the series. I was wondering whether we could use the Fibonacci series ...
1
vote
1answer
77 views

Explanation of this L2 minimization equation

I am trying to understand the last two lines of this math notation (from this paper). How did Var and double summation of Cov come to the equation? The first two lines I understood something like $(a-...
0
votes
1answer
28 views

How to understand slope of a (non-convex) function at a point in domain?

Consider the following paragraph from Numerical Computation of deep learning book that says derivative as a slope of the function curve at a point Suppose we have a function $y= f(x)$, where both $x$ ...
0
votes
1answer
43 views

What does it mean "having Lipschitz continuous derivatives"?

We can enforce some constraints on functions used in deep learning in order to guarantee optimizations. You can find it in Numerical Computation of the deep learning book. In the context of deep ...
0
votes
0answers
57 views

How can the V and Q functions take the expectation over a sum where the number of summands is random?

Assume the existence of a Markov Decision Process consisting of: State space $S$ Action space $A$ Transition model $T: S \times A \times S \to [0,1]$ Reward function $R: S \times A \times S \to \...
2
votes
0answers
68 views

REINFORCE differentiation on sum or single value?

I'm currently learning Policy-gradient Methods for RL and encountered REINFORCE algorithm. I learned from this site : https://towardsdatascience.com/policy-gradient-methods-104c783251e0 that the ...
0
votes
1answer
79 views

Why can a neural network use more than one activation function?

From trying to understand neural networks better, I've come upon a tentative notion that an activation function aims to build a function it's approximating via linear combinations with biases and ...
0
votes
1answer
25 views

Is the range of inception score flexible or bounded based on number of classes?

Inception score is used to evaluate the generative models. It is a score given based on quality and diversity of images generated. I have doubt about the range of inception score because of the reason ...
0
votes
0answers
69 views

What's wrong with my understanding of how RNNs work?

Recently, I've been trying to derive the mathematics behind various Neural Network structures. I managed to derive the MLP and tested it to be on par with a Keras implementation (Using the MNIST ...
0
votes
0answers
24 views

What are the different types of geometry in literature that may be used for deep learning?

Recently, I asked a question on the concept of a manifold and received an answer that points to a relatively new subfield of deep learning named geometric deep learning. In the ...
0
votes
0answers
30 views

What should the value of $ρ$ in the $w(n+1) = w(n) + \rho*\text{error}(i)x(i)$ formula of Least Mean Squares be?

I am trying to better understand the Least Mean Squares algorithm, in order to implement it programmatically. If we consider its weight updating formula $$w(n+1) = w(n) + \rho * \text{error}(i)x(i),$$ ...
0
votes
0answers
36 views

What are mathematically the factors of variation in deep learning?

The following paragraph from an answer tells us about factors of variation Factors of variation are some factors which determine varieties in observed data. If that factors change, the behaviour of ...
1
vote
2answers
247 views

What is Lipschitz constraint and why it is enforced on discriminator?

The following is the abstract for the research paper titled Improved Training of Wasserstein GANs Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training ...
0
votes
1answer
26 views

Isssue in understanding the derivation regarding mean squared error

The following derivation is taken from Chapter 5: Machine Learning Basics from the book titled Deep Learning (by Aaron Courville et al.) I am facing difficulty in understanding the zero derivative ...
1
vote
2answers
24 views

Why not undefined expression is different from numerical underflow?

Consider an architecture or programming language that uses $n$ bits for storing a floating point number in a particular format. Then each and every floating point number it can store should be in a ...
1
vote
0answers
33 views

Is it true that real world data is highly discontinuous?

A function $f$ is said to be continuous at a point $c$ if it satisfies three properties: Should be defined at the point $c$ Left and right-hand limits at $c$ must be equal i.e., the limit must exist ...
2
votes
3answers
102 views

Is there any domain in machine learning that solves a problem by using only analytical algorithms?

Most of the algorithms in machine learning I am aware of use datasets and learning happens in an iterative manner given some examples. The examples can also be understood as experience in the case of ...
5
votes
1answer
234 views

Defining formula for fuzzy equation

I'm learning fuzzy logic and more or less understand the basic concept, but i'm having a hard time understanding how to apply it to a method. I tried browsing online for explanation on how to use it, ...
0
votes
0answers
14 views

Predict a part of the input based of the output

I'm working on a fun project where I have a dataset of input and output data, both having a fixed size of characters. I would like to predict a part of the input based on a known output as follows: $$...
0
votes
2answers
33 views

Reason for relaxing limit in derivative in this context?

Consider the following paragraph from NUMERICAL COMPUTATION of the deep learning book.. Suppose we have a function $y = f(x)$, where both $x$ and $y$ are real numbers. The derivative of this function ...
1
vote
1answer
59 views

Is there a full and precise formulation of Theorem 1 in the Integrated Gradients paper?

Theorem 1 (page 5) in the paper about Integrated Gradients states that Integrated gradients is the unique path method that is symmetry-preserving. What I miss is A precise formulation of the ...
1
vote
0answers
17 views

Is there any concept like 'applying affine transformation on multiple inputs'?

Affine transformation on $X$ is a transformation of the following form $$Y = wX + b$$ In general, $w, X, Y$ and $b$ tensors. We generally call tensor $X$ as an input to affine transformation or the ...
0
votes
0answers
31 views

Is there any geometrical interpretation on overcoming gradient related problems by adjusting/changing loss function?

There are instances in literature where we need to change loss function in order to escape from gradient problems. Let $L_f$ be a loss function for a model I need to train on. Some times $L_f$ leads ...
1
vote
1answer
352 views

In a single neuron output layer should the output be a scalar?

Given a neural network with 3 inputs, 4 hidden layers, and 1 output, should the output neuron be a vector or a scalar? I thought that at the end of the summation only one number between 0 and 1 would ...
0
votes
0answers
14 views

A 2D distance measure that gives weightage to the angle between the two points?

n1 n8 n7 n2 c n6 n3 n4 n5 Assume that all neighbors ni are (Euclidean) ...
1
vote
1answer
37 views

What are the iid random variables for a dataset in the GAN framework?

I am trying to understand why mean is used for expectation in training Generative Adversarial Networks. The answer tells that it is due to the law of large numbers which is based on the assumption ...
0
votes
1answer
95 views

Mathematical calculation behind decision tree classifier with continuous variables

I am working on a binary classification problem having continuous variables (Gene expression Values). My goal is to classify the samples as case or ...
0
votes
1answer
33 views

How to calculate the gradient penalty proposed in "Improved Training of Wasserstein GANs"?

The research paper titled Improved Training of Wasserstein GANs proposed a gradient penalty in order to avoid undesired behavior due to weight clipping of the discriminator. We now propose an ...
0
votes
2answers
103 views

What is meant by an axis of a tensor?

Tensor is an ordered collection of elements. The elements are generally real numbers. Tensors are used in deep learning for storing data. There is a wide usage of the word "axis" related to ...
1
vote
1answer
91 views

What math should I learn before and while using and applying deep learning?

I want to learn deep learning. After researching a little, I came to the conclusion that I need a lot of math. I've started a linear algebra course, and it takes a long time (2-3 weeks). I want to ...
3
votes
1answer
57 views

What are the necessary mathematical properties to be a loss function in gradient based optimizations?

Loss functions are used in training neural networks. I am interested in knowing the mathematical properties that are necessary for a loss function to participate in gradient descent optimization. I ...
1
vote
1answer
75 views

Questions about a research paper on salient region detection and segmentation

I am reading this paper in an attempt to recreate the salient region detection and segmentation model employed. I have the following questions pertaining to section 3 of the paper and I would highly ...
1
vote
0answers
40 views

How can I compute a mathematical formula for my CNN?

Let's say, for example, I have built the following CNN model using Keras: ...
1
vote
1answer
36 views

How to interpret the policy gradient expression in reinforcement learning?

I'm currently going through the OpenAI's spinning up introduction course to reinforcement learning. On one of the final sections, they derive an expression for the gradient of the undiscounted return ...
3
votes
1answer
146 views

Why is tanh a "smoothly" differentiable function?

The sigmoid, tanh, and ReLU are popular and useful activation functions in the literature. The following excerpt taken from p4 of Neural Networks and Neural Language Models says that ...
0
votes
0answers
24 views

(Deep) feature engineering for lambda terms (mathematical expressions, higher order logic formulas) - is such thing?

Automated theorem proving with (deep) reinforcement learning (DRL) approach is hot topic in current AI research when domains of games are becoming saturated and completed research topics. For example, ...

1
2 3 4 5