Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [math]

For questions about mathematics related to artificial intelligence.

Filter by
Sorted by
Tagged with
1 vote
0 answers
37 views

Does all GAN's in literature need to satisfy the properties of objective function of initial GAN? [closed]

Consider the following value function of the initial GAN $V(D, G) = \mathbb{E}_{x \sim p_{data(x)}} [\log D(x)] + \mathbb{E}_{z \sim p_z(z)} [1- \log D(G(z))]$ The min-max game on the value function: $...
user avatar
  • 3,059
0 votes
1 answer
20 views

What is the right way to find the alphas in this equation?

In the Grad-CAM++ paper the following equation (7) is posed (written here without the relu function): $$ Y^c = \sum_k \Bigl( \Bigl\{ \sum_{a,b} \alpha_{ab}^{kc} \cdot \frac{\partial Y^c}{\...
user avatar
  • 131
0 votes
1 answer
26 views

Usability of power series in AI analysis

In mathematics, power series is given by $$f(x) = \sum\limits_{n=0}^{\infty} c_n (x-a)^n$$ where $c_n , a \in \mathbb{R}$ Although most of the courses in academics cover moment generating functions in ...
user avatar
  • 3,059
0 votes
1 answer
51 views

What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?

I have a question about the Grad-CAM++ paper. I do not understand how the following equation (10) for the alphas is obtained: $$ \alpha_{ij}^{kc} = \frac{\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}} {...
user avatar
  • 131
0 votes
0 answers
12 views

How do I compute the convolution of two kernels of the same size in practice?

Suppose I have a 256-by-256 input matrix called $X$ and two 3-by-3 kernels called $K_1$ and $K_2$. By the associativity of convolution \begin{equation} (X \star K_1) \star K_2 = X \star (K_1 \star K_2)...
user avatar
  • 131
1 vote
1 answer
39 views

Should I need to interpret the word "metric" in "performance metric" rigorously?

Consider the following abstract from the research paper titled A Note on the Inception Score for instance Deep generative models are powerful tools that have produced impressive results in recent ...
user avatar
  • 3,059
0 votes
1 answer
77 views

What does it mean by "gradient flow" in the context of neural networks?

Several research papers and textbooks (e.g. this) contain the phrase "gradient flow" in the context of neural networks. I am confused about whether it has any rigorous and formal way of ...
user avatar
  • 3,059
0 votes
1 answer
35 views

What does it mean by "dynamics of a sequence" mathematically?

Consider the following paragraph from the topic named sequential models from the textbook titled Dive into Deep Learning Both cases raise the obvious question of how to generate training data. One ...
user avatar
  • 3,059
1 vote
1 answer
75 views

What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$?

Here is the formula of state-value function in Reinforcement Learning. What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$? Why use square brackets? Why use $\mid$? Why do ...
user avatar
  • 113
0 votes
0 answers
29 views

Can we compute probability of sample in R1-Natural Evolution Strategy with linear time and space?

In R1-NES, the sample is drawn from a multivariate normal distribution, $\vec{\theta} \sim \mathcal{N}\left( \vec{\mu},\mathbf{\Sigma} \right)$, with the covariance matrix is parametrized by $s$ and a ...
user avatar
  • 143
0 votes
0 answers
53 views

How to prove that "w will converge to TD fixed point once A is positive definite"

In Reinforcement Learning: An Introduction 2nd edition section 9.4 (p. 206), it says that when we use TD(0) as target and use semi-gradient method to update : In general, $w_t$ will be reduced ...
user avatar
0 votes
0 answers
14 views

Nearest-Neighbour recommendation distance ranking where some recommendations occur more than once

A nearest neighbour recommendation engine may, given a query vector identify the ranked top-K nearest vectors within a dataset using a distance measure such as cosine similarity. In my problem domain, ...
user avatar
0 votes
1 answer
72 views

How to derive the dual function step by step in relative entropy policy search (REPS)?

TL:DR, (Why) is one of the terms in the expectation not derived properly? Relative entropy policy search or REPS is used to optimize a policy in an MDP. The update step is limited in the policy space (...
user avatar
  • 143
0 votes
1 answer
160 views

What is the definition of a trace of a tensor?

Tensor is a multi-dimensional ordered collection of elements, which is used in deep learning to store and process data as well as intermediate steps. We are aware of the trace of a two-dimensional ...
user avatar
  • 3,059
2 votes
1 answer
25 views

Are the domains of objective functions in AI always equals to $\mathbb{R}^D$ or subset of it?

Consider the following paragraph from the chapter named Vector Calculus from the textbook titled Mathematics for Machine Learning by Marc Peter Deisenroth et al. Central to this chapter is the ...
user avatar
  • 3,059
1 vote
1 answer
52 views

What is the analytical formula for "Kaiming He" probability density function?

A probability density function is a real-valued function that roughly gives the density of probability at a particular value of a random variable. For example, the probability density function of a ...
user avatar
  • 3,059
0 votes
1 answer
135 views

In this paper, if region $R_{2}$ moves in a sliding window manner, won't the saliency map have a smaller size than the original image?

In the paper Salient Region Detection and Segmentation, I have a question pertaining to section 3 on the convolution-like operation being performed. I had already asked a few questions about the paper ...
user avatar
1 vote
1 answer
70 views

Mathematically speaking, Is it only the product operation used in the chain rule causing the vanishing or exploding gradient?

I am asking this question from the mathematical perspective of the vanishing and exploding gradient problems that we face generally during training deep neural networks. The chain rule of ...
user avatar
  • 3,059
0 votes
0 answers
71 views

What's wrong with my understanding of how RNNs work?

Recently, I've been trying to derive the mathematics behind various Neural Network structures. I managed to derive the MLP and tested it to be on par with a Keras implementation (Using the MNIST ...
user avatar
0 votes
0 answers
26 views

What are the different types of geometry in literature that may be used for deep learning?

Recently, I asked a question on the concept of a manifold and received an answer that points to a relatively new subfield of deep learning named geometric deep learning. In the ...
user avatar
  • 3,059
0 votes
0 answers
39 views

What are mathematically the factors of variation in deep learning?

The following paragraph from an answer tells us about factors of variation Factors of variation are some factors which determine varieties in observed data. If that factors change, the behaviour of ...
user avatar
  • 3,059
2 votes
0 answers
71 views

REINFORCE differentiation on sum or single value?

I'm currently learning Policy-gradient Methods for RL and encountered REINFORCE algorithm. I learned from this site : https://towardsdatascience.com/policy-gradient-methods-104c783251e0 that the ...
user avatar
  • 49
0 votes
0 answers
31 views

What should the value of $ρ$ in the $w(n+1) = w(n) + \rho*\text{error}(i)x(i)$ formula of Least Mean Squares be?

I am trying to better understand the Least Mean Squares algorithm, in order to implement it programmatically. If we consider its weight updating formula $$w(n+1) = w(n) + \rho * \text{error}(i)x(i),$$ ...
user avatar
0 votes
1 answer
35 views

Isssue in understanding the derivation regarding mean squared error

The following derivation is taken from Chapter 5: Machine Learning Basics from the book titled Deep Learning (by Aaron Courville et al.) I am facing difficulty in understanding the zero derivative ...
user avatar
  • 3,059
0 votes
1 answer
37 views

How to understand slope of a (non-convex) function at a point in domain?

Consider the following paragraph from Numerical Computation of deep learning book that says derivative as a slope of the function curve at a point Suppose we have a function $y= f(x)$, where both $x$ ...
user avatar
  • 3,059
0 votes
1 answer
101 views

What are the Calculus books recommended for beginner to advanced researchers in artificial intelligence?

Calculus is a branch of mathematics that primarily deals with the rate of change of outputs of a function w.r.t the inputs. It contains several concepts including limits, first-order derivatives, ...
user avatar
  • 3,059
1 vote
1 answer
75 views

What does it mean "having Lipschitz continuous derivatives"?

We can enforce some constraints on functions used in deep learning in order to guarantee optimizations. You can find it in Numerical Computation of the deep learning book. In the context of deep ...
user avatar
  • 3,059
1 vote
0 answers
37 views

Is it true that real world data is highly discontinuous?

A function $f$ is said to be continuous at a point $c$ if it satisfies three properties: Should be defined at the point $c$ Left and right-hand limits at $c$ must be equal i.e., the limit must exist ...
user avatar
  • 3,059
0 votes
0 answers
16 views

Predict a part of the input based of the output

I'm working on a fun project where I have a dataset of input and output data, both having a fixed size of characters. I would like to predict a part of the input based on a known output as follows: $$...
user avatar
  • 1
0 votes
2 answers
36 views

Reason for relaxing limit in derivative in this context?

Consider the following paragraph from NUMERICAL COMPUTATION of the deep learning book.. Suppose we have a function $y = f(x)$, where both $x$ and $y$ are real numbers. The derivative of this function ...
user avatar
  • 3,059
3 votes
3 answers
88 views

What are the mathematical properties of natural exponential function that lead to its usefulness in artificial intelligence?

In mathematics, there is a proof that the following infinite series converges to a constant irrational number, denoted by $e$, called as Euler's number or Euler's constant or Napier's constant. The ...
user avatar
  • 3,059
1 vote
2 answers
24 views

Why not undefined expression is different from numerical underflow?

Consider an architecture or programming language that uses $n$ bits for storing a floating point number in a particular format. Then each and every floating point number it can store should be in a ...
user avatar
  • 3,059
2 votes
3 answers
108 views

Is there any domain in machine learning that solves a problem by using only analytical algorithms?

Most of the algorithms in machine learning I am aware of use datasets and learning happens in an iterative manner given some examples. The examples can also be understood as experience in the case of ...
user avatar
  • 3,059
1 vote
0 answers
20 views

Is there any concept like 'applying affine transformation on multiple inputs'?

Affine transformation on $X$ is a transformation of the following form $$Y = wX + b$$ In general, $w, X, Y$ and $b$ tensors. We generally call tensor $X$ as an input to affine transformation or the ...
user avatar
  • 3,059
0 votes
1 answer
37 views

Is the range of inception score flexible or bounded based on number of classes?

Inception score is used to evaluate the generative models. It is a score given based on quality and diversity of images generated. I have doubt about the range of inception score because of the reason ...
user avatar
  • 3,059
0 votes
0 answers
33 views

Is there any geometrical interpretation on overcoming gradient related problems by adjusting/changing loss function?

There are instances in literature where we need to change loss function in order to escape from gradient problems. Let $L_f$ be a loss function for a model I need to train on. Some times $L_f$ leads ...
user avatar
  • 3,059
0 votes
0 answers
15 views

A 2D distance measure that gives weightage to the angle between the two points?

n1 n8 n7 n2 c n6 n3 n4 n5 Assume that all neighbors ni are (Euclidean) ...
user avatar
1 vote
1 answer
56 views

What are the iid random variables for a dataset in the GAN framework?

I am trying to understand why mean is used for expectation in training Generative Adversarial Networks. The answer tells that it is due to the law of large numbers which is based on the assumption ...
user avatar
  • 3,059
0 votes
1 answer
148 views

How to calculate the gradient penalty proposed in "Improved Training of Wasserstein GANs"?

The research paper titled Improved Training of Wasserstein GANs proposed a gradient penalty in order to avoid undesired behavior due to weight clipping of the discriminator. We now propose an ...
user avatar
  • 3,059
3 votes
2 answers
1k views

What is Lipschitz constraint and why it is enforced on discriminator?

The following is the abstract for the research paper titled Improved Training of Wasserstein GANs Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training ...
user avatar
  • 3,059
0 votes
2 answers
317 views

What is meant by an axis of a tensor?

Tensor is an ordered collection of elements. The elements are generally real numbers. Tensors are used in deep learning for storing data. There is a wide usage of the word "axis" related to ...
user avatar
  • 3,059
3 votes
1 answer
103 views

What are the necessary mathematical properties to be a loss function in gradient based optimizations?

Loss functions are used in training neural networks. I am interested in knowing the mathematical properties that are necessary for a loss function to participate in gradient descent optimization. I ...
user avatar
  • 3,059
2 votes
0 answers
26 views

Is there any closed form analytical expression to represent fractional max pooling?

There are Nineteen types of pooling layers in PyTorch. Almost all of the layers are provided with corresponding analytical formulae. But analytical formulae are not provided for the fractional max-...
user avatar
  • 3,059
1 vote
1 answer
86 views

Questions about a research paper on salient region detection and segmentation [closed]

I am reading this paper in an attempt to recreate the salient region detection and segmentation model employed. I have the following questions pertaining to section 3 of the paper and I would highly ...
user avatar
1 vote
0 answers
78 views

How can I compute a mathematical formula for my CNN?

Let's say, for example, I have built the following CNN model using Keras: ...
user avatar
1 vote
1 answer
51 views

How to interpret the policy gradient expression in reinforcement learning?

I'm currently going through the OpenAI's spinning up introduction course to reinforcement learning. On one of the final sections, they derive an expression for the gradient of the undiscounted return ...
user avatar
2 votes
1 answer
328 views

Why is tanh a "smoothly" differentiable function?

The sigmoid, tanh, and ReLU are popular and useful activation functions in the literature. The following excerpt taken from p4 of Neural Networks and Neural Language Models says that ...
user avatar
  • 3,059
1 vote
1 answer
100 views

What math should I learn before and while using and applying deep learning?

I want to learn deep learning. After researching a little, I came to the conclusion that I need a lot of math. I've started a linear algebra course, and it takes a long time (2-3 weeks). I want to ...
user avatar
  • 11
1 vote
1 answer
214 views

What is the meaning or implications of the rank of a dataset for machine learning algorithms?

Consider a dataset with $n$ training examples and $d$ features. Let $D_{n \times d}$ be the data matrix and $r$ be the rank of it. In matrices, rank $r$ is generally useful in Knowing the dimension ...
user avatar
  • 3,059
0 votes
0 answers
23 views

Derivation of an probability expansion used in Word2Vec classifier model

We are using the following notations, for this question, to calculate the probability values \begin{array}{|c|c|} \hline \text{$w$} & \text{target word embedding vector} \\ \hline \text{$c$} &...
user avatar
  • 3,059

1
2 3 4 5