Questions tagged [math]
For questions about mathematics related to artificial intelligence.
190
questions
0
votes
0answers
20 views
Why integrated gradients don't explain samples correctly?
I have a linear tabular dataset made of floats. The dataset follows a simple rule like:
...
1
vote
1answer
24 views
How is the variational lower bound for hard attention derived in Show, Attend and Tell
How is the jump from line 1 to line 2 done in equation 10 of Show, Attend and Tell?
While we're at it, another thing that might be muddying the waters for me is that I'm not clear on what the sum is ...
-2
votes
1answer
48 views
Assumptions of a Linear Regression [closed]
I was going through the concept of Linear Regression and ran into the concept of deciding whether a Linear Regression Model is the best fit for your data by 5 assumptions:
Linearity
Homoscedasticity
...
0
votes
0answers
15 views
How do we get from entropy to KL divergence in this paper?
I'm reading through Regularizing Neural Networks By Penalizing Confident Output Distributions and I'm stuck on the equation in section 3.2. It's not clear to me at all that the self-entropy of the ...
0
votes
0answers
21 views
Do $V_\theta$ and $V_s$ represent partial or total derivatives in the paper “Learning Continuous Control Policies by Stochastic Value Gradients”?
I was reading up on the Stochastic Value Gradients paper by Heess et al. In the paper, they describe a recursive process to calculate path-wise derivatives via equations (3) and (4), at the bottom of ...
1
vote
0answers
31 views
Could the inputs of the mean squared-error loss function be transformed to allow larger learning rates?
In the context of a neural network $\hat{y} = f_\theta(\mathbf{x})$ with parameters $\theta$ that is trained to perform regression such that the prediction $\hat{\mathbf{y}} = [\hat{y}_1,\hat{y}_2,...,...
0
votes
0answers
40 views
What are the math theorems regarding the Multilayer Perceptron?
I've come across a theorem "Convergence theorem
Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish)
...
0
votes
1answer
22 views
What's the difference between a 1d tensor and a 2d tensor with 1 dimension?
I'm doing a TensorFlow tutorial, where they convert an array of the numbers [1,2,3] to a tensor like this:
...
0
votes
1answer
47 views
Why can a neural network use more than one activation function?
From trying to understand neural networks better, I've come upon a tentative notion that an activation function aims to build a function it's approximating via linear combinations with biases and ...
0
votes
0answers
12 views
Is there an arrow missing in the derivation of front-door adjustment formula from do-calculus?
Here is Judea Pearl's derivation of the front-door adjustment formula:
Is there an arrow from Genotype to Cancer missing in the second diagram at the right?
just like this?
1
vote
0answers
17 views
Clonal operator in Immune Clonal Strategy
I was reading about Immune Clonal Strategy, specifically about Monoclonal operator from Immunity clonal strategies, and it goes as follows:
Here $a_i $ is a point and $a_i = \{ x_1, x_2, \cdots, x_m \...
0
votes
0answers
13 views
Meaning of grad_outputs in torch.autograd.grad for complex input and output
Let's say we have a mathematical expression,
$$
\mathbf{y} = \mathbf{Ax},
$$
where $\mathbf{y}$ and $\mathbf{x}$ are a vector, and $\mathbf{A}$ is a matrix.
Let's say the vector $\mathbf{y}$ is used ...
0
votes
1answer
38 views
Is the policy gradient expression in Fundamentals of Deep Learning wrong?
I don't understand the policy gradient as explained in Chapter-9 (Deep Reinforcement Learning) of the book Fundamentals of deep learning.
Here is the whole paragraph:
Policy Learning via Policy ...
-2
votes
1answer
41 views
Why the optimal Bellman operator of a Q-function can be approximated by a single point
I am currently studying reinforcement learning, especially DQN.
In DQN, learning proceeds in such a way as to minimize the norm (least-squares, Huber, etc.) of the optimal Bellman equation and the ...
1
vote
1answer
48 views
Is there a full and precise formulation of Theorem 1 in the Integrated Gradients paper?
Theorem 1 (page 5) in the paper about Integrated Gradients states that
Integrated gradients is the unique path method that is symmetry-preserving.
What I miss is
A precise formulation of the ...
3
votes
1answer
74 views
How do you calculate KL divergence on a three-dimensional space for a Variational Autoencoder?
I'm trying to implement a variational auto-encoder (as seen in Section 3.1 here: https://arxiv.org/pdf/2004.06271.pdf).
It differs from a traditional VAE because it encodes its input images to three-...
1
vote
1answer
67 views
How to compute the gradient of the cross-entropy loss function with respect to the parameters with softmax activation function?
I've seen plenty of examples of people doing Sigmoid + MSE backpropagation implementations, yet I do not seem to understand how to implement backpropagation as stated in the title in the case of multi-...
1
vote
0answers
31 views
Are monotonically increasing functions easier to learn?
A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant.
Logically this seems like ...
2
votes
1answer
58 views
What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?
I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
0
votes
0answers
16 views
Is parameter sharing in AlBERT akin to repeated application of same function on input?
I read the AlBERT and saw that its architecture used "Parameter Sharing" among layers of the encoder. They mentioned that this was done to save model space, make fewer training parameters ...
3
votes
4answers
491 views
What is the fundamental difference between an ML model and a function?
A model can be roughly defined as any design that is able to solve an ML task. Examples of models are the neural network, decision tree, Markov network, etc.
A function can be defined as a set of ...
0
votes
0answers
41 views
Can I solve the below functional equation using neural networks?
I recently watched this video, in which he solves the equation
$$f(x)+f\left(\frac{1}{1-x}\right) = x$$
The answer is
$$f(x) = \frac{x^3-x+1}{2x(x-1)}$$
I tried to solve this functional equation using ...
4
votes
1answer
82 views
Can we use ML to do anything else other than predicting (in the case of mathematical problems)?
(The math problem here just serves as an example, my question is on this type of problems in general).
Given two Schur polynomials, $s_\mu$, $s_\nu$, we know that we can decompose their product into a ...
1
vote
1answer
59 views
Explanation of this L2 minimization equation
I am trying to understand the last two lines of this math notation. How Var and double summation of Cov came to the equation. The first two lines I understood something like $(a-b)^2 = a^2 -2ab +b^2$.
1
vote
1answer
55 views
Is it okay to think of any dataset in artificial intelligence as a mathematical set?
A dataset is a collection of data points. It is known that the data points in the dataset can repeat. And the repetition does matter for building AI models.
So, why does the word dataset contain the ...
1
vote
0answers
33 views
Can any area of math come into play in Machine Learning Research?
As I read online following areas in mathematics comes into play in ML research
Linear Algebra
Calculus
Differential Equations
Probability
Statistics
Discrete Mathematics
Optimization
Analytic ...
1
vote
0answers
32 views
How do I derive the gradient of the log-likelihood of an RBM?
In a Restricted Boltzmann Machine (RBM), the likelihood function is:
$$p(\mathbf{v};\mathbf{\theta}) = \frac{1}{Z} \sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}$$
Where $E$ is the ...
1
vote
0answers
10 views
How do I find the data-point with respect to a given frame?
I've been reading this paper that formulates invariant task-parametrized HSMMs. In section 3.1 (Model Learning), the task parameters are represented in $F$ coordinate systems defined by $\{A_j,b_j\}_{...
1
vote
1answer
85 views
The mathematics in the CBOW and Skip-Gram models
this is my first question on AI Stack Exchange. I am a mathematics student who is learning NLP so I have paid a high amount of attention on the mathematics used in the subject, but my interpretations ...
0
votes
1answer
61 views
Is it possible to know the distance objects are from camera based on only knowing one object's height?
I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
2
votes
1answer
252 views
Why would the lookup table (of a table-driven artificial agent) need to store data at pixel precision?
While reading the book AI A modern approach, 4th ed, I came across the section of "Agent program" with following text:
It is instructive to consider why the table-driven approach to agent
...
1
vote
1answer
103 views
How to mathematically describe the convolution operation (with a Gaussian kernel)?
I have to build a model where I pre-process the data with a Gaussian kernel. The data are an $n\times n$ matrix (i.e one channel), but not an image, thus I can't refer to this matrix as an image and ...
6
votes
1answer
142 views
Formal definition of the Object Detection problem
For many problems in computer science, there is a formal, mathematical problem defition.
Something like: Given ..., the problem is to ...
How can the Object Detection problem (i.e. detecting objects ...
3
votes
0answers
21 views
Mapping given probabilities to empirical probabilities
Consider following problem statement:
You have given $n$ actions. You can perform any of them. Each action gives you success with some probability. The challenge is to perform given finite number of ...
2
votes
1answer
48 views
Research paths/areas for improving the performance of CNNs when faced with limited data
I've been reading through the research literature for image processing, computer vision, and convolutional neural networks. For image classification and object recognition, I know that convolutional ...
1
vote
1answer
71 views
What does $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ mean in the article Hindsight Experience Replay, section 2.1?
Taken from section 2.1 in the article:
We consider the standard reinforcement learning formalism consisting of an agent interacting with an environment. To simplify the exposition we assume that the ...
1
vote
0answers
42 views
Could the neural network automatically calculate and get different one-to-many quantities relative to their parent quantity?
Let's say I have a primary dataset that its secondary dataset is hundreds to match and group like an one-to-many relationship.
I'm new in this world of the AI but my problem is that many child groups ...
0
votes
0answers
30 views
Confusion on Math Notation Definition
I attempt to understand the formulation of dictionary learning for this paper:
Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
Multimodal Task-Driven ...
0
votes
1answer
52 views
Do the rows of the design matrix refer to the observations or predictors?
I attempt to understand the formulation of dictionary learning for this paper:
Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
Multimodal Task-Driven ...
2
votes
2answers
122 views
How does PCA work when we reduce the original space to 2 or higher-dimensional space?
How does PCA work when we reduce the original space to a 2 or higher-dimensional space? I understand the case when we reduce the dimensionality to $1$, but not this case.
$$\begin{array}{ll} \text{...
0
votes
1answer
63 views
Mathematical calculation behind decision tree classifier with continuous variables
Problem Description
I am working on a binary classification problem having continuous variables (Gene expression Values). My goal is to classify the samples as case ...
0
votes
0answers
65 views
Can a computer make a proof by induction?
Can a computer solve the following problem, i.e. make a proof by induction? And why?
Prove by induction that $$\sum_{k=1}^nk^3=\left(\frac{n(n+1)}{2}\right)^2, \, \, \, \forall n\in\mathbb N .$$
I'm ...
2
votes
1answer
151 views
Understand the DDPG algorithm in Keras
I'm trying to understand the DDPG algorithm using Keras
I found the site and started analyzing the code, I can't understand 2 things.
The algorithm used to write the code presented on the page
In the ...
1
vote
1answer
94 views
What is the definition of the “cost” function in the SVM's objective function?
In a course that I am attending, the cost function of a support vector machine is given by
$$J(\theta)=\sum_{i=1}^{m} y^{(i)} \operatorname{cost}_{1}\left(\theta^{T} x^{(i)}\right)+\left(1-y^{(i)}\...
1
vote
1answer
45 views
How to understand mapping function of kernel?
For a kernel function, we have two conditions one is that it should be symmetric which is easy to understand intuitively because dot products are symmetric as well and our kernel should also follow ...
1
vote
0answers
157 views
What is an auto-associator?
What is an auto-associator, and how does it work? How can we design an auto-associator for a given pattern? I couldn't find a clear explanation for this anywhere on the internet.
Here's an example of ...
1
vote
0answers
44 views
Simplifying Log Loss
I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
6
votes
1answer
154 views
Can deep learning be used to help mathematical research?
I am currently learning about deep learning and artificial intelligence and exploring his possibilities, and, as a mathematician at heart, I am inquisitive about how it can be used to solve problems ...
2
votes
0answers
73 views
Is this the correct gradient for log of softmax? [duplicate]
I am currently implementing the very basic version (REINFORCE) of the Monte Carlo policy gradient algorithm. I was wondering if this is the correct gradient for the log of softmax.
\begin{align}
\...
2
votes
1answer
125 views
Does Gödel's second incompleteness theorem put a limitation on artificial intelligence systems?
According to Brian Cantwell Smith
no calculation without representation
Therefore, computers depend on models. So, we can say that AI is limited internally by the model and externally by the ...
