Questions tagged [math]
For questions about mathematics related to artificial intelligence.
174
questions
3
votes
1answer
58 views
How do you calculate KL divergence on a three-dimensional space for a Variational Autoencoder?
I'm trying to implement a variational auto-encoder (as seen in Section 3.1 here: https://arxiv.org/pdf/2004.06271.pdf).
It differs from a traditional VAE because it encodes its input images to three-...
2
votes
1answer
58 views
How to compute the gradient of the cross-entropy loss function with respect to the parameters with softmax activation function?
I've seen plenty of examples of people doing Sigmoid + MSE backpropagation implementations, yet I do not seem to understand how to implement backpropagation as stated in the title in the case of multi-...
1
vote
0answers
28 views
Are monotonically increasing functions easier to learn?
A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant.
Logically this seems like ...
2
votes
1answer
48 views
What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?
I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
0
votes
0answers
15 views
Is parameter sharing in AlBERT akin to repeated application of same function on input?
I read the AlBERT and saw that its architecture used "Parameter Sharing" among layers of the encoder. They mentioned that this was done to save model space, make fewer training parameters ...
4
votes
4answers
458 views
What is the fundamental difference between an ML model and a function?
A model can be roughly defined as any design that is able to solve an ML task. Examples of models are the neural network, decision tree, Markov network, etc.
A function can be defined as a set of ...
0
votes
0answers
38 views
Can I solve the below functional equation using neural networks?
I recently watched this video, in which he solves the equation
$$f(x)+f\left(\frac{1}{1-x}\right) = x$$
The answer is
$$f(x) = \frac{x^3-x+1}{2x(x-1)}$$
I tried to solve this functional equation using ...
4
votes
1answer
72 views
Can we use ML to do anything else other than predicting (in the case of mathematical problems)?
(The math problem here just serves as an example, my question is on this type of problems in general).
Given two Schur polynomials, $s_\mu$, $s_\nu$, we know that we can decompose their product into a ...
2
votes
1answer
44 views
Explanation of this L2 minimization equation
I am trying to understand the last two lines of this math notation. How Var and double summation of Cov came to the equation. The first two lines I understood something like $(a-b)^2 = a^2 -2ab +b^2$.
1
vote
1answer
54 views
Is it okay to think of any dataset in artificial intelligence as a mathematical set?
A dataset is a collection of data points. It is known that the data points in the dataset can repeat. And the repetition does matter for building AI models.
So, why does the word dataset contain the ...
1
vote
0answers
31 views
Can any area of math come into play in Machine Learning Research?
As I read online following areas in mathematics comes into play in ML research
Linear Algebra
Calculus
Differential Equations
Probability
Statistics
Discrete Mathematics
Optimization
Analytic ...
1
vote
0answers
30 views
How do I derive the gradient of the log-likelihood of an RBM?
In a Restricted Boltzmann Machine (RBM), the likelihood function is:
$$p(\mathbf{v};\mathbf{\theta}) = \frac{1}{Z} \sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}$$
Where $E$ is the ...
1
vote
0answers
10 views
How do I find the data-point with respect to a given frame?
I've been reading this paper that formulates invariant task-parametrized HSMMs. In section 3.1 (Model Learning), the task parameters are represented in $F$ coordinate systems defined by $\{A_j,b_j\}_{...
1
vote
1answer
57 views
The mathematics in the CBOW and Skip-Gram models
this is my first question on AI Stack Exchange. I am a mathematics student who is learning NLP so I have paid a high amount of attention on the mathematics used in the subject, but my interpretations ...
0
votes
1answer
54 views
Is it possible to know the distance objects are from camera based on only knowing one object's height?
I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
2
votes
1answer
144 views
Why would the lookup table (of a table-driven artificial agent) need to store data at pixel precision?
While reading the book AI A modern approach, 4th ed, I came across the section of "Agent program" with following text:
It is instructive to consider why the table-driven approach to agent
...
2
votes
1answer
92 views
How to mathematically describe the convolution operation (with a Gaussian kernel)?
I have to build a model where I pre-process the data with a Gaussian kernel. The data are an $n\times n$ matrix (i.e one channel), but not an image, thus I can't refer to this matrix as an image and ...
6
votes
1answer
133 views
Formal definition of the Object Detection problem
For many problems in computer science, there is a formal, mathematical problem defition.
Something like: Given ..., the problem is to ...
How can the Object Detection problem (i.e. detecting objects ...
3
votes
0answers
20 views
Mapping given probabilities to empirical probabilities
Consider following problem statement:
You have given $n$ actions. You can perform any of them. Each action gives you success with some probability. The challenge is to perform given finite number of ...
2
votes
1answer
43 views
Research paths/areas for improving the performance of CNNs when faced with limited data
I've been reading through the research literature for image processing, computer vision, and convolutional neural networks. For image classification and object recognition, I know that convolutional ...
1
vote
1answer
69 views
What does $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ mean in the article Hindsight Experience Replay, section 2.1?
Taken from section 2.1 in the article:
We consider the standard reinforcement learning formalism consisting of an agent interacting with an environment. To simplify the exposition we assume that the ...
1
vote
0answers
42 views
Could the neural network automatically calculate and get different one-to-many quantities relative to their parent quantity?
Let's say I have a primary dataset that its secondary dataset is hundreds to match and group like an one-to-many relationship.
I'm new in this world of the AI but my problem is that many child groups ...
0
votes
0answers
30 views
Confusion on Math Notation Definition
I attempt to understand the formulation of dictionary learning for this paper:
Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
Multimodal Task-Driven ...
0
votes
1answer
51 views
Do the rows of the design matrix refer to the observations or predictors?
I attempt to understand the formulation of dictionary learning for this paper:
Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
Multimodal Task-Driven ...
2
votes
2answers
114 views
How does PCA work when we reduce the original space to 2 or higher-dimensional space?
How does PCA work when we reduce the original space to a 2 or higher-dimensional space? I understand the case when we reduce the dimensionality to $1$, but not this case.
$$\begin{array}{ll} \text{...
1
vote
1answer
49 views
Mathematical calculation behind decision tree classifier with continuous variables
Problem Description
I am working on a binary classification problem having continuous variables (Gene expression Values). My goal is to classify the samples as case ...
0
votes
0answers
56 views
Can a computer make a proof by induction?
Can a computer solve the following problem, i.e. make a proof by induction? And why?
Prove by induction that $$\sum_{k=1}^nk^3=\left(\frac{n(n+1)}{2}\right)^2, \, \, \, \forall n\in\mathbb N .$$
I'm ...
3
votes
1answer
123 views
Understand the DDPG algorithm in Keras
I'm trying to understand the DDPG algorithm using Keras
I found the site and started analyzing the code, I can't understand 2 things.
The algorithm used to write the code presented on the page
In the ...
1
vote
1answer
93 views
What is the definition of the “cost” function in the SVM's objective function?
In a course that I am attending, the cost function of a support vector machine is given by
$$J(\theta)=\sum_{i=1}^{m} y^{(i)} \operatorname{cost}_{1}\left(\theta^{T} x^{(i)}\right)+\left(1-y^{(i)}\...
1
vote
1answer
44 views
How to understand mapping function of kernel?
For a kernel function, we have two conditions one is that it should be symmetric which is easy to understand intuitively because dot products are symmetric as well and our kernel should also follow ...
1
vote
0answers
110 views
What is an auto-associator?
What is an auto-associator, and how does it work? How can we design an auto-associator for a given pattern? I couldn't find a clear explanation for this anywhere on the internet.
Here's an example of ...
1
vote
0answers
43 views
Simplifying Log Loss
I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
5
votes
1answer
152 views
Can deep learning be used to help mathematical research?
I am currently learning about deep learning and artificial intelligence and exploring his possibilities, and, as a mathematician at heart, I am inquisitive about how it can be used to solve problems ...
2
votes
0answers
73 views
Is this the correct gradient for log of softmax? [duplicate]
I am currently implementing the very basic version (REINFORCE) of the Monte Carlo policy gradient algorithm. I was wondering if this is the correct gradient for the log of softmax.
\begin{align}
\...
2
votes
1answer
119 views
Does Gödel's second incompleteness theorem put a limitation on artificial intelligence systems?
According to Brian Cantwell Smith
no calculation without representation
Therefore, computers depend on models. So, we can say that AI is limited internally by the model and externally by the ...
4
votes
1answer
89 views
How is the Jacobian a generalisation of the gradient?
I came across these slides Natural Language Processing with Deep Learning CS224N/Ling284, in the context of natural language processing, which talk about the Jacobian as a generalization of the ...
1
vote
0answers
35 views
How do I approach this problem?
Let's say I have a dataset with multiple types of multiple ingredients (salt1,salt2, etc). Each n-th variation of each ingredient vs flavor may be represented by an n×k matrix that where an ingredient ...
2
votes
1answer
78 views
How is the log-derivative trick of a trajectory derived?
I am looking at this formula which breaks down the gradient of $P(\tau |\theta)$ the first part is clear as is the derivative of $\log(x)$, but I do not see how the first formula is rearranged into ...
4
votes
1answer
68 views
How can a single sample represent the expectation in gradient temporal difference learning?
I was reading the gradient temporal difference learning version 2(GTD2) from rich Sutton's book page-246. At some point, he expressed the whole expectation using a single sample from the environment. ...
0
votes
1answer
43 views
Why are all weights of a neural net updated and not just the weights of the first layer
Why are all weights of a neural net updated and not only the weights of the first hidden layer?
The error-influence of the prediction by the weights of a neural net is calculated using the chain rule....
1
vote
1answer
96 views
Understanding relation between VC Symmetrization Lemma and Generalization Bounds
I am new in the field of Machine Learning so I wanted to start of by reading more about mathematics and history behind it.
I am currently reading, in my opinion, a very good and descriptive paper on ...
2
votes
1answer
50 views
How can I implement policy evaluation when reward is tied to an action outcome?
I'm following Stanford reinforcement learning videos on youtube. One of the assignments asks to write code for policy evaluation for Gym's FrozenLake-v0 environment.
In the course (and books I have ...
2
votes
1answer
80 views
What are the challenges faced by using NLP to convert mathematical texts into formal logic?
From what I've figured
(a) converting mathematical theorems and proofs from English to formal logic is a straightforward job for mathematicians with sufficient background, except that it takes time.
...
1
vote
1answer
53 views
What does equation in the “related work” section of the GAN paper mean?
I was going through the paper on GAN by Ian Goodfellow. Under the related work section, there is an equation. I cannot decipher the equation. Can anyone help me understand the meaning of the equation? ...
3
votes
0answers
42 views
Is maximum likelihood estimation meaningless for a dataset of only outliers?
From my understanding, maximum likelihood estimation chooses the set of parameters for the estimator that maximizes likelihood with the ground truth distribution.
I always interpreted it as the ...
2
votes
1answer
47 views
Is the derivative of the loss wrt a single scalar parameter proportional to the loss?
I am wondering about the correlation between the loss and the derivative of the loss wrt a single scalar parameter, with the same sample. That means: considering a machine learning model with ...
1
vote
0answers
19 views
How do I decide which norm to use for placing a constraint on my adversarial perturbation?
I am performing an adversarial machine learning attack on a neural network for network traffic classification. For adding adversarial perturbations in features such as packet interarrival times and ...
2
votes
0answers
28 views
Is there any wrong in my focal loss derivation?
Assume $\mathbf{X} \in R^{N, C}$ is the input of the softmax $\mathbf{P} \in R^{N, C}$, where $N$ is number of examples and $C$ is number of classes:
$$\mathbf{p}_i = \left[ \frac{e^{x_{ik}}}{\sum_{j=...
2
votes
2answers
121 views
Is ReLU a non-linear activation function?
According to this blog post
The purpose of an activation function is to add some kind of non-linear property to the function
The sigmoid is typically used as an activation function of a unit of a ...
3
votes
1answer
123 views
Why does the variational auto-encoder use the reconstruction loss?
VAE is trained to reduce the following two losses.
KL divergence between inferred latent distribution and Gaussian.
the reconstruction loss
I understand that the first one regularizes VAE to get ...
