Questions tagged [math]
For questions about mathematics related to artificial intelligence.
69
questions with no upvoted or accepted answers
6
votes
0answers
157 views
What are the main benefits of using Bayesian networks?
I have some trouble understanding the benefits of Bayesian networks.
Am I correct that the key benefit of the network is that one does not need to use
chain rule of probability in order to calculate ...
5
votes
1answer
235 views
Defining formula for fuzzy equation
I'm learning fuzzy logic and more or less understand the basic concept, but i'm having a hard time understanding how to apply it to a method. I tried browsing online for explanation on how to use it, ...
4
votes
0answers
93 views
Is there a mathematical formula that describes the learning curve in neural networks?
In training a neural network, you often see the curve showing how fast the neural network is getting better. It usually grows very fast then slows down to almost horizontal.
Is there a mathematical ...
4
votes
0answers
130 views
What characteristics make it difficult for a Neural Network to approximate a function?
What are the characteristics which make a function difficult for the Neural Network to approximate?
Intuitively, one might think uneven functions might be difficult to approximate, but uneven ...
3
votes
0answers
21 views
Mapping given probabilities to empirical probabilities
Consider following problem statement:
You have given $n$ actions. You can perform any of them. Each action gives you success with some probability. The challenge is to perform given finite number of ...
3
votes
0answers
43 views
Is maximum likelihood estimation meaningless for a dataset of only outliers?
From my understanding, maximum likelihood estimation chooses the set of parameters for the estimator that maximizes likelihood with the ground truth distribution.
I always interpreted it as the ...
3
votes
0answers
47 views
How does the memory mechanism (reading and writing) work in a neural Turing machine?
In neural Turing machine (NTM), reading memory is represented as
\begin{align}
r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2}
\end{align}
and writing to memory is represented as
...
3
votes
0answers
109 views
What is the meaning of the words 'bias' and 'variance' in RL?
In reinforcement learning approaches, like temporal-difference (TD) learning or Monte Carlo methods, two of the metrics used to measure their performance are the bias and the variance.
What do these ...
3
votes
0answers
360 views
Solving equations using reinforcement learning
I was lately curious about a reinforcement learning approach that would solve maths equations.
For example, if I have the following equation:
$$ f(g(h(w))) = 0 , with \ w = \begin{matrix}
a_{11} &...
3
votes
0answers
734 views
How to calculate gradient of filter in convolution network
I have similar architecture like in image:CNN.
I don't understand how to calculate gradient of filter F.
I found these equations(source):
Gradient and delta,
where first equation calculate gradient ...
2
votes
0answers
68 views
REINFORCE differentiation on sum or single value?
I'm currently learning Policy-gradient Methods for RL and encountered REINFORCE algorithm. I learned from this site : https://towardsdatascience.com/policy-gradient-methods-104c783251e0 that the ...
2
votes
0answers
23 views
Is there any closed form analytical expression to represent fractional max pooling?
There are Nineteen types of pooling layers in PyTorch.
Almost all of the layers are provided with corresponding analytical formulae. But analytical formulae are not provided for the fractional max-...
2
votes
0answers
36 views
Could the inputs of the mean squared-error loss function be transformed to allow larger learning rates?
In the context of a neural network $\hat{y} = f_\theta(\mathbf{x})$ with parameters $\theta$ that is trained to perform regression such that the prediction $\hat{\mathbf{y}} = [\hat{y}_1,\hat{y}_2,...,...
2
votes
0answers
35 views
Are monotonically increasing functions easier to learn?
A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant.
Logically this seems like ...
2
votes
0answers
31 views
Is there any wrong in my focal loss derivation?
Assume $\mathbf{X} \in R^{N, C}$ is the input of the softmax $\mathbf{P} \in R^{N, C}$, where $N$ is number of examples and $C$ is number of classes:
$$\mathbf{p}_i = \left[ \frac{e^{x_{ik}}}{\sum_{j=...
2
votes
0answers
93 views
Why are conics important in computer vision?
The book Multiple View Geometry in Computer Vision by Richard Hartley and Andrew Zisserman talks about lines, points and conics. A conic is a curve described by a second-degree equation in the plane, ...
2
votes
0answers
43 views
Expected duration in a state
I am going through Rabiner 1989 and he writes that the discrete probability density function of duration $d$ in state $i$ (that is, staying in a state for duration $d$, conditioned on starting in that ...
2
votes
0answers
122 views
Is Gradient Descent algorithm a part of Calculus of Variations?
As in https://en.wikipedia.org/wiki/Calculus_of_variations
The calculus of variations is a field of mathematical analysis that
uses variations, which are small changes in functions and ...
2
votes
0answers
194 views
How does the update rule for the one-step actor-critic method work?
Can you please elucidate the math behind the update rule for the critic? I've seen in other places that just a squared distance of $R + \hat{v}(S', w) - \hat{v}(S,w)$ is used, but Sutton suggests an ...
2
votes
0answers
59 views
Should I use the hyperbolic distance loss in the case of Poincarè Disk Model?
I trained a neural network which makes a regression to a Poincarè Disk Model with radius $r = 1$.
I want to optimize using the hyperbolic distance
$$
\operatorname{arcosh} \left( 1 + \frac{2|pq|^2|...
2
votes
0answers
31 views
Calculating tangent vector of curve s(P,$\alpha$) at given point $\alpha$ = 0
I am reading the paper "Transformation Invariance in Pattern Recognition – Tangent Distance and Tangent Propagation", where the tangent vector is calculated for the given curve $s(P,\alpha)$ at $\...
2
votes
0answers
166 views
Choice of fuzzification function
I'm a relative newbie to fuzzie logic systems but I have some knowledge in mathematics. I have the following problem:
I want to fuzzify certain values. Some are in the range [-$\inf$,$\inf$] and some ...
2
votes
0answers
28 views
Simple question about HS algorithm's formul(Optical flow)
In the below pic, I can not understand what U vector is? It says flow field but I can not imagie what really is the flow field?
1
vote
0answers
33 views
Is it true that real world data is highly discontinuous?
A function $f$ is said to be continuous at a point $c$ if it satisfies three properties:
Should be defined at the point $c$
Left and right-hand limits at $c$ must be equal i.e., the limit must exist
...
1
vote
0answers
17 views
Is there any concept like 'applying affine transformation on multiple inputs'?
Affine transformation on $X$ is a transformation of the following form
$$Y = wX + b$$
In general, $w, X, Y$ and $b$ tensors.
We generally call tensor $X$ as an input to affine transformation or the ...
1
vote
0answers
43 views
How can I compute a mathematical formula for my CNN?
Let's say, for example, I have built the following CNN model using Keras:
...
1
vote
0answers
47 views
What are the math theorems regarding the Multilayer Perceptron?
I've come across a theorem "Convergence theorem
Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish)
...
1
vote
0answers
20 views
Clonal operator in Immune Clonal Strategy
I was reading about Immune Clonal Strategy, specifically about Monoclonal operator from Immunity clonal strategies, and it goes as follows:
Here $a_i $ is a point and $a_i = \{ x_1, x_2, \cdots, x_m \...
1
vote
0answers
49 views
Can I solve the below functional equation using neural networks?
I recently watched this video, in which he solves the equation
$$f(x)+f\left(\frac{1}{1-x}\right) = x$$
The answer is
$$f(x) = \frac{x^3-x+1}{2x(x-1)}$$
I tried to solve this functional equation using ...
1
vote
1answer
77 views
Explanation of this L2 minimization equation
I am trying to understand the last two lines of this math notation (from this paper).
How did Var and double summation of Cov come to the equation?
The first two lines I understood something like $(a-...
1
vote
0answers
37 views
Can any area of math come into play in Machine Learning Research?
As I read online following areas in mathematics comes into play in ML research
Linear Algebra
Calculus
Differential Equations
Probability
Statistics
Discrete Mathematics
Optimization
Analytic ...
1
vote
0answers
41 views
How do I derive the gradient of the log-likelihood of an RBM?
In a Restricted Boltzmann Machine (RBM), the likelihood function is:
$$p(\mathbf{v};\mathbf{\theta}) = \frac{1}{Z} \sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}$$
Where $E$ is the ...
1
vote
0answers
13 views
How do I find the data-point with respect to a given frame?
I've been reading this paper that formulates invariant task-parametrized HSMMs. In section 3.1 (Model Learning), the task parameters are represented in $F$ coordinate systems defined by $\{A_j,b_j\}_{...
1
vote
1answer
80 views
Is it possible to know the distance objects are from camera based on only knowing one object's height?
I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
1
vote
0answers
49 views
Could the neural network automatically calculate and get different one-to-many quantities relative to their parent quantity?
Let's say I have a primary dataset that its secondary dataset is hundreds to match and group like an one-to-many relationship.
I'm new in this world of the AI but my problem is that many child groups ...
1
vote
0answers
214 views
What is an auto-associator?
What is an auto-associator, and how does it work? How can we design an auto-associator for a given pattern? I couldn't find a clear explanation for this anywhere on the internet.
Here's an example of ...
1
vote
0answers
45 views
Simplifying Log Loss
I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
1
vote
0answers
37 views
How do I approach this problem?
Let's say I have a dataset with multiple types of multiple ingredients (salt1,salt2, etc). Each n-th variation of each ingredient vs flavor may be represented by an n×k matrix that where an ingredient ...
1
vote
0answers
24 views
How do I decide which norm to use for placing a constraint on my adversarial perturbation?
I am performing an adversarial machine learning attack on a neural network for network traffic classification. For adding adversarial perturbations in features such as packet interarrival times and ...
1
vote
1answer
356 views
In a single neuron output layer should the output be a scalar?
Given a neural network with 3 inputs, 4 hidden layers, and 1 output, should the output neuron be a vector or a scalar? I thought that at the end of the summation only one number between 0 and 1 would ...
1
vote
2answers
241 views
Formal proof that every purely reactive agent has behaviorally equivalent standard agent
It kind of makes sense intuitively but I'm not sure about a formal proof. I'll start with briefly listing definitions from Intro to Multiagent systems, Wooldridge, 2002 and then give you my reasoning ...
1
vote
1answer
91 views
Given an axis-angle rotation vector, how can I find the unit rotation axis and angle?
I have a robotics assignment, which I am unable to solve. Given the axis-angle rotation vector $\Theta = (2, 2, 0)$, how can I calculate the unit vector of the rotation axis $k$ and the angle $\theta$?...
1
vote
0answers
43 views
Where does the expectation term in the derivative of the soft-max policy come from?
At slide 17 of the David Silver's series, the soft-max policy is defined as follows
$$
\pi_\theta(s, a) \propto e^{\phi(s, a)^T \theta}
$$
that is, the probability of an action $a$ (in state $s$) is ...
1
vote
0answers
27 views
Since there are different types of neurons in adjacent positions in the brain's arrays, should heterogeneous layers be developed?
Below is a taxonomy of neurons. Some of these types occur in different locations in the brain, but there are adjacent neurons of varying types with clearly functional type diversity in many parts of ...
1
vote
0answers
55 views
Optimization step in Apprenticeship Learning via Inverse Reinforcement Learning
Why the optimization step of the algorithm a quadratic program? [See: Apprenticeship Learning via Inverse Reinforcement Learning; page 3]
Isn't the objective function linear? Why don't we treat ...
0
votes
1answer
44 views
What is the definition of a trace of a tensor?
Tensor is a multi-dimensional ordered collection of elements, which is used in deep learning to store and process data as well as intermediate steps.
We are aware of the trace of a two-dimensional ...
0
votes
0answers
69 views
What's wrong with my understanding of how RNNs work?
Recently, I've been trying to derive the mathematics behind various Neural Network structures. I managed to derive the MLP and tested it to be on par with a Keras implementation (Using the MNIST ...
0
votes
0answers
24 views
What are the different types of geometry in literature that may be used for deep learning?
Recently, I asked a question on the concept of a manifold and received an answer that points to a relatively new subfield of deep learning named geometric deep learning.
In the ...
0
votes
0answers
36 views
What are mathematically the factors of variation in deep learning?
The following paragraph from an answer tells us about factors of variation
Factors of variation are some factors which determine varieties in observed data. If that factors change, the behaviour of ...
0
votes
0answers
30 views
What should the value of $ρ$ in the $w(n+1) = w(n) + \rho*\text{error}(i)x(i)$ formula of Least Mean Squares be?
I am trying to better understand the Least Mean Squares algorithm, in order to implement it programmatically.
If we consider its weight updating formula $$w(n+1) = w(n) + \rho * \text{error}(i)x(i),$$ ...
