Questions tagged [math]
For questions about mathematics related to artificial intelligence.
43
questions with no upvoted or accepted answers
4
votes
0answers
81 views
Is there a mathematical formula that describes the learning curve in neural networks?
In training a neural network, you often see the curve showing how fast the neural network is getting better. It usually grows very fast then slows down to almost horizontal.
Is there a mathematical ...
4
votes
0answers
102 views
What characteristics make it difficult for a Neural Network to approximate a function?
What are the characteristics which make a function difficult for the Neural Network to approximate?
Intuitively, one might think uneven functions might be difficult to approximate, but uneven ...
4
votes
0answers
112 views
What are the main benefits of using Bayesian networks?
I have some trouble understanding the benefits of Bayesian networks.
Am I correct that the key benefit of the network is that one does not need to use
chain rule of probability in order to calculate ...
4
votes
1answer
212 views
Defining formula for fuzzy equation
I'm learning fuzzy logic and more or less understand the basic concept, but i'm having a hard time understanding how to apply it to a method. I tried browsing online for explanation on how to use it, ...
3
votes
0answers
20 views
Mapping given probabilities to empirical probabilities
Consider following problem statement:
You have given $n$ actions. You can perform any of them. Each action gives you success with some probability. The challenge is to perform given finite number of ...
3
votes
0answers
42 views
Is maximum likelihood estimation meaningless for a dataset of only outliers?
From my understanding, maximum likelihood estimation chooses the set of parameters for the estimator that maximizes likelihood with the ground truth distribution.
I always interpreted it as the ...
3
votes
0answers
46 views
How does the memory mechanism (reading and writing) work in a neural Turing machine?
In neural Turing machine (NTM), reading memory is represented as
\begin{align}
r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2}
\end{align}
and writing to memory is represented as
...
3
votes
0answers
88 views
What is the meaning of the words 'bias' and 'variance' in RL?
In reinforcement learning approaches, like temporal-difference (TD) learning or Monte Carlo methods, two of the metrics used to measure their performance are the bias and the variance.
What do these ...
3
votes
0answers
309 views
Solving equations using reinforcement learning
I was lately curious about a reinforcement learning approach that would solve maths equations.
For example, if I have the following equation:
$$ f(g(h(w))) = 0 , with \ w = \begin{matrix}
a_{11} &...
3
votes
0answers
689 views
How to calculate gradient of filter in convolution network
I have similar architecture like in image:CNN.
I don't understand how to calculate gradient of filter F.
I found these equations(source):
Gradient and delta,
where first equation calculate gradient ...
2
votes
0answers
28 views
Is there any wrong in my focal loss derivation?
Assume $\mathbf{X} \in R^{N, C}$ is the input of the softmax $\mathbf{P} \in R^{N, C}$, where $N$ is number of examples and $C$ is number of classes:
$$\mathbf{p}_i = \left[ \frac{e^{x_{ik}}}{\sum_{j=...
2
votes
0answers
50 views
Why are conics important in computer vision?
The book Multiple View Geometry in Computer Vision by Richard Hartley and Andrew Zisserman talks about lines, points and conics. A conic is a curve described by a second-degree equation in the plane, ...
2
votes
0answers
42 views
Expected duration in a state
I am going through Rabiner 1989 and he writes that the discrete probability density function of duration $d$ in state $i$ (that is, staying in a state for duration $d$, conditioned on starting in that ...
2
votes
0answers
112 views
How does the update rule for the one-step actor-critic method work?
Can you please elucidate the math behind the update rule for the critic? I've seen in other places that just a squared distance of $R + \hat{v}(S', w) - \hat{v}(S,w)$ is used, but Sutton suggests an ...
2
votes
1answer
106 views
Why do we use the word “kernel” in the expression “Gaussian kernel”?
I've heard the expression "Gaussian kernel" in several contexts (e.g. in the kernel trick used in SVM). A Gaussian kernel usually refers to a Gaussian function (that is, a function similar to the ...
2
votes
0answers
47 views
Should I use the hyperbolic distance loss in the case of Poincarè Disk Model?
I trained a neural network which makes a regression to a Poincarè Disk Model with radius $r = 1$.
I want to optimize using the hyperbolic distance
$$
\operatorname{arcosh} \left( 1 + \frac{2|pq|^2|...
2
votes
0answers
30 views
Calculating tangent vector of curve s(P,$\alpha$) at given point $\alpha$ = 0
I am reading the paper "Transformation Invariance in Pattern Recognition – Tangent Distance and Tangent Propagation", where the tangent vector is calculated for the given curve $s(P,\alpha)$ at $\...
2
votes
0answers
159 views
Choice of fuzzification function
I'm a relative newbie to fuzzie logic systems but I have some knowledge in mathematics. I have the following problem:
I want to fuzzify certain values. Some are in the range [-$\inf$,$\inf$] and some ...
1
vote
0answers
28 views
Are monotonically increasing functions easier to learn?
A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant.
Logically this seems like ...
1
vote
1answer
44 views
Explanation of this L2 minimization equation
I am trying to understand the last two lines of this math notation. How Var and double summation of Cov came to the equation. The first two lines I understood something like $(a-b)^2 = a^2 -2ab +b^2$.
1
vote
0answers
31 views
Can any area of math come into play in Machine Learning Research?
As I read online following areas in mathematics comes into play in ML research
Linear Algebra
Calculus
Differential Equations
Probability
Statistics
Discrete Mathematics
Optimization
Analytic ...
1
vote
0answers
30 views
How do I derive the gradient of the log-likelihood of an RBM?
In a Restricted Boltzmann Machine (RBM), the likelihood function is:
$$p(\mathbf{v};\mathbf{\theta}) = \frac{1}{Z} \sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}$$
Where $E$ is the ...
1
vote
0answers
10 views
How do I find the data-point with respect to a given frame?
I've been reading this paper that formulates invariant task-parametrized HSMMs. In section 3.1 (Model Learning), the task parameters are represented in $F$ coordinate systems defined by $\{A_j,b_j\}_{...
1
vote
0answers
42 views
Could the neural network automatically calculate and get different one-to-many quantities relative to their parent quantity?
Let's say I have a primary dataset that its secondary dataset is hundreds to match and group like an one-to-many relationship.
I'm new in this world of the AI but my problem is that many child groups ...
1
vote
0answers
113 views
What is an auto-associator?
What is an auto-associator, and how does it work? How can we design an auto-associator for a given pattern? I couldn't find a clear explanation for this anywhere on the internet.
Here's an example of ...
1
vote
0answers
43 views
Simplifying Log Loss
I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
1
vote
0answers
35 views
How do I approach this problem?
Let's say I have a dataset with multiple types of multiple ingredients (salt1,salt2, etc). Each n-th variation of each ingredient vs flavor may be represented by an n×k matrix that where an ingredient ...
1
vote
0answers
19 views
How do I decide which norm to use for placing a constraint on my adversarial perturbation?
I am performing an adversarial machine learning attack on a neural network for network traffic classification. For adding adversarial perturbations in features such as packet interarrival times and ...
1
vote
1answer
193 views
In a single neuron output layer should the output be a scalar?
Given a neural network with 3 inputs, 4 hidden layers, and 1 output, should the output neuron be a vector or a scalar? I thought that at the end of the summation only one number between 0 and 1 would ...
1
vote
2answers
134 views
Formal proof that every purely reactive agent has behaviorally equivalent standard agent
It kind of makes sense intuitively but I'm not sure about a formal proof. I'll start with briefly listing definitions from Intro to Multiagent systems, Wooldridge, 2002 and then give you my reasoning ...
1
vote
0answers
72 views
Is Gradient Descent algorithm a part of Calculus of Variations?
As in https://en.wikipedia.org/wiki/Calculus_of_variations
The calculus of variations is a field of mathematical analysis that
uses variations, which are small changes in functions and ...
1
vote
1answer
65 views
Given an axis-angle rotation vector, how can I find the unit rotation axis and angle?
I have a robotics assignment, which I am unable to solve. Given the axis-angle rotation vector $\Theta = (2, 2, 0)$, how can I calculate the unit vector of the rotation axis $k$ and the angle $\theta$?...
1
vote
0answers
41 views
Where does the expectation term in the derivative of the soft-max policy come from?
At slide 17 of the David Silver's series, the soft-max policy is defined as follows
$$
\pi_\theta(s, a) \propto e^{\phi(s, a)^T \theta}
$$
that is, the probability of an action $a$ (in state $s$) is ...
1
vote
0answers
26 views
Since there are different types of neurons in adjacent positions in the brain's arrays, should heterogeneous layers be developed?
Below is a taxonomy of neurons. Some of these types occur in different locations in the brain, but there are adjacent neurons of varying types with clearly functional type diversity in many parts of ...
1
vote
0answers
53 views
Optimization step in Apprenticeship Learning via Inverse Reinforcement Learning
Why the optimization step of the algorithm a quadratic program? [See: Apprenticeship Learning via Inverse Reinforcement Learning; page 3]
Isn't the objective function linear? Why don't we treat ...
1
vote
0answers
27 views
Simple question about HS algorithm's formul(Optical flow)
In the below pic, I can not understand what U vector is? It says flow field but I can not imagie what really is the flow field?
0
votes
0answers
15 views
Is parameter sharing in AlBERT akin to repeated application of same function on input?
I read the AlBERT and saw that its architecture used "Parameter Sharing" among layers of the encoder. They mentioned that this was done to save model space, make fewer training parameters ...
0
votes
0answers
39 views
Can I solve the below functional equation using neural networks?
I recently watched this video, in which he solves the equation
$$f(x)+f\left(\frac{1}{1-x}\right) = x$$
The answer is
$$f(x) = \frac{x^3-x+1}{2x(x-1)}$$
I tried to solve this functional equation using ...
0
votes
1answer
54 views
Is it possible to know the distance objects are from camera based on only knowing one object's height?
I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
0
votes
0answers
30 views
Confusion on Math Notation Definition
I attempt to understand the formulation of dictionary learning for this paper:
Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
Multimodal Task-Driven ...
0
votes
0answers
57 views
Can a computer make a proof by induction?
Can a computer solve the following problem, i.e. make a proof by induction? And why?
Prove by induction that $$\sum_{k=1}^nk^3=\left(\frac{n(n+1)}{2}\right)^2, \, \, \, \forall n\in\mathbb N .$$
I'm ...
0
votes
0answers
67 views
Derivation of regularized cost function w.r.t activation and bias
In regularzied cost function a L2 regularization cost has been added.
Here we have already calculated cross entropy cost w.r.t $A, W$.
As mentioned in the regularization notebook (see below) in ...
0
votes
0answers
47 views
Understanding V- and Q-functions
Assume the existence of a Markov Decision Process consisting of:
State space $S$
Action space $A$
Transition model $T: S \times A \times S \to [0,1]$
Reward function $R: S \times A \times S \to \...
