Questions tagged [math]

For questions about mathematics related to artificial intelligence.

55 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
0answers
140 views

What are the main benefits of using Bayesian networks?

I have some trouble understanding the benefits of Bayesian networks. Am I correct that the key benefit of the network is that one does not need to use chain rule of probability in order to calculate ...
5
votes
1answer
222 views

Defining formula for fuzzy equation

I'm learning fuzzy logic and more or less understand the basic concept, but i'm having a hard time understanding how to apply it to a method. I tried browsing online for explanation on how to use it, ...
4
votes
0answers
87 views

Is there a mathematical formula that describes the learning curve in neural networks?

In training a neural network, you often see the curve showing how fast the neural network is getting better. It usually grows very fast then slows down to almost horizontal. Is there a mathematical ...
4
votes
0answers
110 views

What characteristics make it difficult for a Neural Network to approximate a function?

What are the characteristics which make a function difficult for the Neural Network to approximate? Intuitively, one might think uneven functions might be difficult to approximate, but uneven ...
3
votes
0answers
21 views

Mapping given probabilities to empirical probabilities

Consider following problem statement: You have given $n$ actions. You can perform any of them. Each action gives you success with some probability. The challenge is to perform given finite number of ...
3
votes
0answers
42 views

Is maximum likelihood estimation meaningless for a dataset of only outliers?

From my understanding, maximum likelihood estimation chooses the set of parameters for the estimator that maximizes likelihood with the ground truth distribution. I always interpreted it as the ...
3
votes
0answers
47 views

How does the memory mechanism (reading and writing) work in a neural Turing machine?

In neural Turing machine (NTM), reading memory is represented as \begin{align} r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2} \end{align} and writing to memory is represented as ...
3
votes
0answers
100 views

What is the meaning of the words 'bias' and 'variance' in RL?

In reinforcement learning approaches, like temporal-difference (TD) learning or Monte Carlo methods, two of the metrics used to measure their performance are the bias and the variance. What do these ...
3
votes
0answers
332 views

Solving equations using reinforcement learning

I was lately curious about a reinforcement learning approach that would solve maths equations. For example, if I have the following equation: $$ f(g(h(w))) = 0 , with \ w = \begin{matrix} a_{11} &...
3
votes
0answers
720 views

How to calculate gradient of filter in convolution network

I have similar architecture like in image:CNN. I don't understand how to calculate gradient of filter F. I found these equations(source): Gradient and delta, where first equation calculate gradient ...
2
votes
0answers
30 views

Is there any wrong in my focal loss derivation?

Assume $\mathbf{X} \in R^{N, C}$ is the input of the softmax $\mathbf{P} \in R^{N, C}$, where $N$ is number of examples and $C$ is number of classes: $$\mathbf{p}_i = \left[ \frac{e^{x_{ik}}}{\sum_{j=...
2
votes
0answers
58 views

Why are conics important in computer vision?

The book Multiple View Geometry in Computer Vision by Richard Hartley and Andrew Zisserman talks about lines, points and conics. A conic is a curve described by a second-degree equation in the plane, ...
2
votes
0answers
43 views

Expected duration in a state

I am going through Rabiner 1989 and he writes that the discrete probability density function of duration $d$ in state $i$ (that is, staying in a state for duration $d$, conditioned on starting in that ...
2
votes
0answers
149 views

How does the update rule for the one-step actor-critic method work?

Can you please elucidate the math behind the update rule for the critic? I've seen in other places that just a squared distance of $R + \hat{v}(S', w) - \hat{v}(S,w)$ is used, but Sutton suggests an ...
2
votes
0answers
49 views

Should I use the hyperbolic distance loss in the case of Poincarè Disk Model?

I trained a neural network which makes a regression to a Poincarè Disk Model with radius $r = 1$. I want to optimize using the hyperbolic distance $$ \operatorname{arcosh} \left( 1 + \frac{2|pq|^2|...
2
votes
0answers
31 views

Calculating tangent vector of curve s(P,$\alpha$) at given point $\alpha$ = 0

I am reading the paper "Transformation Invariance in Pattern Recognition – Tangent Distance and Tangent Propagation", where the tangent vector is calculated for the given curve $s(P,\alpha)$ at $\...
2
votes
0answers
163 views

Choice of fuzzification function

I'm a relative newbie to fuzzie logic systems but I have some knowledge in mathematics. I have the following problem: I want to fuzzify certain values. Some are in the range [-$\inf$,$\inf$] and some ...
2
votes
0answers
28 views

Simple question about HS algorithm's formul(Optical flow)

In the below pic, I can not understand what U vector is? It says flow field but I can not imagie what really is the flow field?
1
vote
0answers
32 views

Could the inputs of the mean squared-error loss function be transformed to allow larger learning rates?

In the context of a neural network $\hat{y} = f_\theta(\mathbf{x})$ with parameters $\theta$ that is trained to perform regression such that the prediction $\hat{\mathbf{y}} = [\hat{y}_1,\hat{y}_2,...,...
1
vote
0answers
17 views

Clonal operator in Immune Clonal Strategy

I was reading about Immune Clonal Strategy, specifically about Monoclonal operator from Immunity clonal strategies, and it goes as follows: Here $a_i $ is a point and $a_i = \{ x_1, x_2, \cdots, x_m \...
1
vote
1answer
52 views

Is there a full and precise formulation of Theorem 1 in the Integrated Gradients paper?

Theorem 1 (page 5) in the paper about Integrated Gradients states that Integrated gradients is the unique path method that is symmetry-preserving. What I miss is A precise formulation of the ...
1
vote
0answers
33 views

Are monotonically increasing functions easier to learn?

A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant. Logically this seems like ...
1
vote
1answer
59 views

Explanation of this L2 minimization equation

I am trying to understand the last two lines of this math notation. How Var and double summation of Cov came to the equation. The first two lines I understood something like $(a-b)^2 = a^2 -2ab +b^2$.
1
vote
0answers
35 views

Can any area of math come into play in Machine Learning Research?

As I read online following areas in mathematics comes into play in ML research Linear Algebra Calculus Differential Equations Probability Statistics Discrete Mathematics Optimization Analytic ...
1
vote
0answers
32 views

How do I derive the gradient of the log-likelihood of an RBM?

In a Restricted Boltzmann Machine (RBM), the likelihood function is: $$p(\mathbf{v};\mathbf{\theta}) = \frac{1}{Z} \sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}$$ Where $E$ is the ...
1
vote
0answers
11 views

How do I find the data-point with respect to a given frame?

I've been reading this paper that formulates invariant task-parametrized HSMMs. In section 3.1 (Model Learning), the task parameters are represented in $F$ coordinate systems defined by $\{A_j,b_j\}_{...
1
vote
0answers
42 views

Could the neural network automatically calculate and get different one-to-many quantities relative to their parent quantity?

Let's say I have a primary dataset that its secondary dataset is hundreds to match and group like an one-to-many relationship. I'm new in this world of the AI but my problem is that many child groups ...
1
vote
0answers
167 views

What is an auto-associator?

What is an auto-associator, and how does it work? How can we design an auto-associator for a given pattern? I couldn't find a clear explanation for this anywhere on the internet. Here's an example of ...
1
vote
0answers
44 views

Simplifying Log Loss

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
1
vote
0answers
35 views

How do I approach this problem?

Let's say I have a dataset with multiple types of multiple ingredients (salt1,salt2, etc). Each n-th variation of each ingredient vs flavor may be represented by an n×k matrix that where an ingredient ...
1
vote
0answers
21 views

How do I decide which norm to use for placing a constraint on my adversarial perturbation?

I am performing an adversarial machine learning attack on a neural network for network traffic classification. For adding adversarial perturbations in features such as packet interarrival times and ...
1
vote
1answer
279 views

In a single neuron output layer should the output be a scalar?

Given a neural network with 3 inputs, 4 hidden layers, and 1 output, should the output neuron be a vector or a scalar? I thought that at the end of the summation only one number between 0 and 1 would ...
1
vote
2answers
198 views

Formal proof that every purely reactive agent has behaviorally equivalent standard agent

It kind of makes sense intuitively but I'm not sure about a formal proof. I'll start with briefly listing definitions from Intro to Multiagent systems, Wooldridge, 2002 and then give you my reasoning ...
1
vote
0answers
97 views

Is Gradient Descent algorithm a part of Calculus of Variations?

As in https://en.wikipedia.org/wiki/Calculus_of_variations The calculus of variations is a field of mathematical analysis that uses variations, which are small changes in functions and ...
1
vote
1answer
77 views

Given an axis-angle rotation vector, how can I find the unit rotation axis and angle?

I have a robotics assignment, which I am unable to solve. Given the axis-angle rotation vector $\Theta = (2, 2, 0)$, how can I calculate the unit vector of the rotation axis $k$ and the angle $\theta$?...
1
vote
0answers
41 views

Where does the expectation term in the derivative of the soft-max policy come from?

At slide 17 of the David Silver's series, the soft-max policy is defined as follows $$ \pi_\theta(s, a) \propto e^{\phi(s, a)^T \theta} $$ that is, the probability of an action $a$ (in state $s$) is ...
1
vote
0answers
26 views

Since there are different types of neurons in adjacent positions in the brain's arrays, should heterogeneous layers be developed?

Below is a taxonomy of neurons. Some of these types occur in different locations in the brain, but there are adjacent neurons of varying types with clearly functional type diversity in many parts of ...
1
vote
0answers
55 views

Optimization step in Apprenticeship Learning via Inverse Reinforcement Learning

Why the optimization step of the algorithm a quadratic program? [See: Apprenticeship Learning via Inverse Reinforcement Learning; page 3] Isn't the objective function linear? Why don't we treat ...
0
votes
0answers
13 views

Are there any good references that describe the equations of the forward pass of Graph Neural Networks?

I am trying to program Graph Neural Network from scratch. Can the community please suggest a good reference/s to read about the equations of the forward pass in Graph Neural Networks, especially in ...
0
votes
0answers
17 views

(Deep) feature engineering for lambda terms (mathematical expressions, higher order logic formulas) - is such thing?

Automated theorem proving with (deep) reinforcement learning (DRL) approach is hot topic in current AI research when domains of games are becoming saturated and completed research topcis. E.g. there ...
0
votes
1answer
46 views

Why don't integrated gradients explain samples correctly?

I have a linear tabular dataset made of floats. The dataset follows a simple rule like: ...
0
votes
0answers
17 views

How do we get from entropy to KL divergence in this paper?

I'm reading through Regularizing Neural Networks By Penalizing Confident Output Distributions and I'm stuck on the equation in section 3.2. It's not clear to me at all that the self-entropy of the ...
0
votes
0answers
22 views

Do $V_\theta$ and $V_s$ represent partial or total derivatives in the paper “Learning Continuous Control Policies by Stochastic Value Gradients”?

I was reading up on the Stochastic Value Gradients paper by Heess et al. In the paper, they describe a recursive process to calculate path-wise derivatives via equations (3) and (4), at the bottom of ...
0
votes
0answers
42 views

What are the math theorems regarding the Multilayer Perceptron?

I've come across a theorem "Convergence theorem Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish) ...
0
votes
1answer
60 views

Why can a neural network use more than one activation function?

From trying to understand neural networks better, I've come upon a tentative notion that an activation function aims to build a function it's approximating via linear combinations with biases and ...
0
votes
0answers
12 views

Is there an arrow missing in the derivation of front-door adjustment formula from do-calculus?

Here is Judea Pearl's derivation of the front-door adjustment formula: Is there an arrow from Genotype to Cancer missing in the second diagram at the right? just like this?
0
votes
0answers
22 views

Meaning of grad_outputs in torch.autograd.grad for complex input and output

Let's say we have a mathematical expression, $$ \mathbf{y} = \mathbf{Ax}, $$ where $\mathbf{y}$ and $\mathbf{x}$ are a vector, and $\mathbf{A}$ is a matrix. Let's say the vector $\mathbf{y}$ is used ...
0
votes
0answers
16 views

Is parameter sharing in AlBERT akin to repeated application of same function on input?

I read the AlBERT and saw that its architecture used "Parameter Sharing" among layers of the encoder. They mentioned that this was done to save model space, make fewer training parameters ...
0
votes
0answers
42 views

Can I solve the below functional equation using neural networks?

I recently watched this video, in which he solves the equation $$f(x)+f\left(\frac{1}{1-x}\right) = x$$ The answer is $$f(x) = \frac{x^3-x+1}{2x(x-1)}$$ I tried to solve this functional equation using ...
0
votes
1answer
61 views

Is it possible to know the distance objects are from camera based on only knowing one object's height?

I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...