Questions tagged [math]

For questions about mathematics related to artificial intelligence.

Filter by
Sorted by
Tagged with
3
votes
1answer
87 views

Standard deviation of the total input to a neuron

Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1 calculates the standard deviation of the output of a hidden neuron. From: $$ \sigma^2 = \sum^n_{i=0}E[w_i^2]E[x_i^2] $$ When I ...
4
votes
2answers
469 views

Viola Jones Algorithm

Can Viola Jones algorithm be used to detect the facial emotion. Actually it was used in creating harr-cascade file for object and facial detection, but what confused me is whether it can be used to ...
4
votes
0answers
116 views

What characteristics make it difficult for a Neural Network to approximate a function?

What are the characteristics which make a function difficult for the Neural Network to approximate? Intuitively, one might think uneven functions might be difficult to approximate, but uneven ...
2
votes
0answers
50 views

Should I use the hyperbolic distance loss in the case of Poincarè Disk Model?

I trained a neural network which makes a regression to a Poincarè Disk Model with radius $r = 1$. I want to optimize using the hyperbolic distance $$ \operatorname{arcosh} \left( 1 + \frac{2|pq|^2|...
6
votes
2answers
1k views

How is local minima possible in gradient descent?

Gradient descent works on the equation of mean squared error, which is an equation of a parabola $y=x^2$. We often say that weight adjustment in a neural network by gradient descent algorithm can hit ...
5
votes
2answers
114 views

Are on-line backpropagation iterations perpendicular to the constraint?

Raul Rojas' Neural Networks A Systematic Introduction, section 8.1.2 relates off-line backpropagation and on-line backpropagation with Gauss-Jacobi and Gauss-Seidel methods for finding the ...
4
votes
1answer
95 views

Why does the cost function contain a 2 at the denominator?

A cost function used in machine is often the following $$C = \frac{1}{2} \| y - \hat{y} \| ^2$$ Why is there $\frac{1}{2}$ in front of the squared distance?
1
vote
3answers
92 views

Is it possible to compute $P( F \mid S )$ given $P(F \mid S,A)$, $P(F \mid S, \lnot A)$?

I have a bayesian network, which has the following data: $P(S) = 0.07$ $P(A) = 0.01$ $P(F \mid S,A) = 1.0$ $P(F \mid S, \lnot A) = 0.7$ $P(F \mid \lnot S, A) = 0.9$ $P(F \mid \lnot S, \lnot A) =...
1
vote
1answer
66 views

Is the next state drawn from the joint distribution of the previous state and action?

Suppose $G_t$, the discounted return at time $t$ is defined as: $$ G_t \triangleq R_t+\gamma R_{t+1}+\gamma^{2}R_{t+2} + \cdots = \sum_{j=1}^{\infty} \gamma^{k}R_{t+k}$$ where $R_t$ is the reward at ...
2
votes
0answers
31 views

Calculating tangent vector of curve s(P,$\alpha$) at given point $\alpha$ = 0

I am reading the paper "Transformation Invariance in Pattern Recognition – Tangent Distance and Tangent Propagation", where the tangent vector is calculated for the given curve $s(P,\alpha)$ at $\...
5
votes
0answers
144 views

What are the main benefits of using Bayesian networks?

I have some trouble understanding the benefits of Bayesian networks. Am I correct that the key benefit of the network is that one does not need to use chain rule of probability in order to calculate ...
1
vote
0answers
41 views

Where does the expectation term in the derivative of the soft-max policy come from?

At slide 17 of the David Silver's series, the soft-max policy is defined as follows $$ \pi_\theta(s, a) \propto e^{\phi(s, a)^T \theta} $$ that is, the probability of an action $a$ (in state $s$) is ...
11
votes
5answers
5k views

Why do activation functions need to be differentiable in the context of neural networks?

Why should an activation function of a neural network be differentiable? Is it strictly necessary or is it just advantageous?
1
vote
1answer
175 views

How are vectors and matrices multiplied in supervised machine learning?

I've recently started reading a book about deep learning. The book is titled "Grokking Deep Learning" (by Andrew W Trask). In chapter 3 (pages 44 and 45), it talks about multiplying vectors using dot ...
30
votes
5answers
21k views

Is it possible to train the neural network to solve math equations?

I'm aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations? So given the ...
2
votes
1answer
291 views

Problems getting ADADELTA to converge

I have followed the pseudocode in the ADADELTA paper (top right on page 3), and wrote the following Python code for solving the optimization problem L(x) = x^2: ...
1
vote
0answers
26 views

Since there are different types of neurons in adjacent positions in the brain's arrays, should heterogeneous layers be developed?

Below is a taxonomy of neurons. Some of these types occur in different locations in the brain, but there are adjacent neurons of varying types with clearly functional type diversity in many parts of ...
2
votes
1answer
703 views

Is known math really enough for AI

As an Electronics & Communication Engineering student I've heard some stories and theories about "The math we have is not enough to complete a thinker-learner AI." What is the truth? Is humankind ...
6
votes
1answer
554 views

Is recursion used in practice to improve performance of AI systems?

Is there any methods by which artificial intelligence use recursion(s) to solve a certain issue or to keep up working and calculating?
2
votes
1answer
826 views

Are FFNN (MLP) Lipschitz functions?

My question is regarding standard dense-connected feed forward neural networks with sigmoidal activation. I am studying Bayesian Optimization for hyper-parameter selection for neural networks. There ...
3
votes
0answers
347 views

Solving equations using reinforcement learning

I was lately curious about a reinforcement learning approach that would solve maths equations. For example, if I have the following equation: $$ f(g(h(w))) = 0 , with \ w = \begin{matrix} a_{11} &...
4
votes
2answers
530 views

Why is the derivative 0 if the policy is deterministic?

In the Berkeley RL class they mention the gradient would be 0 if the policy is deterministic. Why is that? https://www.youtube.com/watch?v=XGmd3wcyDg8&feature=youtu.be&t=1071
1
vote
0answers
55 views

Optimization step in Apprenticeship Learning via Inverse Reinforcement Learning

Why the optimization step of the algorithm a quadratic program? [See: Apprenticeship Learning via Inverse Reinforcement Learning; page 3] Isn't the objective function linear? Why don't we treat ...
2
votes
0answers
28 views

Simple question about HS algorithm's formul(Optical flow)

In the below pic, I can not understand what U vector is? It says flow field but I can not imagie what really is the flow field?
1
vote
0answers
77 views

How does this sigma work?(Harris algorithm) [closed]

May someone explains some first iterations of this sigma? Also, how did it convert the above expression to below expression? What it the meaning of I(x) and I(y)?
3
votes
0answers
727 views

How to calculate gradient of filter in convolution network

I have similar architecture like in image:CNN. I don't understand how to calculate gradient of filter F. I found these equations(source): Gradient and delta, where first equation calculate gradient ...
-4
votes
2answers
691 views

What are the approaches to predict sequence of π numbers? [closed]

Given list of fixed numbers from a mathematical constant such as Pi, is it is possible to train AI to attempt to predict the next numbers? Which AI or neural network would be more suitable for this ...

1 2 3 4
5