Questions tagged [math]

For questions about mathematics related to artificial intelligence.

Filter by
Sorted by
Tagged with
0
votes
1answer
46 views

Why don't integrated gradients explain samples correctly?

I have a linear tabular dataset made of floats. The dataset follows a simple rule like: ...
0
votes
0answers
13 views

Are there any good references that describe the equations of the forward pass of Graph Neural Networks?

I am trying to program Graph Neural Network from scratch. Can the community please suggest a good reference/s to read about the equations of the forward pass in Graph Neural Networks, especially in ...
3
votes
1answer
128 views

Why do we use the word “kernel” in the expression “Gaussian kernel”?

I've heard the expression "Gaussian kernel" in several contexts (e.g. in the kernel trick used in SVM). A Gaussian kernel usually refers to a Gaussian function (that is, a function similar to the ...
0
votes
0answers
17 views

(Deep) feature engineering for lambda terms (mathematical expressions, higher order logic formulas) - is such thing?

Automated theorem proving with (deep) reinforcement learning (DRL) approach is hot topic in current AI research when domains of games are becoming saturated and completed research topcis. E.g. there ...
1
vote
1answer
72 views

Is it okay to think of any dataset in artificial intelligence as a mathematical set?

A dataset is a collection of data points. It is known that the data points in the dataset can repeat. And the repetition does matter for building AI models. So, why does the word dataset contain the ...
1
vote
1answer
77 views

Given an axis-angle rotation vector, how can I find the unit rotation axis and angle?

I have a robotics assignment, which I am unable to solve. Given the axis-angle rotation vector $\Theta = (2, 2, 0)$, how can I calculate the unit vector of the rotation axis $k$ and the angle $\theta$?...
2
votes
2answers
85 views

Can a computer make a proof by induction?

Can a computer solve the following problem, i.e. make a proof by induction? And why? Prove by induction that $$\sum_{k=1}^nk^3=\left(\frac{n(n+1)}{2}\right)^2, \, \, \, \forall n\in\mathbb N .$$ I'm ...
-1
votes
1answer
30 views

how to go from mathematical problem to neural network (and back)?

I am a little confused on how, you can find online papers that describe complex Machine Learning formulas in a mathematical/probabilistic way, and, in the other hands, easy tutorials that teach you ...
0
votes
1answer
60 views

Why can a neural network use more than one activation function?

From trying to understand neural networks better, I've come upon a tentative notion that an activation function aims to build a function it's approximating via linear combinations with biases and ...
3
votes
1answer
158 views

How is the state-value function expressed as a product of sums?

The state-value function for a given policy $\pi$ is given by $$\begin{align} V^{\pi}(s) &=E_{\pi}\left\{r_{t+1}+\gamma r_{t+2}+\gamma^{2} r_{t+3}+\cdots \mid s_{t}=s\right\} \\ &=E_{\pi}\...
1
vote
1answer
24 views

How is the variational lower bound for hard attention derived in Show, Attend and Tell

How is the jump from line 1 to line 2 done in equation 10 of Show, Attend and Tell? While we're at it, another thing that might be muddying the waters for me is that I'm not clear on what the sum is ...
1
vote
1answer
59 views

Explanation of this L2 minimization equation

I am trying to understand the last two lines of this math notation. How Var and double summation of Cov came to the equation. The first two lines I understood something like $(a-b)^2 = a^2 -2ab +b^2$.
-2
votes
1answer
50 views

Assumptions of a Linear Regression [closed]

I was going through the concept of Linear Regression and ran into the concept of deciding whether a Linear Regression Model is the best fit for your data by 5 assumptions: Linearity Homoscedasticity ...
0
votes
0answers
22 views

Do $V_\theta$ and $V_s$ represent partial or total derivatives in the paper “Learning Continuous Control Policies by Stochastic Value Gradients”?

I was reading up on the Stochastic Value Gradients paper by Heess et al. In the paper, they describe a recursive process to calculate path-wise derivatives via equations (3) and (4), at the bottom of ...
0
votes
0answers
17 views

How do we get from entropy to KL divergence in this paper?

I'm reading through Regularizing Neural Networks By Penalizing Confident Output Distributions and I'm stuck on the equation in section 3.2. It's not clear to me at all that the self-entropy of the ...
0
votes
0answers
42 views

What are the math theorems regarding the Multilayer Perceptron?

I've come across a theorem "Convergence theorem Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish) ...
5
votes
1answer
222 views

Defining formula for fuzzy equation

I'm learning fuzzy logic and more or less understand the basic concept, but i'm having a hard time understanding how to apply it to a method. I tried browsing online for explanation on how to use it, ...
3
votes
2answers
52 views

Machine mathematical reasoning by clever substitutions, How to do with AI

I have three equations that relates five variables {a, b, c, r, s} with a sum and two ratios. Eq. 1: a = b + c; Eq. 2: s = b / a; Eq. 3: r = b / c. Given two values for any of the five variables I ...
1
vote
1answer
52 views

Is there a full and precise formulation of Theorem 1 in the Integrated Gradients paper?

Theorem 1 (page 5) in the paper about Integrated Gradients states that Integrated gradients is the unique path method that is symmetry-preserving. What I miss is A precise formulation of the ...
1
vote
0answers
32 views

Could the inputs of the mean squared-error loss function be transformed to allow larger learning rates?

In the context of a neural network $\hat{y} = f_\theta(\mathbf{x})$ with parameters $\theta$ that is trained to perform regression such that the prediction $\hat{\mathbf{y}} = [\hat{y}_1,\hat{y}_2,...,...
0
votes
1answer
24 views

What's the difference between a 1d tensor and a 2d tensor with 1 dimension?

I'm doing a TensorFlow tutorial, where they convert an array of the numbers [1,2,3] to a tensor like this: ...
1
vote
0answers
17 views

Clonal operator in Immune Clonal Strategy

I was reading about Immune Clonal Strategy, specifically about Monoclonal operator from Immunity clonal strategies, and it goes as follows: Here $a_i $ is a point and $a_i = \{ x_1, x_2, \cdots, x_m \...
0
votes
0answers
12 views

Is there an arrow missing in the derivation of front-door adjustment formula from do-calculus?

Here is Judea Pearl's derivation of the front-door adjustment formula: Is there an arrow from Genotype to Cancer missing in the second diagram at the right? just like this?
0
votes
0answers
22 views

Meaning of grad_outputs in torch.autograd.grad for complex input and output

Let's say we have a mathematical expression, $$ \mathbf{y} = \mathbf{Ax}, $$ where $\mathbf{y}$ and $\mathbf{x}$ are a vector, and $\mathbf{A}$ is a matrix. Let's say the vector $\mathbf{y}$ is used ...
1
vote
1answer
279 views

In a single neuron output layer should the output be a scalar?

Given a neural network with 3 inputs, 4 hidden layers, and 1 output, should the output neuron be a vector or a scalar? I thought that at the end of the summation only one number between 0 and 1 would ...
0
votes
1answer
42 views

Is the policy gradient expression in Fundamentals of Deep Learning wrong?

I don't understand the policy gradient as explained in Chapter-9 (Deep Reinforcement Learning) of the book Fundamentals of deep learning. Here is the whole paragraph: Policy Learning via Policy ...
-2
votes
1answer
52 views

Why the optimal Bellman operator of a Q-function can be approximated by a single point

I am currently studying reinforcement learning, especially DQN. In DQN, learning proceeds in such a way as to minimize the norm (least-squares, Huber, etc.) of the optimal Bellman equation and the ...
14
votes
2answers
657 views

Is there any scientific/mathematical argument that prevents deep learning from ever producing strong AI?

I read Judea Pearl's The Book of Why, in which he mentions that deep learning is just a glorified curve fitting technology, and will not be able to produce human-like intelligence. From his book ...
17
votes
3answers
1k views

What are the mathematical prerequisites for an AI researcher?

What are the mathematical prerequisites for understanding the core part of various algorithms involved in artificial intelligence and developing one's own algorithms? Please, refer some specific books....
3
votes
2answers
686 views

What does the argmax of the expectation of the log likelihood mean?

What does the following equation mean? What does each part of the formula represent or mean? $$\theta^* = \underset {\theta}{\arg \max} \Bbb E_{x \sim p_{data}} \log {p_{model}(x|\theta) }$$
6
votes
1answer
464 views

Why is the denominator ignored in the Bayes' rule?

The naïve Bayes' generative algorithm is often represented by the following formula: $$\text{argmax}_{y} p(y|x) = \text{argmax}_y \frac{p(x|y)p(y)}{p(x)} \approx \text{argmax}_y p(x|y)p(y)$$ Why do we ...
7
votes
3answers
918 views

Can we get the inverse of the function that a neural network represents?

I was wondering if it's possible to get the inverse of a neural network. If we view a NN as a function, can we obtain its inverse? I tried to build a simple MNIST architecture, with the input of (784,...
0
votes
1answer
61 views

Is it possible to know the distance objects are from camera based on only knowing one object's height?

I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
3
votes
4answers
507 views

What is the fundamental difference between an ML model and a function?

A model can be roughly defined as any design that is able to solve an ML task. Examples of models are the neural network, decision tree, Markov network, etc. A function can be defined as a set of ...
1
vote
2answers
198 views

Formal proof that every purely reactive agent has behaviorally equivalent standard agent

It kind of makes sense intuitively but I'm not sure about a formal proof. I'll start with briefly listing definitions from Intro to Multiagent systems, Wooldridge, 2002 and then give you my reasoning ...
4
votes
1answer
82 views

Can we use ML to do anything else other than predicting (in the case of mathematical problems)?

(The math problem here just serves as an example, my question is on this type of problems in general). Given two Schur polynomials, $s_\mu$, $s_\nu$, we know that we can decompose their product into a ...
9
votes
3answers
4k views

What are the mathematical prerequisites to be able to study artificial general intelligence?

What are the mathematical prerequisites to be able to study artificial general intelligence (AGI) or strong AI?
3
votes
1answer
75 views

How do you calculate KL divergence on a three-dimensional space for a Variational Autoencoder?

I'm trying to implement a variational auto-encoder (as seen in Section 3.1 here: https://arxiv.org/pdf/2004.06271.pdf). It differs from a traditional VAE because it encodes its input images to three-...
0
votes
1answer
54 views

Do the rows of the design matrix refer to the observations or predictors?

I attempt to understand the formulation of dictionary learning for this paper: Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution Multimodal Task-Driven ...
3
votes
0answers
100 views

What is the meaning of the words 'bias' and 'variance' in RL?

In reinforcement learning approaches, like temporal-difference (TD) learning or Monte Carlo methods, two of the metrics used to measure their performance are the bias and the variance. What do these ...
13
votes
3answers
2k views

What sort of mathematical problems are there in AI that people are working on?

I recently got a 18-month postdoc position in a math department. It's a position with relative light teaching duty and a lot of freedom about what type of research that I want to do. Previously I was ...
4
votes
2answers
164 views

Mathematical foundations of the ability to learn

I am an undergraduate student in applied mathematics with an interest in artificial intelligence. I am currently exploring topics where I could do research. Coming from a mathematical background I am ...
1
vote
1answer
71 views

How to compute the gradient of the cross-entropy loss function with respect to the parameters with softmax activation function?

I've seen plenty of examples of people doing Sigmoid + MSE backpropagation implementations, yet I do not seem to understand how to implement backpropagation as stated in the title in the case of multi-...
0
votes
0answers
16 views

Is parameter sharing in AlBERT akin to repeated application of same function on input?

I read the AlBERT and saw that its architecture used "Parameter Sharing" among layers of the encoder. They mentioned that this was done to save model space, make fewer training parameters ...
9
votes
1answer
228 views

How does the forget layer of an LSTM work?

Can someone explain the mathematical intuition behind the forget layer of an LSTM? So as far as I understand it, the cell state is essentially long term memory embedding (correct me if I'm wrong), ...
2
votes
3answers
120 views

What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?

I have looked at the documentation for the NEAT Python API found here, where it's written The error for each genome is $1-\sum_i(e_i-a_i)^2$ I have not yet learned calculus, so I can't understand ...
2
votes
1answer
62 views

What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?

I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
4
votes
1answer
118 views

Why is my derivation of the back-propagation equations inconsistent with Andrew Ng's slides from Coursera?

I am using the cross-entropy cost function to calculate its derivatives using different variables $Z, W$ and $b$ at different instances. Please refer image below for calculation. As per my knowledge, ...
1
vote
1answer
293 views

Understanding the derivation of the first-order model-agnostic meta-learning

According to the authors of this paper, to improve the performance, they decided to drop backward pass and using a first-order approximation I found a blog which discussed how to derive the math ...
1
vote
0answers
33 views

Are monotonically increasing functions easier to learn?

A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant. Logically this seems like ...