# Questions tagged [math]

For questions about mathematics related to artificial intelligence.

190 questions
Filter by
Sorted by
Tagged with
20 views

### Why integrated gradients don't explain samples correctly?

I have a linear tabular dataset made of floats. The dataset follows a simple rule like: ...
24 views

### How is the variational lower bound for hard attention derived in Show, Attend and Tell

How is the jump from line 1 to line 2 done in equation 10 of Show, Attend and Tell? While we're at it, another thing that might be muddying the waters for me is that I'm not clear on what the sum is ...
48 views

### Assumptions of a Linear Regression [closed]

I was going through the concept of Linear Regression and ran into the concept of deciding whether a Linear Regression Model is the best fit for your data by 5 assumptions: Linearity Homoscedasticity ...
15 views

### How do we get from entropy to KL divergence in this paper?

I'm reading through Regularizing Neural Networks By Penalizing Confident Output Distributions and I'm stuck on the equation in section 3.2. It's not clear to me at all that the self-entropy of the ...
21 views

### Do $V_\theta$ and $V_s$ represent partial or total derivatives in the paper “Learning Continuous Control Policies by Stochastic Value Gradients”?

I was reading up on the Stochastic Value Gradients paper by Heess et al. In the paper, they describe a recursive process to calculate path-wise derivatives via equations (3) and (4), at the bottom of ...
31 views

In the context of a neural network $\hat{y} = f_\theta(\mathbf{x})$ with parameters $\theta$ that is trained to perform regression such that the prediction $\hat{\mathbf{y}} = [\hat{y}_1,\hat{y}_2,...,... 0answers 40 views ### What are the math theorems regarding the Multilayer Perceptron? I've come across a theorem "Convergence theorem Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish) ... 1answer 22 views ### What's the difference between a 1d tensor and a 2d tensor with 1 dimension? I'm doing a TensorFlow tutorial, where they convert an array of the numbers [1,2,3] to a tensor like this: ... 1answer 47 views ### Why can a neural network use more than one activation function? From trying to understand neural networks better, I've come upon a tentative notion that an activation function aims to build a function it's approximating via linear combinations with biases and ... 0answers 12 views ### Is there an arrow missing in the derivation of front-door adjustment formula from do-calculus? Here is Judea Pearl's derivation of the front-door adjustment formula: Is there an arrow from Genotype to Cancer missing in the second diagram at the right? just like this? 0answers 17 views ### Clonal operator in Immune Clonal Strategy I was reading about Immune Clonal Strategy, specifically about Monoclonal operator from Immunity clonal strategies, and it goes as follows: Here$a_i $is a point and$a_i = \{ x_1, x_2, \cdots, x_m \...
13 views

Let's say we have a mathematical expression, $$\mathbf{y} = \mathbf{Ax},$$ where $\mathbf{y}$ and $\mathbf{x}$ are a vector, and $\mathbf{A}$ is a matrix. Let's say the vector $\mathbf{y}$ is used ...
38 views

### Is the policy gradient expression in Fundamentals of Deep Learning wrong?

I don't understand the policy gradient as explained in Chapter-9 (Deep Reinforcement Learning) of the book Fundamentals of deep learning. Here is the whole paragraph: Policy Learning via Policy ...
41 views

### Why the optimal Bellman operator of a Q-function can be approximated by a single point

I am currently studying reinforcement learning, especially DQN. In DQN, learning proceeds in such a way as to minimize the norm (least-squares, Huber, etc.) of the optimal Bellman equation and the ...
48 views

### Is there a full and precise formulation of Theorem 1 in the Integrated Gradients paper?

Theorem 1 (page 5) in the paper about Integrated Gradients states that Integrated gradients is the unique path method that is symmetry-preserving. What I miss is A precise formulation of the ...
74 views

### How do you calculate KL divergence on a three-dimensional space for a Variational Autoencoder?

I'm trying to implement a variational auto-encoder (as seen in Section 3.1 here: https://arxiv.org/pdf/2004.06271.pdf). It differs from a traditional VAE because it encodes its input images to three-...
67 views

### How to compute the gradient of the cross-entropy loss function with respect to the parameters with softmax activation function?

I've seen plenty of examples of people doing Sigmoid + MSE backpropagation implementations, yet I do not seem to understand how to implement backpropagation as stated in the title in the case of multi-...
31 views

### Are monotonically increasing functions easier to learn?

A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant. Logically this seems like ...
58 views

### What is $\nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?

I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
16 views

### Is parameter sharing in AlBERT akin to repeated application of same function on input?

I read the AlBERT and saw that its architecture used "Parameter Sharing" among layers of the encoder. They mentioned that this was done to save model space, make fewer training parameters ...
491 views

### What is the fundamental difference between an ML model and a function?

A model can be roughly defined as any design that is able to solve an ML task. Examples of models are the neural network, decision tree, Markov network, etc. A function can be defined as a set of ...
41 views

### Can I solve the below functional equation using neural networks?

I recently watched this video, in which he solves the equation $$f(x)+f\left(\frac{1}{1-x}\right) = x$$ The answer is $$f(x) = \frac{x^3-x+1}{2x(x-1)}$$ I tried to solve this functional equation using ...
82 views

### Can we use ML to do anything else other than predicting (in the case of mathematical problems)?

(The math problem here just serves as an example, my question is on this type of problems in general). Given two Schur polynomials, $s_\mu$, $s_\nu$, we know that we can decompose their product into a ...
59 views

### Explanation of this L2 minimization equation

I am trying to understand the last two lines of this math notation. How Var and double summation of Cov came to the equation. The first two lines I understood something like $(a-b)^2 = a^2 -2ab +b^2$.
55 views

### Is it okay to think of any dataset in artificial intelligence as a mathematical set?

A dataset is a collection of data points. It is known that the data points in the dataset can repeat. And the repetition does matter for building AI models. So, why does the word dataset contain the ...
33 views

### Can any area of math come into play in Machine Learning Research?

As I read online following areas in mathematics comes into play in ML research Linear Algebra Calculus Differential Equations Probability Statistics Discrete Mathematics Optimization Analytic ...
32 views

### How do I derive the gradient of the log-likelihood of an RBM?

In a Restricted Boltzmann Machine (RBM), the likelihood function is: $$p(\mathbf{v};\mathbf{\theta}) = \frac{1}{Z} \sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}$$ Where $E$ is the ...
10 views

45 views

### How to understand mapping function of kernel?

For a kernel function, we have two conditions one is that it should be symmetric which is easy to understand intuitively because dot products are symmetric as well and our kernel should also follow ...
157 views

### What is an auto-associator?

What is an auto-associator, and how does it work? How can we design an auto-associator for a given pattern? I couldn't find a clear explanation for this anywhere on the internet. Here's an example of ...
44 views

### Simplifying Log Loss

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
154 views

### Can deep learning be used to help mathematical research?

I am currently learning about deep learning and artificial intelligence and exploring his possibilities, and, as a mathematician at heart, I am inquisitive about how it can be used to solve problems ...