# Questions tagged [math]

For questions about mathematics related to artificial intelligence.

207 questions
Filter by
Sorted by
Tagged with
76 views

### How do you calculate KL divergence on a three-dimensional space for a Variational Autoencoder?

I'm trying to implement a variational auto-encoder (as seen in Section 3.1 here: https://arxiv.org/pdf/2004.06271.pdf). It differs from a traditional VAE because it encodes its input images to three-...
55 views

### Do the rows of the design matrix refer to the observations or predictors?

I attempt to understand the formulation of dictionary learning for this paper: Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution Multimodal Task-Driven ...
103 views

### What is the meaning of the words 'bias' and 'variance' in RL?

In reinforcement learning approaches, like temporal-difference (TD) learning or Monte Carlo methods, two of the metrics used to measure their performance are the bias and the variance. What do these ...
2k views

### What sort of mathematical problems are there in AI that people are working on?

I recently got a 18-month postdoc position in a math department. It's a position with relative light teaching duty and a lot of freedom about what type of research that I want to do. Previously I was ...
167 views

### Mathematical foundations of the ability to learn

I am an undergraduate student in applied mathematics with an interest in artificial intelligence. I am currently exploring topics where I could do research. Coming from a mathematical background I am ...
71 views

### How to compute the gradient of the cross-entropy loss function with respect to the parameters with softmax activation function?

I've seen plenty of examples of people doing Sigmoid + MSE backpropagation implementations, yet I do not seem to understand how to implement backpropagation as stated in the title in the case of multi-...
16 views

### Is parameter sharing in AlBERT akin to repeated application of same function on input?

I read the AlBERT and saw that its architecture used "Parameter Sharing" among layers of the encoder. They mentioned that this was done to save model space, make fewer training parameters ...
235 views

### How does the forget layer of an LSTM work?

Can someone explain the mathematical intuition behind the forget layer of an LSTM? So as far as I understand it, the cell state is essentially long term memory embedding (correct me if I'm wrong), ...
121 views

### What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?

I have looked at the documentation for the NEAT Python API found here, where it's written The error for each genome is $1-\sum_i(e_i-a_i)^2$ I have not yet learned calculus, so I can't understand ...
62 views

### What is $\nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?

I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
120 views

### Why is my derivation of the back-propagation equations inconsistent with Andrew Ng's slides from Coursera?

I am using the cross-entropy cost function to calculate its derivatives using different variables $Z, W$ and $b$ at different instances. Please refer image below for calculation. As per my knowledge, ...
331 views

### Understanding the derivation of the first-order model-agnostic meta-learning

According to the authors of this paper, to improve the performance, they decided to drop backward pass and using a first-order approximation I found a blog which discussed how to derive the math ...
33 views

### Are monotonically increasing functions easier to learn?

A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant. Logically this seems like ...
94 views

### Interpretation of inverse matrix in mean calculation in Gaussian Process

The formula for mean prediction using Gaussian Process is $k(x_*, x)k(x, x)^{-1}y$, where $k$ is the covariance function. See e.g. equation 2.23 (in chapter 2) from Gaussian Processes for Machine ...
114 views

### Why is the equation $r(s', a, s') =\sum_{r \in \mathcal{R}} r \frac{p\left(s^{\prime}, r \mid s, a\right)}{p\left(s^{\prime} \mid s, a\right)}$true?

I am referring to eq. 3.6 (page 49) based on Sutton's online book and can be found in an image below. I could not make sense of the final derivation of the equation $r(s, a, s')$. My question is ...
336 views

### Which matrix represents the similarity between words when using SVD?

Two words can be similar if they co-occur "a lot" together. They can also be similar if they have similar vectors. This similarity can be captured using cosine similarity. Let $A$ be a $n \times n$ ...
100 views

### Why is the change in cost wrt bias in neural network equal to error in the neuron?

While reading the book on neural networks by Michael Nielson, I had a problem understanding equation (BP3), which is $$\frac{\partial C}{\partial b_{j}^{l}}=\delta_{j}^{l} \tag{BP3}\label{BP3},$$ ...
43 views

### Can I solve the below functional equation using neural networks?

I recently watched this video, in which he solves the equation $$f(x)+f\left(\frac{1}{1-x}\right) = x$$ The answer is $$f(x) = \frac{x^3-x+1}{2x(x-1)}$$ I tried to solve this functional equation using ...
131 views

### Does Gödel's second incompleteness theorem put a limitation on artificial intelligence systems?

According to Brian Cantwell Smith no calculation without representation Therefore, computers depend on models. So, we can say that AI is limited internally by the model and externally by the ...
4k views

### Can neural networks be used to prove conjectures?

Imagine I have a list (in a computer-readable form) of all problems (or statements) and proofs that math relies on. Could I train a neural network in such a way that, for example, I enter a problem ...
72 views

### What does $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ mean in the article Hindsight Experience Replay, section 2.1?

Taken from section 2.1 in the article: We consider the standard reinforcement learning formalism consisting of an agent interacting with an environment. To simplify the exposition we assume that the ...
2k views

### Why does the “reward to go” trick in policy gradient methods work?

In the policy gradient method, there's a trick to reduce the variance of policy gradient. We use causality, and remove part of the sum over rewards so that only actions happened after the reward are ...
124 views

32 views

### How do I derive the gradient of the log-likelihood of an RBM?

In a Restricted Boltzmann Machine (RBM), the likelihood function is: $$p(\mathbf{v};\mathbf{\theta}) = \frac{1}{Z} \sum_{\mathbf{h}} e^{-E(\mathbf{v},\mathbf{h};\mathbf{\theta})}$$ Where $E$ is the ...
111 views

### The mathematics in the CBOW and Skip-Gram models

this is my first question on AI Stack Exchange. I am a mathematics student who is learning NLP so I have paid a high amount of attention on the mathematics used in the subject, but my interpretations ...
11 views