Questions tagged [math]
For questions about mathematics related to artificial intelligence.
249
questions
0
votes
0
answers
13
views
DDPM - why after adding gaussian noise to image, we assume that new image is from normal distribution?
I have a question about forward process in DDPM. It is described as we sample our image from some distribution: $x_0\sim{q(x)}$ then in each time stamp $T$ we are applying gaussian noise $\epsilon\sim\...
- 1
0
votes
0
answers
17
views
What kind of learning architecture can I use to solve a set of nonlinear thermodynamic equations?
I am in the business of solving equations based in thermodynamics. Technically speaking, I am trying to solve for the binodals for a ternary system. The binodal is a curve in a phase diagram.
...
- 101
0
votes
0
answers
65
views
claude 2 doesn't know basic math? [duplicate]
Sometimes, when I see answers like this from large language models, it makes me feel disgusted:
Me: Does Voyager 1 have enough velocity to escape the solar system without using Jupiter's gravity ...
- 1
0
votes
0
answers
14
views
Is it possible tofind a subset of F_p for large p such that no solutions exist
I'm aware that neural networks are probably not designed to do that, however asking hypothetically:
I have a question regarding the possibility of identifying a subset of $\mathbb{F}_p$ in which a ...
- 101
2
votes
1
answer
173
views
Why can AI not understand numbers- even basic ones? [duplicate]
I asked Bard to "Give me a challenging mathematical riddle to which the answer is 23"
It answered: "Here is a challenging mathematical riddle to which the answer is 23:
I am the sum of ...
- 344
-2
votes
2
answers
60
views
Size of a neural network [closed]
For any type of ANN, is there a common formula which can be used to calculate size of a neural network?
2
votes
3
answers
984
views
What will happen if to train an LLM on mathematical exersises?
What will happen if to train an LLM on taking integrals and solving equations? The process of mathematical education can be absolutely automated by a computer algebra system because the verification ...
- 309
0
votes
0
answers
24
views
Understanding the decreasing influence of text embedding in Text-to-Image diffusion models: A Mathematical perspective
I've been reading the paper titled eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers by Yogesh Balaji et. al. Consider the following excerpt from the abstract of the paper
...
- 3,612
-1
votes
1
answer
33
views
Sum of gaussian distribution [closed]
I assume the sum of the Gaussian distribution is 1. However, the actual calculation in JavaScript yields a value greater than 1 in JavaScript.
Is this code wrong?
gaussian_distribution.html]
...
0
votes
1
answer
67
views
Can neural-networks solve a system of non-linear algebraic equation
I am currently doing research as a PhD student in theoretical physics. Currently we are calculating physical quantities described by coupled non-linear algebraic systems of equations. These equations ...
- 1
0
votes
0
answers
102
views
RetNet: Why Diagonalize A Square Matrix?
The paragraph before eq. (3) in The RetNet paper says to diagonalize the $R^{d{\times}d}$ matrix $A$. I am confused as to why one would not just use a vector?
One of the arguments that I have heard on ...
1
vote
0
answers
32
views
Can a Fully Connected Neural Network represent all Neural Networks of smaller size?
A fully connected Neural Network architecture can be characterized by a vector $\mathbf a = (a_0,a_1,\ldots,a_L)\in\mathbb N^{L+1}$ and an activation function $\sigma :\mathbb R\to\mathbb R$. In this ...
1
vote
1
answer
28
views
How do we get from types of activation functions to fitting lines to our data?
I'm completely new to AI and admittedly have never been good at math (also please excuse me if I use the wrong terminology). Despite this, I'm trying wrap my head around activation functions and how ...
- 181
0
votes
2
answers
69
views
How is speech recognition software able to distinguish between different speakers and yet still understand them all?
Some background: I'm an EE major and data science minor, so I have a basic understanding of machine learning - I had a one semester course on it where we covered some of the most commonly used ...
-1
votes
1
answer
77
views
Machine Learning Math Requirements [duplicate]
What area of math do i need for AI and ML? and some recommendations for YouTube math's playlist
I've just started to learn to code in python and i want to go to the path of AI/ML
0
votes
1
answer
90
views
How are the intuitions and mathematics of attention mechanisms related to those of PageRank?
Excuse me if you find this question too vague and not fitting to this forum and feel free to close it. The overall goal of my question is to get a better intuition of the attention concept and ...
1
vote
1
answer
61
views
Is the problem of Language Modelling a Well-Posed Learning Problem?
Hadamard defines (Well-posed problem (Wikipedia)) a well-posed problem as one for which:
a solution exists,
the solution is unique,
the solution depends continuously on the data (e.g. it is stable)
...
1
vote
1
answer
489
views
What operation is ggml_mul_mat performing? (K×Q in LLaMA)
I’m reading the source code of alpaca.cpp in an attempt to understand how a large language model works. (I have a strong programming background, but almost no math, ...
- 111
1
vote
0
answers
26
views
Is Machine learning a good approach for this geometric challenge?
For months I have been working on a conventional algorithm that positions circles with different sized radii and either positive or negative charge within an outer radius in such a way that the charge ...
- 31
2
votes
0
answers
122
views
Best calculus books for Deep Learning
Recommend some calculus books for Deep Learning and neural networks. I know what is integration, differentiation, derivates, limits on a based level. I would like to understand on deep level the ...
- 21
1
vote
1
answer
85
views
Is there a mathematical proof that a binary neural network can approximate any function with arbitrary accuracy?
Since the Universal approximation theorem shows that standard multilayer feedforward networks with as few as a single hidden layer, sufficient hidden units, and arbitrary bounded and nonconstant ...
- 33
1
vote
0
answers
45
views
Computation required for GPT model to choose likely word from n-options where n < total vocabulary size
Let’s imagine two different use cases for a LLM/GPT-3.
Predicting the next most likely word in a sequence using all ~50k words in its dictionary (i.e. the standard method of prompting a LLM)
Checking ...
- 11
0
votes
0
answers
31
views
Fast sub-accurate addition and multiplication
Matrix multiplication lies in the foundation of AI, and efficient algorithms to addition and multiplication lead to massive improvement in practically every aspect of AI lifecycle. (At least that's ...
- 101
2
votes
1
answer
571
views
What exactly is the future of mathematical analysis that can be dealt with AI (especially in the automated theorem proving)?
As a student of numerical analysis, I can see how mathematical analysis involved in making a language program specifically in the convergence analysis of an approximation method. But, while chatting ...
- 123
0
votes
1
answer
84
views
How much pure math should I know for deep learning? [duplicate]
Recently I've been wondering about the necessary amount of math that a deep learning scientist really needs to know. From what I could gather around the internet there are 3 big areas(calculus 1,2 and,...
- 1
0
votes
1
answer
34
views
Is $i$ indexing the first or second dimension in $\mathbf{x}_i$, where $\mathbf{x} \in \mathbb{R}^{n\times d}$?
I was reading the following notes on the math behind transformers and was confused about what $\mathbf{x}_i$ is? If $\mathbf{x} \in \mathbb{R}^{n\times d}$, then is the $i$ indexing the $n$ or the the ...
- 23
1
vote
1
answer
41
views
What does this bracket notation $\langle\phi(x),v\rangle$ mean?
I found it at the bottom of page 2 of the paper Intriguing properties of neural networks (2014), in the form of
$$\underset{x\in\mathcal{I}}{\mathrm{arg\,max}}\langle\phi(x),v\rangle$$
- 121
27
votes
4
answers
33k
views
Why is ChatGPT bad at math?
As opposed to How does ChatGPT know math?, I've been seeing some things floating around the Twitterverse about how ChatGPT can actually be very bad at math. For instance, I asked it "If it takes ...
- 2,885
24
votes
4
answers
10k
views
How does ChatGPT know math?
ChatGPT is a language model. As far as I know and If I'm not wrong, it gets text as tokens and word embeddings. So, how can it do math? For example, I asked:
ME: Which one is bigger 5 or 9.
ChatGPT: ...
- 554
2
votes
2
answers
132
views
Where does the proximal policy optimization objective's ratio term come from?
I will use the notation used in the proximal policy optimization paper.
What approximation is needed to arrive at the surrogate objective (equation (6) above) with the ratio $r_t(\theta)$?
Put ...
- 203
1
vote
1
answer
269
views
Interpretation of the Dynamic Time Warping (DTW) graph
How can I interpret ate the DTW graph.
I understood the algorithm behind DTW, but I struggle to interpret ate the graph.
When I compute the DTW for a signal that is the same signal but shifted in time,...
- 51
2
votes
1
answer
757
views
Are my proofs that the Bellman operators are contractions correct?
Introduction
I'm studying Reinforcement Learning, and in order to increase my understanding I've been challenging myself by trying to write proofs that show that the right hand side of the Bellman ...
- 53
1
vote
0
answers
52
views
Can the Jacobian of a Neural Network be Full Column Rank?
Let $\mathcal{X}$ be the input data space and $\mathcal{Y}$ be the output data space.
$f: \mathcal{X} \to \mathcal{Y}$ is a function represented by some Neural Network.
Is it possible to to check if ...
- 21
0
votes
0
answers
102
views
The Small Set Expansion Hypothesis, this problem was solved or is open problem yet?
I found this problem by article called "Ten Lectures and Forty-Two Open Problems in the Mathematics of Data Science" published at 2016. I`m looking for an open problem at Data Science or/and ...
2
votes
1
answer
264
views
Clarification on GANs for text generation
A GAN-like architecture for text generation is proposed in 'Generative Adversarial Networks for Text Generation'.
The setup is the following:
The generator of the GAN is proposed to be a recurrent ...
- 133
0
votes
0
answers
52
views
Basic question about gradient for nominal regression
Say that we want to binary-classify images using a sigmoid function with the entropy-loss function. This algorithm is quite slow.
The sigmoid function is:
I find that this could be traced to the $L(y,...
- 79
0
votes
0
answers
71
views
Why Is There The Term 1/m In Backpropagation
In backpropagation the gradients are used to update the weights using the formula
$$w = w - \alpha \frac{dL}{dw}$$
and the loss gradient w.r.t. weights is
$$\frac{dL}{dw} = \frac{dL}{dz} \frac{dz}{dw} ...
- 21
1
vote
1
answer
211
views
Mathematics books for reinforcement learning
This question is not about the math prerequisites of reinforcement learning, but about the textbooks of mathematics that are enough to understand the literature on reinforcement learning.
What are the ...
- 3,612
2
votes
1
answer
599
views
What is a "continuous vector"?
I have seen the concept of a "continuous vector" described in the context of embeddings. For instance, this answer to a question on embeddings in the context of deep learning. I obviously ...
- 547
0
votes
2
answers
160
views
Is this the correct way to backpropagate a Neural Network?
I am writing a Neural Network frorm scratch. Below is what I have right now, based off of the math that I think I understand.
...
user55176
0
votes
1
answer
55
views
What do we mean by the notation $\mathbf{x}_{p} \in \mathbb{R}^{N \times\left(P^{2} \cdot C\right)}$?
I was going through this VIT paper, what will it look like in torch , if we are trying to write this expression.
- 101
1
vote
1
answer
62
views
How does Bishop derive $\ln p\left(\mathbf{x} \mid \mu, \sigma^{2}\right)$, when $p$ is a Gaussian?
I am now reading the Bishop Machine Learning Book and going through every single equation.
We know that in the case of a single real-valued variable $x$, the Gaussian distribution is defined by
$$\...
- 13
1
vote
0
answers
42
views
Does all GAN's in literature need to satisfy the properties of objective function of initial GAN? [closed]
Consider the following value function of the initial GAN
$V(D, G) = \mathbb{E}_{x \sim p_{data(x)}} [\log D(x)] + \mathbb{E}_{z \sim p_z(z)} [1- \log D(G(z))]$
The min-max game on the value function: $...
- 3,612
0
votes
1
answer
24
views
What is the right way to find the alphas in this equation?
In the Grad-CAM++ paper the following equation (7) is posed (written here without the relu function):
$$
Y^c =
\sum_k \Bigl( \Bigl\{ \sum_{a,b} \alpha_{ab}^{kc}
\cdot \frac{\partial Y^c}{\...
- 141
0
votes
1
answer
70
views
Usability of power series in AI analysis
In mathematics, power series is given by
$$f(x) = \sum\limits_{n=0}^{\infty} c_n (x-a)^n$$
where $c_n , a \in \mathbb{R}$
Although most of the courses in academics cover moment generating functions in ...
- 3,612
0
votes
1
answer
83
views
What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?
I have a question about the Grad-CAM++ paper. I do not understand how the following equation (10) for the alphas is obtained:
$$
\alpha_{ij}^{kc} =
\frac{\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}}
{...
- 141
1
vote
1
answer
51
views
Should I need to interpret the word "metric" in "performance metric" rigorously?
Consider the following abstract from the research paper titled A Note on the Inception Score for instance
Deep generative models are powerful tools that have produced
impressive results in recent ...
- 3,612
3
votes
2
answers
3k
views
What does it mean by "gradient flow" in the context of neural networks?
Several research papers and textbooks (e.g. this) contain the phrase "gradient flow" in the context of neural networks.
I am confused about whether it has any rigorous and formal way of ...
- 3,612
0
votes
1
answer
44
views
What does it mean by "dynamics of a sequence" mathematically?
Consider the following paragraph from the topic named sequential models from the textbook titled Dive into Deep Learning
Both cases raise the obvious question of how to generate training
data. One ...
- 3,612
1
vote
1
answer
264
views
What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$?
Here is the formula of state-value function in Reinforcement Learning.
What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$? Why use square brackets? Why use $\mid$?
Why do ...
- 293