Questions tagged [math]

For questions about mathematics related to artificial intelligence.

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

DDPM - why after adding gaussian noise to image, we assume that new image is from normal distribution?

I have a question about forward process in DDPM. It is described as we sample our image from some distribution: $x_0\sim{q(x)}$ then in each time stamp $T$ we are applying gaussian noise $\epsilon\sim\...
cvzx's user avatar
  • 1
0 votes
0 answers
17 views

What kind of learning architecture can I use to solve a set of nonlinear thermodynamic equations?

I am in the business of solving equations based in thermodynamics. Technically speaking, I am trying to solve for the binodals for a ternary system. The binodal is a curve in a phase diagram. ...
bad_chemist's user avatar
0 votes
0 answers
65 views

claude 2 doesn't know basic math? [duplicate]

Sometimes, when I see answers like this from large language models, it makes me feel disgusted: Me: Does Voyager 1 have enough velocity to escape the solar system without using Jupiter's gravity ...
Mr Saw's user avatar
  • 1
0 votes
0 answers
14 views

Is it possible tofind a subset of F_p for large p such that no solutions exist

I'm aware that neural networks are probably not designed to do that, however asking hypothetically: I have a question regarding the possibility of identifying a subset of $\mathbb{F}_p$ in which a ...
laura's user avatar
  • 101
2 votes
1 answer
173 views

Why can AI not understand numbers- even basic ones? [duplicate]

I asked Bard to "Give me a challenging mathematical riddle to which the answer is 23" It answered: "Here is a challenging mathematical riddle to which the answer is 23: I am the sum of ...
ben svenssohn's user avatar
-2 votes
2 answers
60 views

Size of a neural network [closed]

For any type of ANN, is there a common formula which can be used to calculate size of a neural network?
Indika Alahapperuma's user avatar
2 votes
3 answers
984 views

What will happen if to train an LLM on mathematical exersises?

What will happen if to train an LLM on taking integrals and solving equations? The process of mathematical education can be absolutely automated by a computer algebra system because the verification ...
Anixx's user avatar
  • 309
0 votes
0 answers
24 views

Understanding the decreasing influence of text embedding in Text-to-Image diffusion models: A Mathematical perspective

I've been reading the paper titled eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers by Yogesh Balaji et. al. Consider the following excerpt from the abstract of the paper ...
hanugm's user avatar
  • 3,612
-1 votes
1 answer
33 views

Sum of gaussian distribution [closed]

I assume the sum of the Gaussian distribution is 1. However, the actual calculation in JavaScript yields a value greater than 1 in JavaScript. Is this code wrong? gaussian_distribution.html] ...
diffusion stable's user avatar
0 votes
1 answer
67 views

Can neural-networks solve a system of non-linear algebraic equation

I am currently doing research as a PhD student in theoretical physics. Currently we are calculating physical quantities described by coupled non-linear algebraic systems of equations. These equations ...
SaFeHe's user avatar
  • 1
0 votes
0 answers
102 views

RetNet: Why Diagonalize A Square Matrix?

The paragraph before eq. (3) in The RetNet paper says to diagonalize the $R^{d{\times}d}$ matrix $A$. I am confused as to why one would not just use a vector? One of the arguments that I have heard on ...
Umar Siregar's user avatar
1 vote
0 answers
32 views

Can a Fully Connected Neural Network represent all Neural Networks of smaller size?

A fully connected Neural Network architecture can be characterized by a vector $\mathbf a = (a_0,a_1,\ldots,a_L)\in\mathbb N^{L+1}$ and an activation function $\sigma :\mathbb R\to\mathbb R$. In this ...
Stratos supports the strike's user avatar
1 vote
1 answer
28 views

How do we get from types of activation functions to fitting lines to our data?

I'm completely new to AI and admittedly have never been good at math (also please excuse me if I use the wrong terminology). Despite this, I'm trying wrap my head around activation functions and how ...
Garrett's user avatar
  • 181
0 votes
2 answers
69 views

How is speech recognition software able to distinguish between different speakers and yet still understand them all?

Some background: I'm an EE major and data science minor, so I have a basic understanding of machine learning - I had a one semester course on it where we covered some of the most commonly used ...
Mikayla Eckel Cifrese's user avatar
-1 votes
1 answer
77 views

Machine Learning Math Requirements [duplicate]

What area of math do i need for AI and ML? and some recommendations for YouTube math's playlist I've just started to learn to code in python and i want to go to the path of AI/ML
Kenezzu Len's user avatar
0 votes
1 answer
90 views

How are the intuitions and mathematics of attention mechanisms related to those of PageRank?

Excuse me if you find this question too vague and not fitting to this forum and feel free to close it. The overall goal of my question is to get a better intuition of the attention concept and ...
Hans-Peter Stricker's user avatar
1 vote
1 answer
61 views

Is the problem of Language Modelling a Well-Posed Learning Problem?

Hadamard defines (Well-posed problem (Wikipedia)) a well-posed problem as one for which: a solution exists, the solution is unique, the solution depends continuously on the data (e.g. it is stable) ...
aren't eistert's user avatar
1 vote
1 answer
489 views

What operation is ggml_mul_mat performing? (K×Q in LLaMA)

I’m reading the source code of alpaca.cpp in an attempt to understand how a large language model works. (I have a strong programming background, but almost no math, ...
Wolfgang's user avatar
  • 111
1 vote
0 answers
26 views

Is Machine learning a good approach for this geometric challenge?

For months I have been working on a conventional algorithm that positions circles with different sized radii and either positive or negative charge within an outer radius in such a way that the charge ...
Vale's user avatar
  • 31
2 votes
0 answers
122 views

Best calculus books for Deep Learning

Recommend some calculus books for Deep Learning and neural networks. I know what is integration, differentiation, derivates, limits on a based level. I would like to understand on deep level the ...
Dan Il's user avatar
  • 21
1 vote
1 answer
85 views

Is there a mathematical proof that a binary neural network can approximate any function with arbitrary accuracy?

Since the Universal approximation theorem shows that standard multilayer feedforward networks with as few as a single hidden layer, sufficient hidden units, and arbitrary bounded and nonconstant ...
user68072's user avatar
1 vote
0 answers
45 views

Computation required for GPT model to choose likely word from n-options where n < total vocabulary size

Let’s imagine two different use cases for a LLM/GPT-3. Predicting the next most likely word in a sequence using all ~50k words in its dictionary (i.e. the standard method of prompting a LLM) Checking ...
Derek's user avatar
  • 11
0 votes
0 answers
31 views

Fast sub-accurate addition and multiplication

Matrix multiplication lies in the foundation of AI, and efficient algorithms to addition and multiplication lead to massive improvement in practically every aspect of AI lifecycle. (At least that's ...
DannyNiu's user avatar
  • 101
2 votes
1 answer
571 views

What exactly is the future of mathematical analysis that can be dealt with AI (especially in the automated theorem proving)?

As a student of numerical analysis, I can see how mathematical analysis involved in making a language program specifically in the convergence analysis of an approximation method. But, while chatting ...
Messi Lio's user avatar
  • 123
0 votes
1 answer
84 views

How much pure math should I know for deep learning? [duplicate]

Recently I've been wondering about the necessary amount of math that a deep learning scientist really needs to know. From what I could gather around the internet there are 3 big areas(calculus 1,2 and,...
Tom PL's user avatar
  • 1
0 votes
1 answer
34 views

Is $i$ indexing the first or second dimension in $\mathbf{x}_i$, where $\mathbf{x} \in \mathbb{R}^{n\times d}$?

I was reading the following notes on the math behind transformers and was confused about what $\mathbf{x}_i$ is? If $\mathbf{x} \in \mathbb{R}^{n\times d}$, then is the $i$ indexing the $n$ or the the ...
play's user avatar
  • 23
1 vote
1 answer
41 views

What does this bracket notation $\langle\phi(x),v\rangle$ mean?

I found it at the bottom of page 2 of the paper Intriguing properties of neural networks (2014), in the form of $$\underset{x\in\mathcal{I}}{\mathrm{arg\,max}}\langle\phi(x),v\rangle$$
Nils André's user avatar
27 votes
4 answers
33k views

Why is ChatGPT bad at math?

As opposed to How does ChatGPT know math?, I've been seeing some things floating around the Twitterverse about how ChatGPT can actually be very bad at math. For instance, I asked it "If it takes ...
Mithical's user avatar
  • 2,885
24 votes
4 answers
10k views

How does ChatGPT know math?

ChatGPT is a language model. As far as I know and If I'm not wrong, it gets text as tokens and word embeddings. So, how can it do math? For example, I asked: ME: Which one is bigger 5 or 9. ChatGPT: ...
Peyman's user avatar
  • 554
2 votes
2 answers
132 views

Where does the proximal policy optimization objective's ratio term come from?

I will use the notation used in the proximal policy optimization paper. What approximation is needed to arrive at the surrogate objective (equation (6) above) with the ratio $r_t(\theta)$? Put ...
user's user avatar
  • 203
1 vote
1 answer
269 views

Interpretation of the Dynamic Time Warping (DTW) graph

How can I interpret ate the DTW graph. I understood the algorithm behind DTW, but I struggle to interpret ate the graph. When I compute the DTW for a signal that is the same signal but shifted in time,...
Skobo Do's user avatar
2 votes
1 answer
757 views

Are my proofs that the Bellman operators are contractions correct?

Introduction I'm studying Reinforcement Learning, and in order to increase my understanding I've been challenging myself by trying to write proofs that show that the right hand side of the Bellman ...
cfml's user avatar
  • 53
1 vote
0 answers
52 views

Can the Jacobian of a Neural Network be Full Column Rank?

Let $\mathcal{X}$ be the input data space and $\mathcal{Y}$ be the output data space. $f: \mathcal{X} \to \mathcal{Y}$ is a function represented by some Neural Network. Is it possible to to check if ...
BJMG's user avatar
  • 21
0 votes
0 answers
102 views

The Small Set Expansion Hypothesis, this problem was solved or is open problem yet?

I found this problem by article called "Ten Lectures and Forty-Two Open Problems in the Mathematics of Data Science" published at 2016. I`m looking for an open problem at Data Science or/and ...
Leonardo Teramatsu's user avatar
2 votes
1 answer
264 views

Clarification on GANs for text generation

A GAN-like architecture for text generation is proposed in 'Generative Adversarial Networks for Text Generation'. The setup is the following: The generator of the GAN is proposed to be a recurrent ...
Ramiro Hum-Sah's user avatar
0 votes
0 answers
52 views

Basic question about gradient for nominal regression

Say that we want to binary-classify images using a sigmoid function with the entropy-loss function. This algorithm is quite slow. The sigmoid function is: I find that this could be traced to the $L(y,...
Mah Neh's user avatar
  • 79
0 votes
0 answers
71 views

Why Is There The Term 1/m In Backpropagation

In backpropagation the gradients are used to update the weights using the formula $$w = w - \alpha \frac{dL}{dw}$$ and the loss gradient w.r.t. weights is $$\frac{dL}{dw} = \frac{dL}{dz} \frac{dz}{dw} ...
rkuang25's user avatar
1 vote
1 answer
211 views

Mathematics books for reinforcement learning

This question is not about the math prerequisites of reinforcement learning, but about the textbooks of mathematics that are enough to understand the literature on reinforcement learning. What are the ...
hanugm's user avatar
  • 3,612
2 votes
1 answer
599 views

What is a "continuous vector"?

I have seen the concept of a "continuous vector" described in the context of embeddings. For instance, this answer to a question on embeddings in the context of deep learning. I obviously ...
The Pointer's user avatar
0 votes
2 answers
160 views

Is this the correct way to backpropagate a Neural Network?

I am writing a Neural Network frorm scratch. Below is what I have right now, based off of the math that I think I understand. ...
user avatar
0 votes
1 answer
55 views

What do we mean by the notation $\mathbf{x}_{p} \in \mathbb{R}^{N \times\left(P^{2} \cdot C\right)}$?

I was going through this VIT paper, what will it look like in torch , if we are trying to write this expression.
TheExorcist's user avatar
1 vote
1 answer
62 views

How does Bishop derive $\ln p\left(\mathbf{x} \mid \mu, \sigma^{2}\right)$, when $p$ is a Gaussian?

I am now reading the Bishop Machine Learning Book and going through every single equation. We know that in the case of a single real-valued variable $x$, the Gaussian distribution is defined by $$\...
Minty Fresh's user avatar
1 vote
0 answers
42 views

Does all GAN's in literature need to satisfy the properties of objective function of initial GAN? [closed]

Consider the following value function of the initial GAN $V(D, G) = \mathbb{E}_{x \sim p_{data(x)}} [\log D(x)] + \mathbb{E}_{z \sim p_z(z)} [1- \log D(G(z))]$ The min-max game on the value function: $...
hanugm's user avatar
  • 3,612
0 votes
1 answer
24 views

What is the right way to find the alphas in this equation?

In the Grad-CAM++ paper the following equation (7) is posed (written here without the relu function): $$ Y^c = \sum_k \Bigl( \Bigl\{ \sum_{a,b} \alpha_{ab}^{kc} \cdot \frac{\partial Y^c}{\...
mlerma54's user avatar
  • 141
0 votes
1 answer
70 views

Usability of power series in AI analysis

In mathematics, power series is given by $$f(x) = \sum\limits_{n=0}^{\infty} c_n (x-a)^n$$ where $c_n , a \in \mathbb{R}$ Although most of the courses in academics cover moment generating functions in ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
83 views

What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?

I have a question about the Grad-CAM++ paper. I do not understand how the following equation (10) for the alphas is obtained: $$ \alpha_{ij}^{kc} = \frac{\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}} {...
mlerma54's user avatar
  • 141
1 vote
1 answer
51 views

Should I need to interpret the word "metric" in "performance metric" rigorously?

Consider the following abstract from the research paper titled A Note on the Inception Score for instance Deep generative models are powerful tools that have produced impressive results in recent ...
hanugm's user avatar
  • 3,612
3 votes
2 answers
3k views

What does it mean by "gradient flow" in the context of neural networks?

Several research papers and textbooks (e.g. this) contain the phrase "gradient flow" in the context of neural networks. I am confused about whether it has any rigorous and formal way of ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
44 views

What does it mean by "dynamics of a sequence" mathematically?

Consider the following paragraph from the topic named sequential models from the textbook titled Dive into Deep Learning Both cases raise the obvious question of how to generate training data. One ...
hanugm's user avatar
  • 3,612
1 vote
1 answer
264 views

What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$?

Here is the formula of state-value function in Reinforcement Learning. What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$? Why use square brackets? Why use $\mid$? Why do ...
huang's user avatar
  • 293

1
2 3 4 5