Questions tagged [statistical-ai]
For questions about the applications/clarifications/intuitions/proofs behind the use of statistical methods in AI/ML programs.
53
questions
1
vote
3
answers
73
views
What is the difference between q and p in Statistical Notation(used in VAE)?
I'm looking at general visuals of Variational Autoencoders and I'm seeing that the encoder is typically expressed as q(z|x) with phi as a subscript while the decoder is p(x|z) with theta as a ...
1
vote
1
answer
27
views
How can we construct a skewed noise distribution using the maximum likelihood approach?
When the probability of observing a large positive error is larger than the probability
of observing a large negative error in binary classification, how can this be modelled by a skewed noise ...
- 492
0
votes
1
answer
74
views
Loss function of logistic Regression Geometric
In linear Regression, I train the Model so the Graph runs best through the Data Points, so the geometric distance between f(x) and y^i is minimized. Now is it correct that in logistic Regression I do ...
- 1
0
votes
0
answers
36
views
Must dataset strictly come from marginal distribution in VAE?
My question is what if the population of the dataset is another marginal distribution, but whose support covers the original marginal distribution $p(\mathbf{x})$, can we use VAE to infer this target ...
- 41
0
votes
0
answers
31
views
Plotting gradient over weights ratio, mean or std dev to synthetize tensors?
I was studying the lecture of Andrej Karpathy about "Activations, Gradients and BatchNorm" that he uploaded on youtube: link here.
At chapter "viz #4: update:data ratio over time" ...
- 1
2
votes
1
answer
64
views
Did old-fashioned, rule-based systems die out when statistical learning broke through into NLP?
It is not quite clear to me whether the statistical approach superseded the rule-based system in the 90s.
McMahon and Smith (1998) report that many other researchers used "hybrids of statistical ...
0
votes
0
answers
102
views
The Small Set Expansion Hypothesis, this problem was solved or is open problem yet?
I found this problem by article called "Ten Lectures and Forty-Two Open Problems in the Mathematics of Data Science" published at 2016. I`m looking for an open problem at Data Science or/and ...
0
votes
1
answer
44
views
What does it mean by "dynamics of a sequence" mathematically?
Consider the following paragraph from the topic named sequential models from the textbook titled Dive into Deep Learning
Both cases raise the obvious question of how to generate training
data. One ...
- 3,612
2
votes
1
answer
204
views
Is logic AI a complement to learning AI?
I want to know the relation between logic AI and learning AI.
Logic AI here refers to the branch of AI that is based on mathematical logic. Learning AI refers to the branch of AI that is based on ...
- 3,612
2
votes
0
answers
112
views
Machine learning with raw data alone / or raw data with its statistics
My question is very general and it does not originate from a specific problem. Let's assume that, through experience, we have learned that some statistical property of a set of data is important in ...
- 61
1
vote
1
answer
274
views
Can I treat "experience" in reinforcement learning as "training data" in statistical learning?
Statistics is a branch of mathematics that extracts useful information from data. The data is generally called as "training data" in statistical (machine) learning.
Consider the following ...
- 3,612
1
vote
0
answers
30
views
How can the probability of two disjoint events be non-zero?
Let $A$ and $B$ be two models for a classification task. Let $x$ be a test set and $M$ be a metric for the classification task. $X$ be a random variable on test sets.
Now,
$M(A,x) = $ Score of model $...
- 3,612
1
vote
2
answers
179
views
Is there any model that is probabilistic but not statistical?
While studying about the n-gram models, I encountered the terms "statistical model" and "probabilistic model" several times.
I got a basic doubt that will there be any ...
- 3,612
1
vote
3
answers
2k
views
How much statistics is involved in AI?
I am a 3rd-year math major, who is interested in computer science, particularly algorithms and competitive programming (did some olympiads in high school, ACM ICPC in university, etc.), and I have ...
- 119
2
votes
2
answers
359
views
Why is the equation $\mathbb{E} \left[ (Y - \hat{Y})^2 \right] = \left(f(X) - \hat{f}(X) \right)^2 + \operatorname{Var} (\epsilon)$ true?
In the book An Introduction to Statistical Learning, the authors claim (equation 2.3, p. 19, chapter 2)
$$\mathbb{E} \left[ (Y - \hat{Y})^2 \right] = \left(f(X) - \hat{f}(X) \right)^2 + \operatorname{...
- 39.6k
1
vote
2
answers
72
views
What would be the reason behind using plots (such as box-plots or histograms) for ML development?
I've been learning Python machine-learning using this project report and the guy who wrote it begins by visualizing his data using various statistical analysis methods: histograms, density plots, box ...
- 13
1
vote
0
answers
23
views
Does distribution of data augmentation parameters matter?
Idea
Let's say we have simple pictures dataset containing 40x40 images of digits. We have only one image of each digit. We want to use that as training set, but we need more data, so we use data ...
- 242
2
votes
2
answers
213
views
Is the target assumed to be a noisy version of the output of the model in machine learning?
I wonder if the following equation (you can find it in almost every ML book) refers to a general assumption that we make when using machine learning:
$$y = f(x)+\epsilon,$$
where $y$ is our output, $f$...
- 99
0
votes
1
answer
74
views
Can AI be understood as a generalized statistics tool? [duplicate]
I am a (soon-to-become, to be honest) theoretical physicist. I want to learn a bit about AI. So as you know in physics we develop theories based on as few and as simple basic equations as possible ...
- 109
3
votes
0
answers
26
views
Mapping given probabilities to empirical probabilities
Consider following problem statement:
You have given $n$ actions. You can perform any of them. Each action gives you success with some probability. The challenge is to perform given finite number of ...
- 221
2
votes
0
answers
119
views
How to calculate the data noise variance for a prediction interval?
I have a neural network that connects $N$ input variables to $M$ output variables (qoi). By default, neural networks just give out point estimations.
Now, I want to plot some of the quantity of ...
- 131
3
votes
0
answers
71
views
Simplification of expected reward under the limit in continuous tasks
I was reading the average reward setting for continuous tasks from rich sutton's book (page 202, 2nd edition). There he perform a simplification over the expected reward under the limit approaching to ...
- 683
1
vote
0
answers
38
views
Do the variance and bias belong to the policy or value functions?
Recently, I read many papers on variance and bias. But I am still confused by the two notions, the variance or bias belongs to who? Policy or value? If the variance or bias is large or low, what ...
- 150
1
vote
1
answer
69
views
Is there any way to apply linear transformations on a vector other than matrix multiplication?
I am trying to optimize the cost function calculation in regression analysis using a non-matrix multiplication based approach.
More specifically, I have a point $x = (1, 1, 2, 3)$, to which I want to ...
- 111
3
votes
0
answers
58
views
Is maximum likelihood estimation meaningless for a dataset of only outliers?
From my understanding, maximum likelihood estimation chooses the set of parameters for the estimator that maximizes likelihood with the ground truth distribution.
I always interpreted it as the ...
- 1,409
1
vote
0
answers
49
views
What is the difference between an generalised estimating equation and a recurrent neural network?
What is the difference between a generalised estimating equation (GEE) model and a recurrent neural network (RNN) model, in terms of what these two models are doing? Apart from the differences in the ...
- 151
4
votes
1
answer
623
views
Is traditional machine learning obsolete given that neural networks typically outperform them?
I have been coming across visualizations showing that the neural nets tend to perform better as compared to the traditional machine learning algorithms (Linear regression, Log regression, etc.)
...
- 143
3
votes
2
answers
2k
views
How can supervised learning be viewed as a conditional probability of the labels given the inputs?
In the literature and textbooks, one often sees supervised learning expressed as a conditional probability, e.g.,
$$\rho(\vec{y}|\vec{x},\vec{\theta})$$
where $\vec{\theta}$ denotes a learned set of ...
- 33
4
votes
2
answers
828
views
What's going on in the equation of the variational lower bound?
I don't really understand what this equation is saying or what the purpose of the ELBO is. How does it help us find the true posterior distribution?
- 351
4
votes
1
answer
111
views
Is there any measure of separability of classes?
I want to know if there is a measure of how well two classes in Y are separable (linearly or not) based on their features in X. Easiest way of explaining this is to compare it to correlation ...
- 1,240
0
votes
1
answer
56
views
How to calculate the false positives and negatives?
I have a huge amount of data and I want to calculate my false positive and false negative. Is there a software that can help me determine it?
- 11
2
votes
1
answer
86
views
When could a linear discriminant give excellent or possibly even the optimal classification accuracy?
I am actually reading the linear classification. There is a question in the question set behind the chapter in the book as follows:
Sketch two multimodal distributions for which a linear discriminant ...
- 21
2
votes
1
answer
123
views
Is explainable AI more feasible through symbolic AI or soft computing?
Is explainable AI more feasible through symbolic AI or soft computing?
How much each paradigm, symbolic AI and soft computing (or hybrid approaches), addresses explanation and argumentation, where ...
1
vote
1
answer
79
views
Auto-regression - Reduce error in prediction
I am trying to develop a time series model using autoregression. The data set is like as follows
...
- 183
14
votes
3
answers
6k
views
Why exactly do neural networks require i.i.d. data?
In reinforcement learning, successive states (actions and rewards) can be correlated. An experience replay buffer was used, in the DQN architecture, to avoid training the neural network (NN), which ...
- 39.6k
1
vote
2
answers
69
views
Can we derive the distribution of a random variable based on a dependent random variable's distribution?
In the diagram below, there are three variables: X3 is a function of (depends on) X1 and X2, ...
- 111
6
votes
3
answers
7k
views
Are neural networks statistical models?
By reading the abstract of Neural Networks and Statistical Models paper it would seem that ANNs are statistical models.
In contrast Machine Learning is not just glorified Statistics.
I am looking ...
- 216
3
votes
2
answers
473
views
What makes a machine learning algorithm a low variance one or a high variance one?
Some examples of low-variance machine learning algorithms include linear regression, linear discriminant analysis, and logistic regression.
Examples of high-variance machine learning algorithms ...
- 358
2
votes
1
answer
524
views
Does the correlation between inputs affect the model performance?
I'm currently working on a regression problem and I have 10 inputs/attributes.
What should I do if there are correlations between different features of the input data? Does the correlation between ...
- 133
6
votes
2
answers
931
views
What are the differences in scope between statistical AI and classical AI?
What are the differences in scope between statistical AI and classical AI?
Real-world examples would be appreciated.
- 323
3
votes
2
answers
261
views
Reinforcement learning objective as conditional expectations
In one of his lectures Levine describes the objective of reinforcement learning as: $$J(\tau) = E_{\tau\sim p_\theta(\tau)}[r(\tau)]$$
where $\tau$ refers to a single trajectory and $p_\theta(\tau)$ ...
- 163
10
votes
2
answers
4k
views
Is Nassim Taleb right about AI not being able to accurately predict certain types of distributions?
So Taleb has two heuristics to generally describe data distributions. One is Mediocristan, which basically means things that are on a Gaussian distribution such as height and/or weight of people.
The ...
- 171
2
votes
1
answer
1k
views
Why do we need Upsampling and Downsampling in Progressive Growing of Gans
I was working recently on Progressive Growing of GANs (aka PGGANs). I have implemented the whole architecture, but the problem that was ticking my mind is that in simple GANs, like DCGAN, PIX2PIX, we ...
1
vote
0
answers
91
views
Confidence interval around a DNN prediction
I am facing a problem and do not know whether it is even solvable: I want to predict the behaviour of a system using a DNN, say a CNN, in the sense that I want to predict the time and intensity of a ...
- 173
-1
votes
1
answer
66
views
How important will statistical learning be to a conscious AI?
Deep learning is based on getting a large number of samples and essentially making statistical deductions and outputting probabilities.
On the other hand, we have formal programming languages, like ...
- 2,186
3
votes
2
answers
109
views
Finding the right questions to increase accuracy in classification
Lets say I have a list of 100k medical cases from my hospital, each row = patient with symptoms (such as fever , funny smell, pain etc.. ) and my labels are medical conditions such as Head trauma, ...
- 133
0
votes
1
answer
422
views
Are standard deviation, variance, skew good features for ML?
Pretty simple question here:
Is it useful to use the standard deviation, skew, kurtosis, or any other extrapolatory stats as features, and if so in which problem sets?
In this case, I am talking ...
- 1,235
3
votes
1
answer
97
views
Is there a way of computing a prominence score based on the prevalence of features in an image?
Is there any previous work on computing some sort of prominence score based on the prevalence of features in an image?
For example, let's say I am classifying images based on whether or not they have ...
- 1,235
4
votes
4
answers
213
views
How to figure out which words have the same meaning in two different languages?
Imagine two languages that have only these words:
Man = 1,
deer = 2,
eat = 3,
grass = 4
And you would form all sentences possible from these words:
...
- 219
3
votes
1
answer
205
views
With gradient descent w/MSE on a regression, must/should every Epoch use the exact same training samples?
Let's say I've got a training sample set of 1 million records, which I pull batches of 100 from to train a basic regression model using gradient descent and MSE as a loss function. Assume test and ...
- 131