10

There is an assumption behind the theory training a neural network, that also applies to many other supervised learning methods, that a training sample is representative of the data set as a whole - that it has been sampled fairly from the population that the learning algorithm has been set up to approximate. The term i.i.d. stands for "independent and ...


10

There are several examples. For example, one instance of using Statistical AI from my workplace is: Analyzing the behavior of the customer and their food-ordering trends, and then trying to upsell by recommending them the dishes which they might like to order/eat. This can be done through the apriori and FP-growth algorithms. We then, automated the ...


9

We typically think of machine learning models as modeling two different parts of the training data--the underlying generalizable truth (the signal), and the randomness specific to that dataset (the noise). Fitting both of those parts increases training set accuracy, but fitting the signal also increases test set accuracy (and real-world performance) while ...


9

Noise in the data, to a reasonable amount, may help the network to generalize better. Sometimes, it has the opposite effect. It partly depends on the kind of noise ("true" vs. artificial). The AI FAQ on ANN gives a good overview. Excerpt: Noise in the actual data is never a good thing, since it limits the accuracy of generalization that can be achieved ...


8

Yes and no! There's no inherent reason that machine learning systems can't deal with extreme events. As a simple version, you can learn the parameters of a Weibull distribution, or another extreme value model, from data. The bigger issue is with known-unknowns vs. unknown-unknowns. If you know that rare events are possible (as with, say, earthquake ...


7

Suppose that we have some optimization criterion $J(x)$, which we aim to optimize (maybe maximize, maybe minimize), which we can compute for a single example $x$. In an "ideal world", where we have no restrictions on computation time and memory, we would generally want to run training algorithms on the complete "ground truth" population. For example, if we'...


5

The University of Maryland published some slides (PDF) from an introductory presentation on this topic. The fourth page explains why SRL is interesting. "Traditional statistical machine learning approaches" process one sort of thing in which there is some uncertaintly. Image identification is a good example of that. "Traditional ILP/relational learning ...


5

"Assuming that we have sufficient data..." — that's quite a big assumption. Also, traditional methods are well understood, while neural networks (and especially deep learning) is still something of a black box: you train it, and then you get a mapping from input to output. But you don't really know how that mapping is achieved. It's not only about ...


4

Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies dependence (see https://stats.stackexchange.com/q/113417/82135 for more details). So, if you have non-zero correlation between two features, it means they are ...


4

There are many online services that use statistical neural networks for recommendations. For example, we have a well known service here in Russia that could give it's users recommendations for movies and shows to watch and books to read. Its recommendation core is based on many things known about a user: what movies/books he or she loves and what not, ...


3

According to Wikipedia: A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form, the data-generating process. Answer to your question: To build any neural network ...


3

From this document, as you found here, $X$ is an observed variable and $Z$ is a hidden variable; $p(X)$ is the density function of $X$. The posterior distribution of the hidden variables can then be written as follows using the Bayes’ Theorem: $$p(Z|X) = \frac{p(X|Z)p(Z)}{p(X)} = \frac{p(X|Z)p(Z)}{\int_Zp(X,Z)}$$ Now base on what you post, if we denote ...


3

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate the space along very non-linear curves, especially if you get a very deep tree. Simpler algorithms, on the other hand, tend to separate the space along linear ...


3

This formulation/interpretation can indeed be confusing (or even misleading), as the output of a neural network is usually deterministic (i.e. given the same input $x$, the output is always the same, so there is no sampling), and there isn't really a probability distribution that models any uncertainty associated with the parameters of the network or the ...


3

Not necessarily. The neural network (or whatever else you use) is a model of what you are trying to do, and usually models are not able to perfectly model reality, as it is too complex. A noise term is generally used to represent that, ie the imperfection of the model's relationship with the actual world.


2

Not strictly examples of AI, but related to the greater AI project: But us in the psychology / cognitive science side of things sure love our bayesian modelling! In fact there are people who believe that a theory grounded in such analysis would ultimately bring us to a unified theory of the brain and cognition! Unfortunately to my knowledge, these theories ...


2

You are implying that such ideas are novel, and that such tools do not exist. But the idea is very popular, and there are numerous tools. We need to write a program that would recognize that a word is connected to other words in the same way in both language. Then it would know those two words must have the same meaning. You are describing the essence of ...


2

Use of Transposed Convolution can lead to checkerboard artifacts. So we prefer to up-sample and then apply convolution. You can check this article for more information https://distill.pub/2016/deconv-checkerboard/.


2

The problem you're trying to address can, in some sense, be viewed as a Feature Selection problem. If you look for literature using only those words, you're not going to find what you're looking for though. In general, "Feature Selection" simply refers to the problem where you already have a large amount of features, and you're simply deciding to select ...


2

In the YouTube depiction of CS294-112 fall 2017 lecture 3 Reinforcement Learning, Levine, the transition of the finite horizon expected reward to a form where each transition is decoupled from the entire Markov chain of state-action marginals is explained between $t_{video}$ = 44:04 and $t_{video}$ = 45:22. At t=44:29, the probabilistic expectation where no ...


2

The use of KL provides a more intuitive way of what the ELBO is attempting to maximize. Basically, we want to find a posterior approximation such that $p(z\mid x) \approx q(z)\in\mathcal{Q}$ $$KL(q(z)\parallel p(z\mid x)) \rightarrow \min_{q(z)\in\mathcal{Q}}$$ As a result of this, while finding this optimal posterior approximation, we maximize the ...


2

Are you thinking something like Information Gain? Information Gain basically uses the concept of information entropy to determine if splitting a variable is useful.


1

Yes, you can use sklearn's confusion_matrix. To explicitly extract the false positives and negatives, you can do from sklearn.metrics import confusion_matrix y_true = [0, 1, 0, 1] y_pred = [1, 1, 1, 0] tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()


1

A linear model has a linear decision boundary. So in the case of the question, you need to draw two multimodal distributions whose domains do not overlap at all and then you can just say the linear model would should all to the right of some number would be class 1 and all to the left would be class 2


1

The predictions tend to move towards the mean of the series as one predicts for longer horizons. Also, in general, optimal long range forecast is the process mean. In other words, the past of the process contains no information on the development of the process in the distant future. And, this might be the reason that you are getting poor forecasts. ...


1

Yes you can, provided you know about $f$ and $g$. Expression $X3 = f(X1, g(X1))$can be written as $X3 = h(X1)$ where $h$ takes into account both $f$ and $g$. After this finding the PDF is simple by differentiating the CDF: $$ F_{X3} (x3) = P(X3 \leq x3) = P(h(X1) \leq x3) = P(X1 \leq h^{-1}(x3))$$ $$ \frac {d F_{X3} (x3)}{dx3} = \frac {d P(X1 \leq h^{-1}(...


1

What is a statistical model? According to Anthony C. Davison (in the book Statistical Models), a statistical model is a probability distribution constructed to enable inferences to be drawn or decisions made from data. The probability distribution represents the variability of the data. Are all neural networks statistical models? All neural networks that ...


1

Statistical AI, arising from machine learning, tends to be more concerned with "inductive" thought: given a set of patterns, induce the trend. Classical AI is the branch of artificial intelligence research that concerns itself with attempting to explicitly represent human knowledge in a declarative form i.e given a set of constraints, deduce a conclusion. ...


1

Feature Extraction Patterson and Gibson's Deep Learning, A Practitioner's Approach, O'Reiley, 2017 states, "Convolutional Neural Networks (CNNs) ... consistently top image classification competitions," which is consistent with our experience in the lab. If your data is multi-dimensional in that pain is on a scale from one to ten, fever is in degrees, and ...


1

I would say it is useful if you have an extensive knowledge in the domain you want to apply your model in. You also need more data for it to yield reasonable results. As for real world uses l can only think of trading at the moment.


Only top voted, non community-wiki answers of a minimum length are eligible