Skip to main content

Questions tagged [machine-learning]

For questions related to machine learning (ML), which is a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data). ML is usually divided into supervised, unsupervised and reinforcement learning. Deep learning is a subfield of ML that uses deep artificial neural networks.

Filter by
Sorted by
Tagged with
104 votes
9 answers
17k views

What is the difference between artificial intelligence and machine learning?

These two terms seem to be related, especially in their application in computer science and software engineering. Is one a subset of another? Is one a tool used to build a system for the other? What ...
intcreator's user avatar
  • 1,335
102 votes
4 answers
89k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
Asciiom's user avatar
  • 1,171
97 votes
3 answers
86k views

What is self-supervised learning in machine learning?

What is self-supervised learning in machine learning? How is it different from supervised learning?
nbro's user avatar
  • 41k
74 votes
4 answers
106k views

Why does the transformer do better than RNN and LSTM in long-range context dependencies?

I am reading the article How Transformers Work where the author writes Another problem with RNNs, and LSTMs, is that it’s hard to parallelize the work for processing sentences, since you have to ...
DRV's user avatar
  • 1,713
68 votes
10 answers
47k views

Why is Python such a popular language in the AI field?

First of all, I'm a beginner studying AI and this is not an opinion-oriented question or one to compare programming languages. I'm not implying that Python is the best language. But the fact is that ...
Douglas Ferreira's user avatar
66 votes
4 answers
17k views

Are neural networks prone to catastrophic forgetting?

Imagine you show a neural network a picture of a lion 100 times and label it with "dangerous", so it learns that lions are dangerous. Now imagine that previously you have shown it millions ...
zooby's user avatar
  • 2,216
43 votes
4 answers
59k views

What is the time complexity for training a neural network using back-propagation?

Suppose that a NN contains $n$ hidden layers, $m$ training examples, $x$ features, and $n_i$ nodes in each layer. What is the time complexity to train this NN using back-propagation? I have a basic ...
user avatar
40 votes
3 answers
33k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, ...
Alecto Irene Perez's user avatar
39 votes
5 answers
21k views

What is the difference between latent and embedding spaces?

In general, the word "latent" means "hidden" and "to embed" means "to incorporate". In machine learning, the expressions "hidden (or latent) space" ...
nbro's user avatar
  • 41k
38 votes
1 answer
43k views

What is the "temperature" in the GPT models?

What does the temperature parameter mean when talking about the GPT models? I know that a higher temperature value means more randomness, but I want to know how randomness is introduced. Does ...
Tom Dörr's user avatar
  • 503
36 votes
7 answers
8k views

What are examples of promising AI/ML techniques that are computationally intractable?

To produce tangible results in the field of AI/ML, one must take theoretical results under the lens of computational complexity. Indeed, minimax effectively solves any two-person "board game"...
k.c. sayz 'k.c sayz''s user avatar
31 votes
7 answers
5k views

Is artificial intelligence vulnerable to hacking? [closed]

The paper The Limitations of Deep Learning in Adversarial Settings explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. ...
Surya Sg's user avatar
  • 495
31 votes
2 answers
1k views

How is a deep neural network different from other neural networks?

How is a neural network having the "deep" adjective actually distinguished from other similar networks?
kenorb's user avatar
  • 10.5k
30 votes
7 answers
17k views

How can an AI train itself if no one is telling it if its answer is correct or wrong?

I am a programmer but not in the field of AI. A question constantly confuses me is that how can an AI be trained if we human beings are not telling it its calculation is correct? For example, news ...
user avatar
29 votes
4 answers
79k views

Why does C++ seem less widely used than Python in AI?

I just want to know why do machine learning engineers and AI programmers use languages like Python to perform AI tasks and not C++, even though C++ is technically a more powerful language than Python.
Mark ellon's user avatar
29 votes
4 answers
41k views

Can a neural network be used to predict the next pseudo random number?

Is it possible to feed a neural network the output from a random number generator and expect it learn the hashing (or generator) function, so that it can predict what will be the next generated pseudo-...
AshTyson's user avatar
  • 401
28 votes
4 answers
12k views

How could we build a neural network that is invariant to permutations of the inputs?

Given a neural network $f$ that takes as input $n$ data points: $x_1, \dots, x_n$. We say $f$ is permutation invariant if $$f(x_1 ... x_n) = f(\sigma(x_1 ... x_n))$$ for any permutation $\sigma$. How ...
Josef Ondrej's user avatar
25 votes
3 answers
90k views

How do I choose the optimal batch size?

Batch size is a term used in machine learning and refers to the number of training examples utilised in one iteration. The batch size can be one of three options: batch mode: where the batch size is ...
Sebastian Nielsen's user avatar
24 votes
2 answers
1k views

Are there any ongoing projects which use the Stack Exchange for machine learning?

Are there any ongoing AI projects which use the Stack Exchange for machine learning?
Techidiot's user avatar
  • 349
23 votes
5 answers
4k views

What is the difference between machine learning and deep learning?

Can someone explain to me the difference between machine learning and deep learning? Is it possible to learn deep learning without knowing machine learning?
Addis's user avatar
  • 333
21 votes
1 answer
7k views

Why do you not see dropout layers on reinforcement learning examples?

I've been looking at reinforcement learning, and specifically playing around with creating my own environments to use with the OpenAI Gym AI. I am using agents from the stable_baselines project to ...
Matt Hamilton's user avatar
21 votes
5 answers
4k views

Why does Batch Normalization work?

Adding BatchNorm layers improves training time and makes the whole deep model more stable. That's an experimental fact that is widely used in machine learning practice. My question is - why does it ...
Kostya's user avatar
  • 2,544
20 votes
1 answer
36k views

What is a fully convolution network?

I was surveying some literature related to Fully Convolutional Networks and came across the following phrase, A fully convolutional network is achieved by replacing the parameter-rich fully ...
r4bb1t's user avatar
  • 335
20 votes
4 answers
14k views

Why do we need floats for using neural networks?

Is it possible to make a neural network that uses only integers by scaling input and output of each function to [-INT_MAX, INT_MAX]? Is there any drawbacks?
elimohl's user avatar
  • 311
20 votes
2 answers
4k views

What is the "Hello World" problem of Reinforcement Learning?

As we all know, "Hello World" is usually the first program that any programmer learns/implements in any language/framework. As Aurélien Géron mentioned in his book that MNIST is often called ...
Arpit-Gole's user avatar
19 votes
1 answer
5k views

Why has the cross-entropy become the classification standard loss function and not Kullback-Leibler divergence?

The cross-entropy is identical to the KL divergence plus the entropy of the target distribution. The KL divergence equals zero when the two distributions are the same, which seems more intuitive to me ...
Josh Albert's user avatar
18 votes
1 answer
507 views

Are these two versions of back-propagation equivalent?

Just for fun, I am trying to develop a neural network. Now, for backpropagation I saw two techniques. The first one is used here and in many other places too. What it does is: It computes the ...
Aspie96's user avatar
  • 181
17 votes
2 answers
2k views

When is deep learning overkill?

For example, for classifying emails as spam, is it worthwhile - from a time/accuracy perspective - to apply deep learning (if possible) instead of another machine learning algorithm? Will deep ...
Alexander's user avatar
  • 293
17 votes
3 answers
1k views

How would an AI learn language?

I was think about AIs and how they would work, when I realised that I couldn't think of a way that an AI could be taught language. A child tends to learn language through associations of language and ...
AvahW's user avatar
  • 275
17 votes
1 answer
534 views

Are search engines considered AI?

Are search engines considered AI because of the way they analyze what you search for and remember it? Or how they send you ads of what you've searched for recently? Is this considered AI or just ...
baranskistad's user avatar
17 votes
2 answers
7k views

What is the difference between active learning and online learning?

The definitions for these two appear to be very similar, and frankly, I've been only using the term "active learning" the past couple of years. What is the actual difference between the two? ...
David's user avatar
  • 313
17 votes
1 answer
1k views

Are information processing rules from Gestalt psychology still used in computer vision today?

Decades ago there were and are books in machine vision, which by implementing various information processing rules from gestalt psychology, got impressive results with little code or special hardware ...
Gottfried William's user avatar
16 votes
8 answers
29k views

How to classify data which is spiral in shape?

I have been messing around in tensorflow playground. One of the input data sets is a spiral. No matter what input parameters I choose, no matter how wide and deep the neural network I make, I cannot ...
Souradeep Nanda's user avatar
16 votes
1 answer
1k views

How to stay a up-to-date researcher in ML/RL community?

As a student who wants to work on machine learning, I would like to know how it is possible to start my studies and how to follow it to stay up-to-date. For example, I am willing to work on RL and MAB ...
Amin's user avatar
  • 481
16 votes
1 answer
388 views

Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data. So we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our ...
Philipp Cannons's user avatar
15 votes
6 answers
7k views

Why do many AI bots feel the need to be know-it-alls?

Having used various AI bots often over recent months, I noticed that often it will claim to know something, even if it doesn't. It would then either explain something which is clearly nonsense, or by ...
ben svenssohn's user avatar
15 votes
4 answers
6k views

Why did machine learning only become viable after Nvidia's chips were available?

I listened to a talk attended by a panel consisting of two influential Chinese scientists: Wang Gang and Yu Kai, among others. When asked about the biggest bottleneck in the development of artificial ...
Lerner Zhang's user avatar
15 votes
3 answers
4k views

Does Monte Carlo tree search qualify as machine learning?

To the best of my understanding, the Monte Carlo tree search (MCTS) algorithm is an alternative to minimax for searching a tree of nodes. It works by choosing a move (generally, the one with the ...
Inertial Ignorance's user avatar
15 votes
1 answer
2k views

What are the implications of the "No Free Lunch" theorem for machine learning?

The No Free Lunch (NFL) theorem states (see the paper Coevolutionary Free Lunches by David H. Wolpert and William G. Macready) any two algorithms are equivalent when their performance is averaged ...
user avatar
14 votes
3 answers
1k views

How does noise affect generalization?

Does increasing the noise in data help to improve the learning ability of a network? Does it make any difference or does it depend on the problem being solved? How is it affect the generalization ...
kenorb's user avatar
  • 10.5k
14 votes
3 answers
230 views

Is there a way to understand neural networks without using the concept of brain?

Is there a way to understand, for instance, a multi-layered perceptron without hand-waving about them being similar to brains, etc? For example, it is obvious that what a perceptron does is ...
Evgeniy's user avatar
  • 249
14 votes
1 answer
15k views

How can the convolution operation be implemented as a matrix multiplication?

How can the convolution operation used by CNNs be implemented as a matrix-vector multiplication? We often think of the convolution operation in CNNs as a kernel that slides across the input. However, ...
nbro's user avatar
  • 41k
14 votes
2 answers
833 views

Should deep residual networks be viewed as an ensemble of networks?

The question is about the architecture of Deep Residual Networks (ResNets). The model that won the 1-st places at "Large Scale Visual Recognition Challenge 2015" (ILSVRC2015) in all five main tracks: ...
Erba Aitbayev's user avatar
13 votes
3 answers
14k views

What is the relation between semi-supervised and self-supervised visual representation learning?

What's the differences between semi-supervised learning and self-supervised visual representation learning, and how they are connected?
0x90's user avatar
  • 281
13 votes
4 answers
3k views

What are the purposes of autoencoders?

Autoencoders are neural networks that learn a compressed representation of the input in order to later reconstruct it, so they can be used for dimensionality reduction. They are composed of an encoder ...
nbro's user avatar
  • 41k
13 votes
2 answers
831 views

Are the shortcomings of neural networks diminishing?

Having worked with neural networks for about half a year, I have experienced first-hand what are often claimed as their main disadvantages, i.e. overfitting and getting stuck in local minima. However, ...
user avatar
13 votes
2 answers
7k views

What are other examples of theoretical machine learning books?

I am looking for a book about machine learning that would suit my physics background. I am more or less familiar with classical and complex analysis, theory of probability, сcalculus of variations, ...
Ilya's user avatar
  • 133
13 votes
3 answers
21k views

Is it possible to train a neural network to estimate a vehicle's length?

I have a large dataset (over 100k samples) of vehicles with the ground truth of their lengths. Is it possible to train a deep network to measure/estimate vehicle length? I haven't seen any papers ...
Naji's user avatar
  • 139
13 votes
1 answer
3k views

How exactly can ReLUs approximate non-linear and curved functions?

Currently, the most commonly used activation functions are ReLUs. So I answered this question What is the purpose of an activation function in neural networks? and, while writing the answer, it struck ...
user avatar
12 votes
3 answers
5k views

Why is the derivative of the activation functions in neural networks important?

I'm new to NN. I am trying to understand some of its foundations. One question that I have is: why the derivative of an activation function is important (not the function itself), and why it's the ...
Mary's user avatar
  • 983

1
2 3 4 5
48