Skip to main content

Questions tagged [machine-learning]

For questions related to machine learning (ML), which is a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data). ML is usually divided into supervised, unsupervised and reinforcement learning. Deep learning is a subfield of ML that uses deep artificial neural networks.

Filter by
Sorted by
Tagged with
97 votes
3 answers
86k views

What is self-supervised learning in machine learning?

What is self-supervised learning in machine learning? How is it different from supervised learning?
nbro's user avatar
  • 41k
66 votes
4 answers
17k views

Are neural networks prone to catastrophic forgetting?

Imagine you show a neural network a picture of a lion 100 times and label it with "dangerous", so it learns that lions are dangerous. Now imagine that previously you have shown it millions ...
zooby's user avatar
  • 2,216
5 votes
4 answers
2k views

What is the fundamental difference between an ML model and a function?

A model can be roughly defined as any design that is able to solve an ML task. Examples of models are the neural network, decision tree, Markov network, etc. A function can be defined as a set of ...
hanugm's user avatar
  • 3,930
43 votes
4 answers
59k views

What is the time complexity for training a neural network using back-propagation?

Suppose that a NN contains $n$ hidden layers, $m$ training examples, $x$ features, and $n_i$ nodes in each layer. What is the time complexity to train this NN using back-propagation? I have a basic ...
user avatar
20 votes
1 answer
36k views

What is a fully convolution network?

I was surveying some literature related to Fully Convolutional Networks and came across the following phrase, A fully convolutional network is achieved by replacing the parameter-rich fully ...
r4bb1t's user avatar
  • 335
4 votes
2 answers
927 views

What is machine learning?

What is the definition of machine learning? What are the advantages of machine learning?
Israr Ali's user avatar
  • 103
104 votes
9 answers
17k views

What is the difference between artificial intelligence and machine learning?

These two terms seem to be related, especially in their application in computer science and software engineering. Is one a subset of another? Is one a tool used to build a system for the other? What ...
intcreator's user avatar
  • 1,335
14 votes
1 answer
15k views

How can the convolution operation be implemented as a matrix multiplication?

How can the convolution operation used by CNNs be implemented as a matrix-vector multiplication? We often think of the convolution operation in CNNs as a kernel that slides across the input. However, ...
nbro's user avatar
  • 41k
3 votes
2 answers
2k views

How can supervised learning be viewed as a conditional probability of the labels given the inputs?

In the literature and textbooks, one often sees supervised learning expressed as a conditional probability, e.g., $$\rho(\vec{y}|\vec{x},\vec{\theta})$$ where $\vec{\theta}$ denotes a learned set of ...
Jammy's user avatar
  • 33
3 votes
2 answers
1k views

What is a probability distribution in machine learning?

If we were learning or working in the machine learning field, then we frequently come across the term "probability distribution". I know what probability, conditional probability, and ...
Eka's user avatar
  • 1,066
102 votes
4 answers
89k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
Asciiom's user avatar
  • 1,171
39 votes
5 answers
21k views

What is the difference between latent and embedding spaces?

In general, the word "latent" means "hidden" and "to embed" means "to incorporate". In machine learning, the expressions "hidden (or latent) space" ...
nbro's user avatar
  • 41k
31 votes
7 answers
5k views

Is artificial intelligence vulnerable to hacking? [closed]

The paper The Limitations of Deep Learning in Adversarial Settings explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. ...
Surya Sg's user avatar
  • 495
31 votes
2 answers
1k views

How is a deep neural network different from other neural networks?

How is a neural network having the "deep" adjective actually distinguished from other similar networks?
kenorb's user avatar
  • 10.5k
29 votes
4 answers
41k views

Can a neural network be used to predict the next pseudo random number?

Is it possible to feed a neural network the output from a random number generator and expect it learn the hashing (or generator) function, so that it can predict what will be the next generated pseudo-...
AshTyson's user avatar
  • 401
13 votes
4 answers
3k views

What are the purposes of autoencoders?

Autoencoders are neural networks that learn a compressed representation of the input in order to later reconstruct it, so they can be used for dimensionality reduction. They are composed of an encoder ...
nbro's user avatar
  • 41k
13 votes
2 answers
7k views

What are other examples of theoretical machine learning books?

I am looking for a book about machine learning that would suit my physics background. I am more or less familiar with classical and complex analysis, theory of probability, сcalculus of variations, ...
Ilya's user avatar
  • 133
7 votes
3 answers
7k views

Are neural networks statistical models?

By reading the abstract of Neural Networks and Statistical Models paper it would seem that ANNs are statistical models. In contrast Machine Learning is not just glorified Statistics. I am looking ...
Leo Gallucci's user avatar
40 votes
3 answers
33k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, ...
Alecto Irene Perez's user avatar
38 votes
1 answer
43k views

What is the "temperature" in the GPT models?

What does the temperature parameter mean when talking about the GPT models? I know that a higher temperature value means more randomness, but I want to know how randomness is introduced. Does ...
Tom Dörr's user avatar
  • 503
15 votes
4 answers
6k views

Why did machine learning only become viable after Nvidia's chips were available?

I listened to a talk attended by a panel consisting of two influential Chinese scientists: Wang Gang and Yu Kai, among others. When asked about the biggest bottleneck in the development of artificial ...
Lerner Zhang's user avatar
9 votes
1 answer
559 views

What are the different approaches used in Machine Learning?

There seem to be so many sub-fields, so I'm interested in getting a better understanding of the approaches. I'm looking for information on a single framework per answer, in order to allow for ...
kakaz's user avatar
  • 271
8 votes
3 answers
4k views

What is the difference between hypothesis space and representational capacity?

I am reading Goodfellow et al Deeplearning Book. I found it difficult to understand the difference between the definition of the hypothesis space and representation capacity of a model. In Chapter 5,...
Qwarzix's user avatar
  • 83
8 votes
3 answers
1k views

How can I start learning mathematics for machine learning?

I am an Android programmer. Now, I would like to learn machine learning. I know it requires a mathematical background, like statistics, probability, calculus and linear algebra. However, I am a bit ...
Anko6's user avatar
  • 93
7 votes
2 answers
6k views

How can I handle overfitting in reinforcement learning problems?

So this is my current result (loss and score per episode) of my RL model in a simple two players game: I use DQN with CNN as a policy and target networks. I train my model using Adam optimizer and ...
malioboro's user avatar
  • 2,829
5 votes
2 answers
1k views

Why do we need both the validation set and test set?

I know that this has been asked a hundred times before, however, I was not able to find a question (and an answer) which actually answered what I wanted to know, respectively, which explained it in a ...
Golo Roden's user avatar
5 votes
3 answers
2k views

What is the actual learning algorithm: back-propagation or gradient descent?

What is the actual learning algorithm: back-propagation or gradient descent (or, in general, the optimization algorithm)? I am reading through chapter 8 of Parallel Distributed Processing hand book ...
Sreedhar Veluri's user avatar
74 votes
4 answers
106k views

Why does the transformer do better than RNN and LSTM in long-range context dependencies?

I am reading the article How Transformers Work where the author writes Another problem with RNNs, and LSTMs, is that it’s hard to parallelize the work for processing sentences, since you have to ...
DRV's user avatar
  • 1,713
68 votes
10 answers
47k views

Why is Python such a popular language in the AI field?

First of all, I'm a beginner studying AI and this is not an opinion-oriented question or one to compare programming languages. I'm not implying that Python is the best language. But the fact is that ...
Douglas Ferreira's user avatar
25 votes
3 answers
90k views

How do I choose the optimal batch size?

Batch size is a term used in machine learning and refers to the number of training examples utilised in one iteration. The batch size can be one of three options: batch mode: where the batch size is ...
Sebastian Nielsen's user avatar
23 votes
5 answers
4k views

What is the difference between machine learning and deep learning?

Can someone explain to me the difference between machine learning and deep learning? Is it possible to learn deep learning without knowing machine learning?
Addis's user avatar
  • 333
21 votes
5 answers
4k views

Why does Batch Normalization work?

Adding BatchNorm layers improves training time and makes the whole deep model more stable. That's an experimental fact that is widely used in machine learning practice. My question is - why does it ...
Kostya's user avatar
  • 2,544
17 votes
3 answers
1k views

How would an AI learn language?

I was think about AIs and how they would work, when I realised that I couldn't think of a way that an AI could be taught language. A child tends to learn language through associations of language and ...
AvahW's user avatar
  • 275
15 votes
1 answer
2k views

What are the implications of the "No Free Lunch" theorem for machine learning?

The No Free Lunch (NFL) theorem states (see the paper Coevolutionary Free Lunches by David H. Wolpert and William G. Macready) any two algorithms are equivalent when their performance is averaged ...
user avatar
13 votes
1 answer
3k views

How exactly can ReLUs approximate non-linear and curved functions?

Currently, the most commonly used activation functions are ReLUs. So I answered this question What is the purpose of an activation function in neural networks? and, while writing the answer, it struck ...
user avatar
12 votes
3 answers
601 views

What are the main problems hindering current AI development?

I have a background in Computer Engineering and have been working on developing better algorithms to mimic human thought. (One of my favorites is Analogical Modeling as applied to language processing ...
callyalater's user avatar
10 votes
2 answers
789 views

What are the learning limitations of neural networks trained with backpropagation?

In 1969, Seymour Papert and Marvin Minsky showed that Perceptrons could not learn the XOR function. This was solved by the backpropagation network with at least one hidden layer. This type of network ...
S.L. Barth is on codidact.com's user avatar
7 votes
5 answers
2k views

How can action recognition be achieved?

For example, I would like to train my neural network to recognize the type of actions (e.g. in commercial movies or some real-life videos), so I can "ask" my network in which video or movie (and at ...
kenorb's user avatar
  • 10.5k
6 votes
3 answers
2k views

Why are traditional ML models still used over deep neural networks?

I'm still on my first steps in the Data Science field. I played with some DL frameworks, like TensorFlow (pure) and Keras (on top) before, and know a little bit of some "classic machine learning" ...
Douglas Ferreira's user avatar
5 votes
2 answers
434 views

Mathematical foundations of the ability to learn

I am an undergraduate student in applied mathematics with an interest in artificial intelligence. I am currently exploring topics where I could do research. Coming from a mathematical background I am ...
Matheo's user avatar
  • 153
5 votes
1 answer
745 views

How does an unsupervised learning model learn?

How does an unsupervised learning model learn, if it does not involve any target values?
kenorb's user avatar
  • 10.5k
5 votes
1 answer
1k views

Which functions can be activation functions?

What are the required characteristics of an activation function (in a neural network)? Which functions can be activation functions? For example, which of the functions below can be used as an ...
mBabaee's user avatar
  • 51
3 votes
1 answer
1k views

What is teacher forcing?

In the paper Neural Programmer-Interpreters, the authors use the teacher forcing technique, but what exactly is it?
nbro's user avatar
  • 41k
3 votes
1 answer
470 views

How are exploding numbers in a forward pass of a CNN combated?

Take AlexNet for example: In this case, only the activation function ReLU is used. Due to the fact ReLU cannot be saturated, it instead explodes, like in the following example: Say I have a weight ...
Recessive's user avatar
  • 1,406
2 votes
2 answers
197 views

AI with conflicting objectives?

A recent question on AI and acting recalled me to the idea that in drama, there are not only conflicting motives between agents (characters), but a character may themselves have objectives that are in ...
DukeZhou's user avatar
  • 6,237
1 vote
1 answer
395 views

Can I always interpret features as random variables in machine learning safely?

Consider the following statements from Chapter 5: Machine Learning Basics from the book titled Deep Learning (by Aaron Courville et al.) Machine learning tasks are usually described in terms of how ...
hanugm's user avatar
  • 3,930
36 votes
7 answers
8k views

What are examples of promising AI/ML techniques that are computationally intractable?

To produce tangible results in the field of AI/ML, one must take theoretical results under the lens of computational complexity. Indeed, minimax effectively solves any two-person "board game"...
k.c. sayz 'k.c sayz''s user avatar
29 votes
4 answers
79k views

Why does C++ seem less widely used than Python in AI?

I just want to know why do machine learning engineers and AI programmers use languages like Python to perform AI tasks and not C++, even though C++ is technically a more powerful language than Python.
Mark ellon's user avatar
20 votes
2 answers
4k views

What is the "Hello World" problem of Reinforcement Learning?

As we all know, "Hello World" is usually the first program that any programmer learns/implements in any language/framework. As Aurélien Géron mentioned in his book that MNIST is often called ...
Arpit-Gole's user avatar
20 votes
4 answers
14k views

Why do we need floats for using neural networks?

Is it possible to make a neural network that uses only integers by scaling input and output of each function to [-INT_MAX, INT_MAX]? Is there any drawbacks?
elimohl's user avatar
  • 311