Questions tagged [machine-learning]

For questions related to machine learning (ML), which is a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data). ML is usually divided into supervised, unsupervised and reinforcement learning. Deep learning is a subfield of ML that uses deep artificial neural networks.

Filter by
Sorted by
Tagged with
78
votes
17answers
12k views

What is the difference between artificial intelligence and machine learning?

These two terms seem to be related, especially in their application in computer science and software engineering. Is one a subset of another? Is one a tool used to build a system for the other? ...
49
votes
8answers
28k views

Why is Python such a popular language in the AI field?

First of all, I'm a beginner studying AI and this is not an opinion oriented question or one to compare programming languages. I'm not implying that Python is the best language. But the fact is that ...
47
votes
3answers
30k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
38
votes
4answers
10k views

Are neural networks prone to catastrophic forgetting?

Imagine you show a neural network a picture of a lion 100 times and label with "dangerous", so it learns that lions are dangerous. Now imagine that previously you have shown it millions of images of ...
30
votes
3answers
18k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, with ...
28
votes
9answers
5k views

Is artificial intelligence vulnerable to hacking?

The paper The Limitations of Deep Learning in Adversarial Settings explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. ...
28
votes
4answers
42k views

How to select number of hidden layers and number of memory cells in an LSTM?

I am trying to find some existing research on how to select the number of hidden layers and the size of these of an LSTM-based RNN. Is there an article where this problem is being investigated, i.e., ...
27
votes
7answers
7k views

Computationally expensive AI techniques (that are promising)

To produce tangible results in the field of ML/AI, one must take theoretical results under the lens of computational complexity. Indeed, minimax effectively solves any two-person "board game" with ...
25
votes
10answers
13k views

How can an AI train itself if no one is telling it if its answer is correct or wrong?

I am a programmer but not in the field of AI. A question constantly confuses me is that how can an AI be trained if we human beings are not telling it its calculation is correct? For example, news ...
25
votes
2answers
941 views

How is a deep neural network different from other neural networks?

How is a neural network having the "deep" adjective actually distinguished from other similar networks?
23
votes
2answers
8k views

Is it possible to train a neural network incrementally?

I would like to train a neural network where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every time ...
23
votes
2answers
899 views

Are there any ongoing projects which use the Stack Exchange for machine learning?

Are there any ongoing AI projects which use the Stack Exchange for machine learning?
20
votes
4answers
6k views

How to handle invalid moves in reinforcement learning?

I want to create an AI which can play five-in-a-row/gomoku. As I mentioned in the title, I want to use reinforcement learning for this. I use policy gradient method, namely REINFORCE, with baseline. ...
18
votes
7answers
2k views

If digital values are mere estimates, why not return to analog for AI?

The impetus behind the twentieth century transition from analog to digital circuitry was driven by the desire for greater accuracy and lower noise. Now we are developing software where results are ...
16
votes
3answers
22k views

Understanding GAN loss function

I'm struggling to understand the GAN loss function as provided in Understanding Generative Adversarial Networks (a blog post written by Daniel Seita). In the standard cross-entropy loss, we have an ...
16
votes
1answer
372 views

Differences between backpropagation techniques

Just for fun, I am trying to develop a neural network. Now, for backpropagation I saw two techniques. The first one is used here and in many other places too. What it does is: It computes the ...
15
votes
1answer
322 views

Are search engines considered AI?

Are search engines considered AI because of the way they analyze what you search for and remember it? Or how they send you ads of what you've searched for recently? Is this considered AI or just ...
15
votes
2answers
3k views

Why has cross entropy become the classification standard loss function and not Kullbeck Leibler divergence?

Cross entropy is identical to the KL divergence plus entropy of target distribution. KL equals to zero when the two distributions are the same, which seems more intuitive to me than the entropy of the ...
15
votes
2answers
9k views

What is the time complexity for training a neural network using back-propagation?

Suppose that a NN contains $n$ hidden layers, $m$ training examples, $x$ features, and $n_i$ nodes in each layer. What is the time complexity to train this NN using back-propagation? I have a basic ...
14
votes
5answers
2k views

What is the difference between machine learning and deep learning?

Can someone explain to me the difference between machine learning and deep learning? Is it possible to learn deep learning without knowing machine learning?
14
votes
2answers
929 views

When is deep learning overkill?

For example, for classifying emails as spam, is it worthwhile - from a time/accuracy perspective - to apply deep learning (if possible) instead of another machine learning algorithm? Will deep ...
14
votes
3answers
12k views

What is self-supervised learning in machine learning?

What is self-supervised learning in machine learning? How is it different from supervised learning?
14
votes
3answers
2k views

Permutation invariant neural networks

Given a neural network $f$ that takes as input $n$ data points: $x_1, \dots, x_n$. We say $f$ is permutation invariant if $$f(x_1 ... x_n) = f(pi(x_1 ... x_n))$$ for any permutation $pi$. ...
13
votes
5answers
22k views

Why does C++ seem less widely used in AI?

I just want to know why do Machine Learning engineers and AI programmers use languages like python to perform AI task and not C++ even though C++ is technically a more powerful language than python.
13
votes
5answers
14k views

Can a neural network be used to predict the next pseudo random number?

Is it possible to feed a neural network the output from a random number generator and expect it learn the hashing (or generator) function, so that it can predict what will be the next generated pseudo-...
13
votes
4answers
2k views

What activation function does the human brain use?

Does the human brain use a specific activation function? I've tried doing some research, and as it's a treshold for whether the signal is sent through a neuron or not, it sounds a lot like ReLU. ...
13
votes
1answer
861 views

Are information processing rules from Gestalt psychology still used in computer vision today?

Decades ago there were and are books in machine vision, which by implementing various information processing rules from gestalt psychology, got impressive results with little code or special hardware ...
12
votes
2answers
1k views

How do generative adversarial networks work?

I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative ($...
12
votes
2answers
598 views

Should deep residual networks be viewed as an ensemble of networks?

The question is about the architecture of Deep Residual Networks (ResNets). The model that won the 1-st places at "Large Scale Visual Recognition Challenge 2015" (ILSVRC2015) in all five main tracks: ...
12
votes
4answers
245 views

What are some tactics for recognizing artificially made media?

With the growing ability to cheaply create fake pictures, fake soundbites, and fake video there becomes an increasing problem with recognizing what is real and what isn't. Even now we see a number of ...
11
votes
3answers
404 views

How would an AI learn language?

I was think about AIs and how they would work, when I realised that I couldn't think of a way that an AI could be taught language. A child tends to learn language through associations of language and ...
11
votes
2answers
9k views

What are bottleneck features?

In the blog post Building powerful image classification models using very little data, bottleneck features are mentioned. What are the bottleneck features? Do they change with the architecture that is ...
11
votes
1answer
268 views

How to stay a up-to-date researcher in ML/RL community?

As a student who wants to work on machine learning, I would like to know how it is possible to start my studies and how to follow it to stay up-to-date. For example, I am willing to work on RL and MAB ...
11
votes
3answers
373 views

What are the main problems hindering current AI development?

I have a background in Computer Engineering and have been working on developing better algorithms to mimic human thought. (One of my favorites is Analogical Modeling as applied to language processing ...
11
votes
1answer
244 views

What are all the different kinds of neural networks used for?

I found the following neural network cheat sheet (Cheat Sheets for AI, Neural Networks, Machine Learning, Deep Learning & Big Data). What are all these different kinds of neural networks used ...
11
votes
2answers
4k views

How to choose an activation function?

I choose the activation function for the output layer depending on the output that I need and the properties of the activation function that I know. For example, I choose the sigmoid function when I'm ...
11
votes
4answers
1k views

What are the latest methods to train a chat bot?

I would like to train a bot that uses text input, memorizes a few categories and answers questions accordingly. In addition as version 2.0, I want to make the bot to answer voice inputs as well. Which ...
10
votes
3answers
513 views

How does noise affect generalization?

Does increasing the noise in data help to improve the learning ability of a network? Does it make any difference or does it depend on the problem being solved? How is it affect the generalization ...
10
votes
4answers
235 views

What are the domains where SVMs are still state-of-the-art?

It seems that deep neural networks and other neural network based models are dominating many current areas like computer vision, object classification, reinforcement learning, etc. Are there domains ...
10
votes
2answers
135 views

What does it mean for a neuron in a neural network to be activated?

I just stumbled upon the concept of neuron coverage, which is the ratio of activated neurons and total neurons in a neural network. But what does it mean for a neuron to be "activated"? I know what ...
10
votes
1answer
2k views

Why do you not see dropout layers on reinforcement learning examples?

I've been looking at reinforcement learning, and specifically playing around with creating my own environments to use with the OpenAI Gym AI. I am using agents from the stable_baselines project to ...
10
votes
1answer
3k views

Loss jumps abruptly when I decay the learning rate with Adam optimizer in PyTorch

I'm training an auto-encoder network with Adam optimizer (with amsgrad=True) and ...
10
votes
3answers
11k views

Measuring Object size using Deep Neural Network

I have a large dataset of vehicles with the ground truth of their lengths (Over 100k samples). Is it possible to train a deep network to measure/estimate vehicle length ? I haven't seen any papers ...
10
votes
1answer
264 views

What happens when I mix activation functions?

There are several activation functions, such as ReLU, sigmoid or $\tanh$. What happens when I mix activation functions? I recently found that Google has developed Swish activation function which is (...
10
votes
1answer
348 views

AI that can generate programs

I have been looking into Viv an artificial intelligent agent in development. Based on what I understand, this AI can generate new code and execute it based on a query from the user. What I am curious ...
10
votes
3answers
1k views

Using neural network to recognise patterns in matrices

I am trying to develop a neural network which can identify design features in CAD models (i.e. slots, bosses, holes, pockets, steps). The input data I intend to use for the network is a n x n matrix (...
10
votes
4answers
194 views

How do I select the relevant features of the data?

Recently I was working on a problem to do some cost analysis of my expenditure for some particular resource. I usually make some manual decisions from the analysis and plan accordingly. I have a big ...
9
votes
6answers
948 views

What do I need to study for machine learning?

Starting from last year, I have been studying various subjects in order to understand some of the most important thesis of machine learning like S. Hochreiter, & J. Schmidhuber. (1997). Long ...
9
votes
3answers
1k views

What are the purposes of autoencoders?

Autoencoders are neural networks that learn a compressed representation of the input in order to later reconstruct it, so they can be used for dimensionality reduction. They are composed of an encoder ...
9
votes
2answers
459 views

Does Monte Carlo Search (specifically used by AlphaZero) Qualify as Machine Learning?

To the best of my understanding, Monte Carlo Search is an alternative method to Minimax for searching a tree of nodes. It works by choosing a move (generally the one with the highest chance of being ...