43
votes
What is the difference between a convolutional neural network and a regular neural network?
TLDR:
The convolutional-neural-network is a subclass of neural-networks which have at least one convolution layer. They are great for capturing local information (e.g. neighbor pixels in an image or ...
43
votes
Accepted
What is the "temperature" in the GPT models?
In sequence generating models, for vocabulary of size $N$ (number of words, parts of words, any other kind of token), one predicts the next token from distribution of the form:
$$
\mathrm{softmax} (...
- 2,754
39
votes
Accepted
What are "bottlenecks" in neural networks?
The bottleneck in a neural network is just a layer with fewer neurons than the layer below or above it. Having such a layer encourages the network to compress feature representations (of salient ...
- 33.3k
37
votes
Accepted
What is the difference between latent and embedding spaces?
Embedding vs Latent Space
Due to Machine Learning's recent and rapid renaissance, and the fact that it draws from many distinct areas of mathematics, statistics, and computer science, it often has a ...
- 1,031
33
votes
How can an AI train itself if no one is telling it if its answer is correct or wrong?
By "company A has a large human face database so that it can train its facial recognition program more efficiently" the article probably means that there is a training dataset $S$ of the form
$$
S = ...
- 41.4k
32
votes
Accepted
What is the Bellman operator in reinforcement learning?
The notation I'll be using is from two different lectures by David Silver and is also informed by these slides.
The expected Bellman equation is
$$v_\pi(s) = \sum_{a\in \cal{A}} \pi(a|s) \left(\cal{R}...
- 2,064
20
votes
What is the difference between an observation and a state in reinforcement learning?
Sometimes observation and state overlap completely, which is convenient. However, there is no reason to expect it in all cases, and that's where interesting problems occur.
Reinforcement learning ...
- 33.3k
19
votes
What is the difference between a convolutional neural network and a regular neural network?
Convolutional Neural Networks (CNNs) are neural networks with architectural constraints to reduce computational complexity and ensure translational invariance (the network interprets input patterns ...
- 353
18
votes
Accepted
What is the difference between actor-critic and advantage actor-critic?
Actor-Critic is not just a single algorithm, it should be viewed as a "family" of related techniques. They're all techniques based on the policy gradient theorem, which train some form of ...
- 10.5k
17
votes
What is the difference between artificial intelligence and machine learning?
Artificial intelligence
According to the book Artificial Intelligence: A Modern Approach (section 1.1), artificial intelligence (AI) has been defined in multiple ways, which can be organized into 4 ...
15
votes
Accepted
What is the difference between active learning and online learning?
Active learning (AL) is a weakly supervised learning (WSL) technique where you can have both labelled and unlabelled data [1]. The main idea behind AL is that the learner (or learning algorithm) can ...
- 41.4k
15
votes
Accepted
Which tasks are called as downstream tasks?
In the context of self-supervised learning (which is also used in NLP), a downstream task is the task that you actually want to solve. This definition makes sense if you're familiar with transfer ...
- 41.4k
14
votes
Accepted
What is a "trajectory" in reinforcement learning?
In answer that you linked, I may have used an informal definition of "trajectory", but essentially the same thing as the quote. A "trajectory" is the sequence of what has happened (in terms of state, ...
- 33.3k
14
votes
Accepted
What algorithms are considered reinforcement learning algorithms?
The dynamic programming (DP) algorithms like policy iteration (PI) and value iteration (VI) are often presented in the context of reinforcement learning (in particular, in the book Reinforcement ...
- 41.4k
13
votes
Accepted
What is "planning" in the context of reinforcement learning, and how is it different from RL and SL?
The concept of "planning" is not just related to RL. In general (as the name suggests), planning consists in creating a "plan" which you will use to reach a "goal". The ...
- 41.4k
12
votes
What is the definition of "soft label" and "hard label"?
According to Galstyan and Cohen (2007), a hard label is a label assigned to a member of a class where membership is binary: either the element in question is a member of the class (has the label), or ...
- 5,417
12
votes
Accepted
What are options in reinforcement learning?
An option is a generalization of the concept of action. The concept of an option (or macro-action) was introduced in the context of reinforcement learning in the paper Between MDPs and semi-MDPs: A ...
- 41.4k
11
votes
What is the difference between artificial intelligence and machine learning?
Machine learning is a subfield of artificial intelligence, as the following diagram (taken from this blog post) illustrates.
11
votes
What are "bottlenecks" in neural networks?
Imagine, you want to re-compute the last layer of a pre-trained model :
Input->[Freezed-Layers]->[Last-Layer-To-Re-Compute]->Output
To train [Last-...
- 211
11
votes
What is the purpose of an activation function in neural networks?
If you only had linear layers in a neural network, all the layers would essentially collapse to one linear layer, and, therefore, a "deep" neural network architecture effectively wouldn't be deep ...
11
votes
What is the difference between a receptive field and a feature map?
Feature map
A feature map is the output of the convolution operation (performed by a convolutional layer).
When you apply a kernel $K_1$ to the input image $I$ (i.e. you perform the convolution ...
10
votes
What are bottleneck features?
In the blog post Building powerful image classification models using very little data, bottleneck features are mentioned. What are the bottleneck features?
It's clearly written in the link you gave ...
- 341
10
votes
Accepted
What does "stationary" mean in the context of reinforcement learning?
A stationary policy is a policy that does not change. Although strictly that is a time-dependent issue, that is not what the distinction refers to in reinforcement learning. It generally means that ...
- 33.3k
10
votes
Accepted
What is the difference between reinforcement learning and optimal control?
The same book Reinforcement learning: an introduction (2nd edition, 2018) by Sutton and Barto has a section, 1.7 Early History of Reinforcement Learning, that describes what optimal control is and how ...
- 41.4k
10
votes
Accepted
Is LSTM a subcategory of RNN?
The Wikipedia article is more technically correct, in that the term RNN is formally taken to mean "a neural network with recurrent connections", and that includes many architectures that ...
- 33.3k
9
votes
What is an activity vector in capsule neural networks?
In a traditional neural network, the network's vertices are neurons and the output of a single neuron is a single value (a "scalar"). This number is called its activation. A layer of neurons ...
- 426
Only top scored, non community-wiki answers of a minimum length are eligible