New answers tagged comparison
1
Your interpretation is definitely correct. As you correctly pointed out, the derivative of softplus is continuous and $n$-times differentiable, that makes the function smooth, which is not the case for ReLU.
What is quite interesting here is why softplus can be called an approximation to ReLU.
If we break down the definition of softplus, we note that the ...
1
I think this is best explained using an analogy. Also you seen to have the misconception that you don't tune hyper-parameters for training data. You want to increase the accuracy of the training set AND validation set at the same time, but the validation set is more important so you want to maximise that accuracy more.
Imagine you had a toddler, and you were ...
0
Idea is to optimize with regards to unseen data in each step in order to avoid overfitting and data leakage so that the final network will be most generalizable to novel data.
First, you initialize your network weights randomly. For those weights, training data is unseen so network is optimized with regards to loss function that is calculated using training ...
-1
You generally split your data into two segements, a training set, and a test set. The test set is used to evaluate the performance of the training, and thus has to be distinct from the training data — the idea is that the data has not been previously seen by the trained system.
As far as I am aware, the test set is only ever used to evaluate the performance ...
2
The original transformer is a feedforward neural network (FFNN)-based architecture that makes use of an attention mechanism. So, this is the difference: an attention mechanism (in particular, a self-attention operation) is used by the transformer, which is not just this attention mechanism, but it's an encoder-decoder architecture, which makes use of other ...
1
If your goal is to create a controller for the mountain car problem, and you have access to the model, then RL probably offers no advantage over your code. I am saying probably, because I am taking you at your word that the code performs well over multiple tests, and it doesn't matter too much if it does not because there are many equivalent solutions based ...
1
In addition to those mentioned differences, a perceptron can be thought of as a standalone model (which is trained with a specific algorithm, the perceptron algorithm), while the artificial neuron (sometimes only referred to as neuron, in a similar way that an artificial neuron network is commonly abbreviated to neural network) is the smallest computational ...
Top 50 recent answers are included
