New answers tagged softmax
0
One Neuron on its own can only solve linearly separable problems.
You need a combination of Neurons to solve non-linearly separable problems.
For the XOR case, you need at least 2 Neuron at the first layer, and 1 Neuron at the Output layer to properly classify it.
Keep in mind sometimes the 3 Neuron network might get stuck in a local minima as well, you will ...
1
Let's first recap the definition of the binary cross-entropy (BCE) and the categorical cross-entropy (CCE).
Here's the BCE (equation 4.90 from this book)
$$-\sum_{n=1}^{N}\left( t_{n} \ln y_{n}+\left(1-t_{n}\right) \ln \left(1-y_{n}\right)\right) \label{1}\tag{1},$$
where
$t_{n} \in\{0,1\}$ is the target
$y_n \in [0, 1]$ is the prediction (as produced by ...
Top 50 recent answers are included
Related Tags
softmax × 15neural-networks × 4
backpropagation × 4
gradient-descent × 4
deep-learning × 3
cross-entropy × 3
categorical-crossentropy × 3
machine-learning × 2
comparison × 2
loss-functions × 2
linear-algebra × 2
sigmoid × 2
reinforcement-learning × 1
convolutional-neural-networks × 1
classification × 1
tensorflow × 1
python × 1
training × 1
papers × 1
math × 1
policy-gradients × 1
transformer × 1
implementation × 1
attention × 1
history × 1