# Tag Info

4

It's an uppercase "J" from the math calligraphy alphabet, i.e. \mathcal{J} in latex. $\mathcal{J}$

1

do we also want to consider the subset of invalid actions for the $\max\limits_{a}Q(s_{t+1},a)$ No. Doing so would go against the theory behind the Bellman equation from which the update derives. The value of $r_{t+1} + \gamma \max\limits_{a'}Q(s_{t+1},a')$ needs to match to a realisable trajectory, otherwise the eventual expected values may be estimates ...

1

Assume we have a binary classification problem, which we want to solve with a simple single-layer perceptron. For a 2d space, a perceptron will have 2 inputs $x_1$ and $x_2$, and a bias denoted $x_0$ which is always $x_0=1$. It also has corresponding learnable weights $w_0$, $w_1$ and $w_2$. This can be vectorized:  \overline{x} = \begin{bmatrix} 1 \\ x_1 \...

1

The loss function is simply a way to measure how wrong a neural network is, it doesn't affect the output of the neuron. Say we have a neural network with 3 output neurons that attempts to classify images of cats, dogs, and humans. The output it gives is the confidence of the neural network's classification. For example if the output is [0, 0.2, 0.8] (0 being ...

1

Loss function is a function used to measure the loss. It is not used in any component of a neuron. It is used in updating the weights of the neuron i.e., in order to train the neuron. The contribution of a loss function is in the updation of $\bar{W}$. For a given $\bar{X}$ and $\bar{W}$, the neuron gives a post-action value $h$. But the desired output may ...

1

I assume you intended to write compute the evaluation metric over the validation set in batches; you do not compute loss over the validation set! That is quite a standard practice in many academic implementations (because, when the validation set is large enough, the memory will be a constraint), however, be sure to take the average of the values over all ...

Only top voted, non community-wiki answers of a minimum length are eligible