For questions related to the gradient, a way of packing together all the partial derivative information of a function

7 questions
Filter by
Sorted by
Tagged with
30 views

### Why don't integrated gradients explain samples correctly?

I have a linear tabular dataset made of floats. The dataset follows a simple rule like: ...
121 views

### Why is it a problem if the outputs of an activation function are not zero-centered?

In this lecture, the professor says that one problem with the sigmoid function is that its outputs aren't zero-centered. Are the explanation provided by the professor regarding why this is bad is that ...
137 views

### How can the sum of squared errors have negative gradient if it's defined as the squared of the error?

The formula for the sum of squared errors (SSE) is: $$\frac{1}{2} \sum_{i=1}^n (t^i - o^i)^2$$ I have a few related questions. If $t^i - o^i$ is negative, doesn't the power of 2 eliminate any ...
69 views

### Why is tf.abs non-differentiable in Tensorflow?

I understand why tf.abs is non-differentiable in principle (discontinuity at 0) but the same applies to tf.nn.relu yet, in case of this function gradient is simply set to 0 at 0. Why the same logic is ...
1k views

### How is the gradient of the loss function in DQN derived?

In the original DQN paper, page 1, the loss function of the DQN is $$L_{i}(\theta_{i}) = \mathbb{E}_{(s,a,r,s') \sim U(D)} [(r+\gamma \max_{a'} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}))^2]$$ ...
### During Backpropagation in LSTM, why is the previous output $h_{t-1}$ considered constant w.r.t any $W$ while computing derivative?
I've just started learning LSTM, and some points in the process of calculating the gradients are getting me confused. Say, for example, we want to compute $\frac{\partial}{\partial W_i}L$, where $L$ ...