6
votes
How is this Pytorch expression equivalent to the KL divergence?
This is the analytical form of the KL divergence between two multivariate Gaussian densities with diagonal covariance matrices (i.e. we assume independence). More precisely, it's the KL divergence ...
- 33.8k
6
votes
Accepted
What exactly is an XPU?
XPU is a device abstraction for Intel heterogeneous computation architectures, which can be mapped to CPU, GPU, FPGA and other accelerators. The "X" from XPU is just like a variable, like in ...
- 1,018
5
votes
Accepted
How is this Pytorch expression equivalent to the KL divergence?
The code is correct. Since OP asked for a proof, one follows.
The usage in the code is straightforward if you observe that the authors are using the symbols unconventionally: ...
- 236
5
votes
How can the input order of pairs into a neural network not matter (i.e. symmetry)?
The problem you're describing is related to (if not a subset of) Shift Invariance. Shift invariance refers to any geometric translation of an input, but concatenation of a pair of tenors in 2 ...
- 3,513
4
votes
Accepted
Is there a reason to use TensorFlow over PyTorch for research purposes?
In the past, I have used TensorFlow (1 and 2), Keras and PyTorch, so I will give an answer based on my experience. Currently, I use TF 2 and Keras (the version shipped with TF 2).
In my (but not only)...
- 33.8k
3
votes
Accepted
What are the differences between TensorFlow and PyTorch?
TensorFlow was developed by Google and is based on Theano (Python library), while Facebook developed PyTorch using the Torch library. Both frames are useful and have a great community behind them. ...
- 175
3
votes
Accepted
Why is the number of output channels 16 in the hidden layer of this CNN?
I understand your question as: "How did the author select the number of neurons in their hidden layer?"
The number of neurons in the hidden layer is how you can control the complexity of the function ...
- 450
3
votes
Accepted
Can TensorFlow, PyTorch, and other mainstream ML frameworks be used for research-grade work in AI?
Your statement that researchers build their network from the ground-up using C++ or some other low level library couldn't be further from the truth.
You could take a look at this analysis showing the ...
- 3,093
3
votes
Accepted
When should you not use the bias in a layer?
The most usual case of bias=False is in layers before/after Batch Normalization with no activators in between. The BatchNorm layer will re-center the data anyway, ...
- 1,813
3
votes
What is the Intermediate (dense) layer in between attention-output and encoder-output dense layers within transformer block in PyTorch implementation?
Feedforward layer is an important part of the transformer architecture.
Transformer architecture, in addition to the self-attention layer, that aggregates ...
- 1,824
3
votes
Accepted
Why do we multipy context_size with embedding_dim? (PyTorch)
An n-gram language model is a language model trained with n context words. This means you're not feeding the model a single word but n. This is why the dimension of the input layer is "...
- 3,513
2
votes
Accepted
Heavy loss and inaccurate answer in pytorch
There are a few things you could do to improve this NN, but are probably worth covering in different questions.
Your main problem though is that you forgot to reset the gradient after each training ...
- 23.9k
2
votes
Accepted
How does the policy gradient's derivative work?
You cannot do this:
$\mathop{\mathbb{E}_\pi }[r(\tau )\bigtriangledown log \pi (\tau )] \\= \mathop{\mathbb{E}_\pi }[r(\tau )] \,\, \mathop{\mathbb{E}_\pi }[\bigtriangledown log \pi (\tau )]$
That ...
- 23.9k
2
votes
Accepted
Training network with 4 GPUs performance is not exactly 4 times over one GPU why?
Your dataset class probably have a lot of preprocessing code. You should use a dataloader. It will prefetch data from the dataset when the GPU is processing. Also, you can process all the data ...
- 1,715
2
votes
Accepted
Should the policy parameters be updated at each time step or at the end of the episode in REINFORCE?
The essence of your observation is that Sutton's version of REINFORCE is taking into consideration all of the trajectory to compute the returns, while in the pytorch version only the future is taken ...
- 141
2
votes
Accepted
Are the training loss and validation loss plotted per sample or per batch?
You want to compute the mean loss over all batches. What you need to do is to divide the sum of batch losses with the number of batches!
In your case:
You have a training set of $21700$ samples and ...
- 3,093
2
votes
Policy Gradient on Tic-Tac-Toe not working
Some suggestions:
You have a loop in which illegal moves by the RL agent are ignored. In other words, when the agent makes illegal moves, it is not punished, nor is there any +/- rewards for it ...
- 235
2
votes
Accepted
Why isn't my implementation of A2C for the the atari pong game converging?
Here is the commit
I fixed few minor errors, but the major one was when I saw what the line histories = [deque(maxlen=self.reward_steps)] * len(self.env.envs) was ...
- 161
2
votes
Advantage computed the wrong way?
Yeah, it seems like it's a wrong implementation.
vals_ref_v is a matrix of 1 row, and 128 columns.
value_v.detach() is a matrix of 128 row
- 33
2
votes
Is it possible to have a fixed trajectory size in the vanilla policy gradient algorithm?
Trajectory size can be fixed, but in this case problem would be formulated as something similar to the multi-armed bandit problem where there is a single state and a set of actions to choose from. ...
- 2,226
2
votes
Accepted
Why is the policy loss the mean of $-Q(s, \mu(s))$ in the DDPG algorithm?
This is not quite the loss that is stated in the paper.
For standard policy gradient methods the objective is to maximise $v_{\pi_\theta}(s_0)$ -- note that this is analogous to minimising $-v_{\pi_\...
- 4,055
2
votes
How to implement or avoid masking for transformer?
If you're using a library such as Trax which contains great submodules for various Transformers (Skipping, BERT, Vanilla and Reformer) you can use the inbuilt ...
- 696
2
votes
Accepted
Is there ever a need to combine deep learning frameworks? (Eg. TensorFlow & Torch)?
In the time since I asked this question, I have been able to combine Tensorflow and Chainer considerably well. That being said, one should try to avoid combining deep learning frameworks if one can ...
- 3,587
2
votes
Accepted
Transformers: How to use the target mask properly?
The main issue during training is that you haven't right-shifted the input of the decoder, which is probably why you set the diagonals of mask to -inf (when it ...
- 1,230
2
votes
Accepted
How to incorporate a symmetry constraint in the loss function to train a CNN?
If you know it is symmetric, then you could do a couple things.
Zero out a half.
Don't bother learning both halves of the image. Just put a zero mask over the upper or lower half of the output ...
- 463
2
votes
Is there any difference between affine transformation and linear transformation?
In linear algebra, a linear transformation (aka linear map or linear transform) $f: \mathcal{V} \rightarrow \mathcal{W}$ is a function that satisfies the following two conditions
$f(u + v)=f(u)+f(v)$ ...
- 33.8k
2
votes
Is there any difference between affine transformation and linear transformation?
The fact is you can always express an affine transformation as a linear transformation (more convenient because it is just a matrix/dot product).
For instance, given an input $\textbf{x}=[x_1, ..., ...
- 21
2
votes
What does it mean by "zeros the networks parameters gradients" in the context of training a neural network?
In the automatic differentiation procedure after backward pass
the gradient with respect to the scalar is added to the current gradient. Without calling zero_grad you will have the sum of all ...
- 1,824
2
votes
What does 'input planes' mean in the phrase 'input signal/image composed of several input planes'?
Yes, it is a bit misleading. What it really means is input channels, so it would be: nn.Conv2d: Applies a 2D convolution over an input signal composed of several input channels.
So, why don't just use ...
- 1,018
2
votes
Accepted
Is "kernel" different from "filter" in convolutional neural networks?
The term "filter" is (usually) a synonym for "kernel" in the context of convolutional neural networks and image processing.
The reason why the ...
- 33.8k
Only top scored, non community-wiki answers of a minimum length are eligible