Hot answers tagged

6 votes

How is this Pytorch expression equivalent to the KL divergence?

This is the analytical form of the KL divergence between two multivariate Gaussian densities with diagonal covariance matrices (i.e. we assume independence). More precisely, it's the KL divergence ...
user avatar
  • 33.8k
6 votes
Accepted

What exactly is an XPU?

XPU is a device abstraction for Intel heterogeneous computation architectures, which can be mapped to CPU, GPU, FPGA and other accelerators. The "X" from XPU is just like a variable, like in ...
user avatar
  • 1,018
5 votes
Accepted

How is this Pytorch expression equivalent to the KL divergence?

The code is correct. Since OP asked for a proof, one follows. The usage in the code is straightforward if you observe that the authors are using the symbols unconventionally: ...
user avatar
  • 236
5 votes

How can the input order of pairs into a neural network not matter (i.e. symmetry)?

The problem you're describing is related to (if not a subset of) Shift Invariance. Shift invariance refers to any geometric translation of an input, but concatenation of a pair of tenors in 2 ...
user avatar
4 votes
Accepted

Is there a reason to use TensorFlow over PyTorch for research purposes?

In the past, I have used TensorFlow (1 and 2), Keras and PyTorch, so I will give an answer based on my experience. Currently, I use TF 2 and Keras (the version shipped with TF 2). In my (but not only)...
user avatar
  • 33.8k
3 votes
Accepted

What are the differences between TensorFlow and PyTorch?

TensorFlow was developed by Google and is based on Theano (Python library), while Facebook developed PyTorch using the Torch library. Both frames are useful and have a great community behind them. ...
user avatar
3 votes
Accepted

Why is the number of output channels 16 in the hidden layer of this CNN?

I understand your question as: "How did the author select the number of neurons in their hidden layer?" The number of neurons in the hidden layer is how you can control the complexity of the function ...
user avatar
  • 450
3 votes
Accepted

Can TensorFlow, PyTorch, and other mainstream ML frameworks be used for research-grade work in AI?

Your statement that researchers build their network from the ground-up using C++ or some other low level library couldn't be further from the truth. You could take a look at this analysis showing the ...
user avatar
  • 3,093
3 votes
Accepted

When should you not use the bias in a layer?

The most usual case of bias=False is in layers before/after Batch Normalization with no activators in between. The BatchNorm layer will re-center the data anyway, ...
user avatar
  • 1,813
3 votes

What is the Intermediate (dense) layer in between attention-output and encoder-output dense layers within transformer block in PyTorch implementation?

Feedforward layer is an important part of the transformer architecture. Transformer architecture, in addition to the self-attention layer, that aggregates ...
user avatar
3 votes
Accepted

Why do we multipy context_size with embedding_dim? (PyTorch)

An n-gram language model is a language model trained with n context words. This means you're not feeding the model a single word but n. This is why the dimension of the input layer is "...
user avatar
2 votes
Accepted

Heavy loss and inaccurate answer in pytorch

There are a few things you could do to improve this NN, but are probably worth covering in different questions. Your main problem though is that you forgot to reset the gradient after each training ...
user avatar
  • 23.9k
2 votes
Accepted

How does the policy gradient's derivative work?

You cannot do this: $\mathop{\mathbb{E}_\pi }[r(\tau )\bigtriangledown log \pi (\tau )] \\= \mathop{\mathbb{E}_\pi }[r(\tau )] \,\, \mathop{\mathbb{E}_\pi }[\bigtriangledown log \pi (\tau )]$ That ...
user avatar
  • 23.9k
2 votes
Accepted

Training network with 4 GPUs performance is not exactly 4 times over one GPU why?

Your dataset class probably have a lot of preprocessing code. You should use a dataloader. It will prefetch data from the dataset when the GPU is processing. Also, you can process all the data ...
user avatar
  • 1,715
2 votes
Accepted

Should the policy parameters be updated at each time step or at the end of the episode in REINFORCE?

The essence of your observation is that Sutton's version of REINFORCE is taking into consideration all of the trajectory to compute the returns, while in the pytorch version only the future is taken ...
user avatar
2 votes
Accepted

Are the training loss and validation loss plotted per sample or per batch?

You want to compute the mean loss over all batches. What you need to do is to divide the sum of batch losses with the number of batches! In your case: You have a training set of $21700$ samples and ...
user avatar
  • 3,093
2 votes

Policy Gradient on Tic-Tac-Toe not working

Some suggestions: You have a loop in which illegal moves by the RL agent are ignored. In other words, when the agent makes illegal moves, it is not punished, nor is there any +/- rewards for it ...
user avatar
2 votes
Accepted

Why isn't my implementation of A2C for the the atari pong game converging?

Here is the commit I fixed few minor errors, but the major one was when I saw what the line histories = [deque(maxlen=self.reward_steps)] * len(self.env.envs) was ...
user avatar
  • 161
2 votes

Advantage computed the wrong way?

Yeah, it seems like it's a wrong implementation. vals_ref_v is a matrix of 1 row, and 128 columns. value_v.detach() is a matrix of 128 row
user avatar
  • 33
2 votes

Is it possible to have a fixed trajectory size in the vanilla policy gradient algorithm?

Trajectory size can be fixed, but in this case problem would be formulated as something similar to the multi-armed bandit problem where there is a single state and a set of actions to choose from. ...
user avatar
  • 2,226
2 votes
Accepted

Why is the policy loss the mean of $-Q(s, \mu(s))$ in the DDPG algorithm?

This is not quite the loss that is stated in the paper. For standard policy gradient methods the objective is to maximise $v_{\pi_\theta}(s_0)$ -- note that this is analogous to minimising $-v_{\pi_\...
user avatar
2 votes

How to implement or avoid masking for transformer?

If you're using a library such as Trax which contains great submodules for various Transformers (Skipping, BERT, Vanilla and Reformer) you can use the inbuilt ...
user avatar
2 votes
Accepted

Is there ever a need to combine deep learning frameworks? (Eg. TensorFlow & Torch)?

In the time since I asked this question, I have been able to combine Tensorflow and Chainer considerably well. That being said, one should try to avoid combining deep learning frameworks if one can ...
user avatar
2 votes
Accepted

Transformers: How to use the target mask properly?

The main issue during training is that you haven't right-shifted the input of the decoder, which is probably why you set the diagonals of mask to -inf (when it ...
user avatar
  • 1,230
2 votes
Accepted

How to incorporate a symmetry constraint in the loss function to train a CNN?

If you know it is symmetric, then you could do a couple things. Zero out a half. Don't bother learning both halves of the image. Just put a zero mask over the upper or lower half of the output ...
user avatar
2 votes

Is there any difference between affine transformation and linear transformation?

In linear algebra, a linear transformation (aka linear map or linear transform) $f: \mathcal{V} \rightarrow \mathcal{W}$ is a function that satisfies the following two conditions $f(u + v)=f(u)+f(v)$ ...
user avatar
  • 33.8k
2 votes

Is there any difference between affine transformation and linear transformation?

The fact is you can always express an affine transformation as a linear transformation (more convenient because it is just a matrix/dot product). For instance, given an input $\textbf{x}=[x_1, ..., ...
user avatar
2 votes

What does it mean by "zeros the networks parameters gradients" in the context of training a neural network?

In the automatic differentiation procedure after backward pass the gradient with respect to the scalar is added to the current gradient. Without calling zero_grad you will have the sum of all ...
user avatar
2 votes

What does 'input planes' mean in the phrase 'input signal/image composed of several input planes'?

Yes, it is a bit misleading. What it really means is input channels, so it would be: nn.Conv2d: Applies a 2D convolution over an input signal composed of several input channels. So, why don't just use ...
user avatar
  • 1,018
2 votes
Accepted

Is "kernel" different from "filter" in convolutional neural networks?

The term "filter" is (usually) a synonym for "kernel" in the context of convolutional neural networks and image processing. The reason why the ...
user avatar
  • 33.8k

Only top scored, non community-wiki answers of a minimum length are eligible