Tag Info

How is this Pytorch expression equivalent to the KL divergence?

This is the analytical form of the KL divergence between two multivariate Gaussian densities with diagonal covariance matrices (i.e. we assume independence). More precisely, it's the KL divergence ...
• 33.8k
Accepted

What exactly is an XPU?

XPU is a device abstraction for Intel heterogeneous computation architectures, which can be mapped to CPU, GPU, FPGA and other accelerators. The "X" from XPU is just like a variable, like in ...
• 1,018
Accepted

How is this Pytorch expression equivalent to the KL divergence?

The code is correct. Since OP asked for a proof, one follows. The usage in the code is straightforward if you observe that the authors are using the symbols unconventionally: ...
• 236

How can the input order of pairs into a neural network not matter (i.e. symmetry)?

The problem you're describing is related to (if not a subset of) Shift Invariance. Shift invariance refers to any geometric translation of an input, but concatenation of a pair of tenors in 2 ...
• 3,513
Accepted

Is there a reason to use TensorFlow over PyTorch for research purposes?

In the past, I have used TensorFlow (1 and 2), Keras and PyTorch, so I will give an answer based on my experience. Currently, I use TF 2 and Keras (the version shipped with TF 2). In my (but not only)...
• 33.8k
Accepted

What are the differences between TensorFlow and PyTorch?

TensorFlow was developed by Google and is based on Theano (Python library), while Facebook developed PyTorch using the Torch library. Both frames are useful and have a great community behind them. ...
• 175
Accepted

Why is the number of output channels 16 in the hidden layer of this CNN?

I understand your question as: "How did the author select the number of neurons in their hidden layer?" The number of neurons in the hidden layer is how you can control the complexity of the function ...
• 450
Accepted

Can TensorFlow, PyTorch, and other mainstream ML frameworks be used for research-grade work in AI?

Your statement that researchers build their network from the ground-up using C++ or some other low level library couldn't be further from the truth. You could take a look at this analysis showing the ...
• 3,093
Accepted

When should you not use the bias in a layer?

The most usual case of bias=False is in layers before/after Batch Normalization with no activators in between. The BatchNorm layer will re-center the data anyway, ...
• 1,813

What is the Intermediate (dense) layer in between attention-output and encoder-output dense layers within transformer block in PyTorch implementation?

Feedforward layer is an important part of the transformer architecture. Transformer architecture, in addition to the self-attention layer, that aggregates ...
Accepted

Why do we multipy context_size with embedding_dim? (PyTorch)

An n-gram language model is a language model trained with n context words. This means you're not feeding the model a single word but n. This is why the dimension of the input layer is "...
• 3,513
Accepted

Heavy loss and inaccurate answer in pytorch

There are a few things you could do to improve this NN, but are probably worth covering in different questions. Your main problem though is that you forgot to reset the gradient after each training ...
• 23.9k
Accepted

How does the policy gradient's derivative work?

You cannot do this: $\mathop{\mathbb{E}_\pi }[r(\tau )\bigtriangledown log \pi (\tau )] \\= \mathop{\mathbb{E}_\pi }[r(\tau )] \,\, \mathop{\mathbb{E}_\pi }[\bigtriangledown log \pi (\tau )]$ That ...
• 23.9k
Accepted

Training network with 4 GPUs performance is not exactly 4 times over one GPU why?

Your dataset class probably have a lot of preprocessing code. You should use a dataloader. It will prefetch data from the dataset when the GPU is processing. Also, you can process all the data ...
• 1,715
Accepted

Should the policy parameters be updated at each time step or at the end of the episode in REINFORCE?

The essence of your observation is that Sutton's version of REINFORCE is taking into consideration all of the trajectory to compute the returns, while in the pytorch version only the future is taken ...
Accepted

Are the training loss and validation loss plotted per sample or per batch?

You want to compute the mean loss over all batches. What you need to do is to divide the sum of batch losses with the number of batches! In your case: You have a training set of $21700$ samples and ...
• 3,093

Policy Gradient on Tic-Tac-Toe not working

Some suggestions: You have a loop in which illegal moves by the RL agent are ignored. In other words, when the agent makes illegal moves, it is not punished, nor is there any +/- rewards for it ...
• 235
Accepted

Why isn't my implementation of A2C for the the atari pong game converging?

Here is the commit I fixed few minor errors, but the major one was when I saw what the line histories = [deque(maxlen=self.reward_steps)] * len(self.env.envs) was ...
• 161

Yeah, it seems like it's a wrong implementation. vals_ref_v is a matrix of 1 row, and 128 columns. value_v.detach() is a matrix of 128 row
• 33

Is it possible to have a fixed trajectory size in the vanilla policy gradient algorithm?

Trajectory size can be fixed, but in this case problem would be formulated as something similar to the multi-armed bandit problem where there is a single state and a set of actions to choose from. ...
• 2,226
Accepted

What does it mean by "zeros the networks parameters gradients" in the context of training a neural network?

In the automatic differentiation procedure after backward pass the gradient with respect to the scalar is added to the current gradient. Without calling zero_grad you will have the sum of all ...

What does 'input planes' mean in the phrase 'input signal/image composed of several input planes'?

Yes, it is a bit misleading. What it really means is input channels, so it would be: nn.Conv2d: Applies a 2D convolution over an input signal composed of several input channels. So, why don't just use ...
• 1,018