6
This is the analytical form of the KL divergence between two multivariate Gaussian densities with diagonal covariance matrices (i.e. we assume independence). More precisely, it's the KL divergence between the variational distribution
$$
q_{\boldsymbol{\phi}}(\mathbf{z})
=
\mathcal{N}\left(\mathbf{z} ; \boldsymbol{\mu}, \mathbf{\Sigma} = \boldsymbol{\sigma}^...
5
The code is correct. Since OP asked for a proof, one follows.
The usage in the code is straightforward if you observe that the authors are using the symbols unconventionally: sigma is the natural logarithm of the variance, where usually a normal distribution is characterized in terms of a mean $\mu$ and variance. Some of the functions in OP's link even have ...
5
XPU is a device abstraction for Intel heterogeneous computation architectures, which can be mapped to CPU, GPU, FPGA and other accelerators. The "X" from XPU is just like a variable, like in maths, so you can do X=C and you get CPU accceleration, or X=G and you get GPU acceleration... That's the intuition behind that abstract name.
In order to ...
4
In the past, I have used TensorFlow (1 and 2), Keras and PyTorch, so I will give an answer based on my experience. Currently, I use TF 2 and Keras (the version shipped with TF 2).
In my (but not only) opinion, TF 1 is really ugly and painful, given that it involves sessions, placeholders and, in general, you need to define the computational graph before ...
3
TensorFlow was developed by Google and is based on Theano (Python library), while Facebook developed PyTorch using the Torch library. Both frames are useful and have a great community behind them. Both provide machine learning libraries to accomplish various tasks and do the job. TensorFlow is a powerful and deep learning tool with active visualization and ...
3
I understand your question as: "How did the author select the number of neurons in their hidden layer?"
The number of neurons in the hidden layer is how you can control the complexity of the function you are trying to generate to map the inputs to an output. The more neurons in the hidden layer the more complex the function thus you can capture more ...
3
Your statement that researchers build their network from the ground-up using C++ or some other low level library couldn't be further from the truth.
You could take a look at this analysis showing the popularity of these two frameworks in the top ML conferences. The following Figure is taken from there.
In CVPR-2020, for example, TensorFlow and pytorch ...
3
The most usual case of bias=False is in layers before/after Batch Normalization with no activators in between. The BatchNorm layer will re-center the data anyway, removing the bias and making it a useless trainable parameter. Quoting the original BatchNorm paper:
Note that, since we normalize $Wu+b$, the bias $b$ can be ignored since its effect will be ...
3
An n-gram language model is a language model trained with n context words. This means you're not feeding the model a single word but n. This is why the dimension of the input layer is "context_size * embedding_dim" or "n * embedding_dims"
2
There are a few things you could do to improve this NN, but are probably worth covering in different questions.
Your main problem though is that you forgot to reset the gradient after each training batch. You need to call optim.zero_grad() in order to do this, at the start of each training loop. Otherwise, using PyTorch, the gradient values keep ...
2
You cannot do this:
$\mathop{\mathbb{E}_\pi }[r(\tau )\bigtriangledown log \pi (\tau )] \\= \mathop{\mathbb{E}_\pi }[r(\tau )] \,\, \mathop{\mathbb{E}_\pi }[\bigtriangledown log \pi (\tau )]$
That is because $r(\tau )$ and $\bigtriangledown log \pi (\tau )$ are correlated by their dependence on $\tau$. In a simpler concrete example, if your expectation ...
2
Your dataset class probably have a lot of preprocessing code. You should use a dataloader. It will prefetch data from the dataset when the GPU is processing. Also, you can process all the data beforehand and save to a file. Multiple GPU cannot scale as the GPU have to get all data to one GPU to calculate the loss. The performance of 4 GPU is around 3.5x. A ...
2
You want to compute the mean loss over all batches. What you need to do is to divide the sum of batch losses with the number of batches!
In your case:
You have a training set of $21700$ samples and a batch size of $500$. This means that you take $21700/500 \approx 43$ training iterations. This means that for each epoch the model is updated $43$ times! So ...
2
Some suggestions:
You have a loop in which illegal moves by the RL agent are ignored. In other words, when the agent makes illegal moves, it is not punished, nor is there any +/- rewards for it whatsoever. In my program I treat illegal moves the same as losing the game.
Try to play a few "pre-moves" to make the game easier. For example I start ...
2
Here is the commit
I fixed few minor errors, but the major one was when I saw what the line histories = [deque(maxlen=self.reward_steps)] * len(self.env.envs) was doing. It was just repeating the same queue.
In [2]: histories = [deque(maxlen=5)] * 4
In [3]: histories ...
2
Yeah, it seems like it's a wrong implementation.
vals_ref_v is a matrix of 1 row, and 128 columns.
value_v.detach() is a matrix of 128 row
2
In the time since I asked this question, I have been able to combine Tensorflow and Chainer considerably well. That being said, one should try to avoid combining deep learning frameworks if one can for a few reasons:
It doubles the amount of documentation one needs to reference
It makes it difficult for new developers to become familiar with the
code base. ...
2
Trajectory size can be fixed, but in this case problem would be formulated as something similar to the multi-armed bandit problem where there is a single state and a set of actions to choose from. There is no sequential decision making since samples are not correlated, they are picked at random. So, if you take a batch of 20 examples then you would basically ...
2
This is not quite the loss that is stated in the paper.
For standard policy gradient methods the objective is to maximise $v_{\pi_\theta}(s_0)$ -- note that this is analogous to minimising $-v_{\pi_\theta}(s_0)$. This is for a stochastic policy. In DDPG the policy is now assumed to be deterministic.
In general, we can write
$$v_\pi(s) = \mathbb{E}_{a\sim\pi}[...
2
The main issue during training is that you haven't right-shifted the input of the decoder, which is probably why you set the diagonals of mask to -inf (when it should be $0$).
Also, just an FYI, although you haven't focused on evaluation/prediction yet, I will explain the evaluation/prediction here as well for completeness, since it works so differently than ...
2
If you know it is symmetric, then you could do a couple things.
Zero out a half.
Don't bother learning both halves of the image. Just put a zero mask over the upper or lower half of the output matrix and just have the network regress the other half. Just don't make the network do more work than it needs to do.
Learn both, but add symmetric loss
In your ...
2
In linear algebra, a linear transformation (aka linear map or linear transform) $f: \mathcal{V} \rightarrow \mathcal{W}$ is a function that satisfies the following two conditions
$f(u + v)=f(u)+f(v)$ (additivity)
$f(\alpha u) = \alpha f(u)$ (scalar multiplication),
where
$u$ and $v$ vectors (i.e. elements of a vector space, which can also be $\mathbb{R}$ [...
2
The fact is you can always express an affine transformation as a linear transformation (more convenient because it is just a matrix/dot product).
For instance, given an input $\textbf{x}=[x_1, ..., x_n]$, some weights $\textbf{a} = [a_1, a_2, ..., a_n]$ and a bias $b \in \mathbb{R}$, you can express the affine operation $y = \textbf{a}\cdot \textbf{x} + b$ ...
2
In the automatic differentiation procedure after backward pass
the gradient with respect to the scalar is added to the current gradient. Without calling zero_grad you will have the sum of all gradients, calcluated before, with the current gradient.
Therefore, optimizer.step() will do not this:
w = w - eta * grad L[i] # L[i] - loss function for the i-th ...
2
Yes, it is a bit misleading. What it really means is input channels, so it would be: nn.Conv2d: Applies a 2D convolution over an input signal composed of several input channels.
So, why don't just use channels instead of input planes? Well, initially the major deep learning applications were used for computer vision or image processing approaches. In CV or ...
2
The term "filter" is (usually) a synonym for "kernel" in the context of convolutional neural networks and image processing.
The reason why the kernel_size is specified as $3 \times 3$ and then you see that the actual size of the kernel (aka filter) is 3d is that the depth of the kernel can be automatically inferred from in_channels, the ...
2
This one is a bit crazy:
pool1 = nn.AvgPool3d(kernel_size = (361, 1, 1), stride= 1)
because it averages large numbers of the features at once. Very little information about individual features will remain after doing that.
The most obvious one you have not tried is this:
pool3 = nn.AvgPool3d(kernel_size = (3, 1, 1), stride= (3, 1, 1))
which includes all ...
2
The size of the parameters tensor is depended on what type of layer that you want to build. Convolutional, fully connected, attention or even custom layer, each layer has a difference in the way it treats input, reading the documents is the good way to start (CS231n of Stanford University describes in detail each layer's properties).
In your case, the layer ...
2
How about dividing the problem?
You can first train a classification model that predicts the type of function (linear or exponential).
Then you can use your seperately trained nn depending on the classification output.
P.S.
I'm not sure why you would use a neural network for this problem. Fitting a linear/exponential function seems to be a relatively simple ...
Only top voted, non community-wiki answers of a minimum length are eligible