New answers tagged pytorch
0
This was recently answered on the PyTorch Forums.
Yes, torch.inference_mode is indeed preferable to torch.no_grad in all situations where inference mode does not throw a runtime error.
The reason it took until version 1.9 to be implemented was precisely because it was originally difficult to ensure that all "unsafe" operations in inference mode ...
2
This one is a bit crazy:
pool1 = nn.AvgPool3d(kernel_size = (361, 1, 1), stride= 1)
because it averages large numbers of the features at once. Very little information about individual features will remain after doing that.
The most obvious one you have not tried is this:
pool3 = nn.AvgPool3d(kernel_size = (3, 1, 1), stride= (3, 1, 1))
which includes all ...
2
Gradient clipping is a technique that tackles exploding gradients. The idea of gradient clipping is very simple: If the gradient gets too large, we rescale it to keep it small. More precisely,
$$
\text{if } \Vert \mathbf{g} \Vert \geq c, \text{then }
\mathbf{g} \leftarrow c \frac{\mathbf{g}}{\Vert \mathbf{g} \Vert}
$$
where $c$ is a hyperparameter, $\mathbf{...
0
It might be a good idea to normalized the coordinates with their counterparts in all 4 objects: for example, if you use min-max scaler, you should scale x1, x2, x3, x4 together, and the same for y and z coordinates.
This is assuming your coordinate system is infinite. If you have a finite coordinate system, or there exists some natural limits to the maximum ...
0
This is probably an issue of complete underfitting. How many training data do you use? What is your vocab size? What is your batch size and how many epochs did you train? Transformers always need more data than RNNs to reach good text quality.
Top 50 recent answers are included