New answers tagged

1

I am currently working on a similar problem. I think your approach is good. As for setting the parameter lambda, since you are using deep neural networks, you can make it a learnable parameter, instead of a hyperparameter you set. This way, as the two losses fluctuate over your training iterations/epochs, the model will be able to adjust the lambda parameter ...


1

This looks like overfitting. You can try stop training earlier by using a validation dataset to prevent this, or you can try other regularization effects such as weight-decay, dropout etc.


1

I think, that $\mathbb{R}^{D}$ is most natural choice in practical situations since many kinds of data can be described in a this way: Image is 2d array $H \times W$ with each pixel taking value in $\mathbb{R}^{c}$ (say, $c=3$) or a continuous subset of $\mathbb{R}^{c} = [-1, 1]$ In the sequence modeling problems one gives an embedding vector to each token ...


Top 50 recent answers are included