New answers tagged

1 vote
Accepted

Can I scale subsets of my dataset independently to handle different feature ranges?

Scaling individual parts of a dataset individually is generally not that trivial. Just remember that this means that the statistical properties of the subsets also may change. E.g., let's assume that ...
BanDoP's user avatar
  • 193
1 vote

LSTM text classifier shows unexpected cyclical pattern in loss

This weird pattern can be caused by a big learning rate. Check this: https://stackoverflow.com/a/49095437/13164928
ZappaBoy's user avatar
2 votes

Wouldn't residual connections in RNNs solve the vanishing/exploding gradient problem?

Your idea is exactly the idea behind state-space models. They have a linear "residual" connection from previous hidden states, skipping activations. In fact, it works very well! I'd ...
programjames's user avatar
3 votes
Accepted

Wouldn't residual connections in RNNs solve the vanishing/exploding gradient problem?

In my opinion your idea indeed holds merit. Something worth noting though is that it is cruder than the LSTM/GRU that have trainable weights that guide what features are remembered and forgotten. ...
Victor Björkgren's user avatar

Top 50 recent answers are included