New answers tagged recurrent-neural-networks
1
I'm currently working with Temporal Convolution Networks (TCNs) for making predictions with time series data (link to article here: https://medium.com/@raushan2807/temporal-convolutional-networks-bfea16e6d7d2). These types of networks, like other types of convolutional networks for time series, use a dilated convolution operation, which, unlike the standard ...
1
Parameters is a synonym for weights, which is the term most people use for a neural networks parameters (and indeed in my experience it is a term that machine learners will use in general whereas parameters is more often found in statistics literature). Batch size, learning rate etc. are hyper-parameters which basically means they are user specified, whereas ...
0
As it says in the documentation, you can simply reverse the order of dimensions by providing the argument batch_first=True when constructing the RNN.
Then, the dimensionality will be: (batch, seq, feature), i.e. batch-size times sequence length times the dimension of your input (however dimensional that may be). Then, everything is gonna work as you are used ...
0
Transformers can be trained with parallelization. So you might have 100 cores training the transformer for an hour (100 total hours of CPU time), whereas the LSTM has one core training for ten hours (10 total hours of CPU time). Even though the LSTM takes longer, it is less expensive to train.
0
Since neural networks rely on stochasticity (i.e. randomness) to initialize their parameters and gradient descent selects random batches of training data at each iteration, is perfectly normal if the value of loss function fluctuates instead of decreasing monotonically.
3
There are tasks in computer vision where recurrent neural networks (RNNs) can be useful because there's some sequential sub-task in the main task.
For instance, in the paper Long-Term Recurrent Convolutional Networks for Visual Recognition and Description, the authors investigate the use of a neural network that is both recurrent and convolutional to solve ...
0
In general, there's nothing wrong with training loss to increase from time-to-time during training.
This is because GD with minibatch is a stochastic process and doesn't guarantee that the loss will decrease at each step.
1
Your idea is a good one.
Another idea is to upsample or aggregate your data. For example, average by week if you generally have a couple of missing days in every week.
A similar question on Stack Exchange:
https://stats.stackexchange.com/questions/374935/how-to-deal-with-really-sparse-time-series-data-for-a-binary-classification-task
Top 50 recent answers are included
