New answers tagged recurrent-neural-networks
0
I don't think you need a recurrent neural network for this. Why not just train a feedforward model with angle of attack etc as input and translation velocities as output? The size of the output will depend on how frequently you want updates to the velocities. e.g. if the update rate is 0.5 seconds and the network is predicting 2 seconds in advance it could ...
1
How about a Temporal Convolutional Network? It feels like for such a long sequences having the recurrent/memory based approach is not too feasible. But, intuitively, the 1D convolutions should be able to pick out those rare features from your extremely long sequences.
There are also claims that TCNs are comparable to RNNs in performance on common tasks, so ...
1
The purpose of a clockwork RNN is to help with long term dependencies. Let's say in this case, we have a sentence that starts with "John went to..." and at no point again is John's name mentioned throughout the few paragraphs we are passing to our model.
As mentioned in the paper, the most common method to combat this (at the time) was using an ...
1
It seems this paper defines a "clock period ${T}_n$" that it uses to express the topology of the network: "Each module is internally fully-interconnected, but the recurrent connections from module j to module i exists only if the period $T_i$ is smaller than period $T_j$.".
This definition is, however, only in this paper, as far as I know....
2
Not quite sure about RNN & LSTM (and it always depends on the task), but for CNN the answer is clearly no; CNN routinely include FC layers. Quoting from the highly popular (and recommended) Stanford course CS231n: Convolutional Neural Networks for Visual Recognition:
ConvNet Architectures
We have seen that Convolutional Networks are commonly made up of ...
1
You are talking about model parallelism. But, that's not the reason RNNs/LSTMs are not in vogue.
Imagine your ability to read the first line of a page and going on reading and still making connections to the first line until the end of the page.
Can RNNs/LSTMs do that? No.
Can Attention (i.e. Transformers) do it? Yes.
The reason is simple Attention is ...
1
Yes, it is possible. What you have shown in case of ANN is what happens in a regression model using NNs. What you have shown in case of RNN is what happens when you are doing sequence-to-sequence translation (like French to English).
If you want to get single values like in case of ANN, suppose you are doing regression, then, in the end, you will flatten the ...
0
If the text is really that regular, you can do simple pattern matching: Search for the keywords "limit of liability", "deductible", and "sum insured". The take the next numerical value (possibly preceded by "INR"), and assign it to the corresponding value.
However, this is very simplistic and brittle, and will fail if ...
0
I think the problem is that you're only training the network on words. Every example in your training data has a desired label of "is a word," and so your network could achieve the lowest possible loss by simply giving a probability of 100% to "is a word" all of the time.
The most straightforward way to fix this would be to also include ...
0
while using a neural network for this type of problem is not the ideal use-case, it is a good exercise.
In terms of conceptual issues, the most concerning that I see is the loss: $\sum_{i=1}^N L_i$.
First issue, is that it validates loss at each time step equivalently. This is probably not ideal because in the example (cat), we dont expect it to know its ...
Top 50 recent answers are included
