Questions tagged [recurrent-neural-networks]

Use for questions about Recurrent Neural Networks

Filter by
Sorted by
Tagged with
1
vote
2answers
232 views

How to use RNN With Attention Mechanism on Non Textual Data?

Recurrent Neural Networks (RNN) With Attention Mechanism is generally used for Machine Translation and Natural Language Processing. In Python, implementation of RNN With Attention Mechanism is ...
0
votes
1answer
120 views

LSTM language model not working

I am trying to use a Keras LSTM neural network for character level language modelling. As the input, I give it the last 50 characters and it has to output the next one. It has 3 layers of 400 neurons ...
-3
votes
1answer
565 views

RNN LSTM not converging with Adam

I am trying to train a RNN with text from wikipedia but I having having trouble getting the RNN to converge. I have tried increasing the batch size but it doesn't seem to be helping. All data is one ...
1
vote
0answers
191 views

Number of nodes in hidden layer for LSTM

I am starting to learn LSTM by understanding how it is used for creating a char-RNN and had a fundamental question. Does the number of nodes in the hidden layer need to be the same as that of the ...
2
votes
1answer
167 views

Active Learning and RNN

We have a series of data which want to label part of each series. As we do not have any training data, we try using Active Learning as a solution. But, the problem is our classifier is something like ...
3
votes
1answer
60 views

Does an advanced Dialogue state tracking eliminate the need of intent classifier and slot filling models in dialogue systems/ chatbots?

I am learning to create a dialogue system. The various parts of such a system are Intent classifier, slot filling, Dialogue state tracking (DST), dialogue policy optimization and NLG. While reading ...
2
votes
0answers
85 views

Why are all the actions converging to the same index?

I am using PPO with an LSTM agent. My agent is performing 10 actions for each episode, one action is corresponding to one LSTM timestep and the action space is discrete. I have only one reward per ...
1
vote
0answers
36 views

Train a recurrent neural network by concatenating time series. Is it safe?

As the title says, I want to train a Jordan network (i.e. a particular kind of recurrent neural network) using a certain number of time series. Let's say that $x_1, x_2, \ldots x_N$ are $N$ input ...
2
votes
0answers
95 views

Update of weights in Recurrent Neural Network through back propagation

How does Recurrent Neural Network updates its weights and bias through backpropagation? Is time taken into account while updating the weights of a RNN using Backpropagation through time(BPTT)?"
-1
votes
1answer
24 views

Getting better results in improving the configuration

Currently, I found the right recipe for a time series regression problem to finally get acceptable to good results. Here is the config file ...
1
vote
2answers
212 views

What should I do when I have a variable-length sequence when instantiating an LSTM in Keras?

In keras, when we use an LSTM/RNN model, we need to specify the node [i.e., LSTM(128)]. I have a doubt regarding how it actually works. From the LSTM/RNN unfolding image or description, I found that ...
3
votes
1answer
489 views

Fourier Transform inputs (Frequency) for RNN

Can the recurrent neural network input come from short time fourier transform in MATLAB? I mean the input is not from time series domain.
5
votes
1answer
2k views

Where can I find the original paper that introduced RNNs?

I was able to find the original paper on LSTM, I was not able to find the paper that introduced "vanilla" RNNs. Where can I find it?
1
vote
1answer
59 views

predict waste generation

I am starting a project to predict the generation of urban waste. I have found very little information on this topic on the internet. I would be very useful advice on how to approach this topic, and ...
1
vote
1answer
66 views

Why would giving my AI more data make it perform worse?

So I trained an AI to generate shakespeare, which it did somewhat well. I used this 10,000 character sample. Next I tried to get it to generate limericks using these 100,000 limericks. It generated ...
2
votes
1answer
83 views

Is it possible to use an RNN to predict a feature that is not an input feature?

I came across RNN's a few minutes ago, which might solve a problem with sequenced data I've had for a while now. Let's say I have a set of input features, generated every second. Corresponding with ...
3
votes
2answers
295 views

Where can I find pre-trained language models in English and German?

Where can I find (more) pre-trained language models? I am especially interested in neural network based models for English and German. And I specifically mean language model in its standard sense. I ...
1
vote
0answers
42 views

How recurrent neural network work when predict many days?

I use recurrent neural network, RNNs have to get input one value per step and it will show one value output. If I have daily sale demand time series data. I want to predict sale demand for three ...
4
votes
1answer
685 views

Will attention based networks prevail over RNN and LSTM?

There is no point in picking one of the growing number of articles that come up in a web search for, "Deep learning attention networks," however the bold claims in Attention Is All You Need, Ashish ...
1
vote
1answer
292 views

Backward Pass for LSTMs

TL;DR I am currently trying to understand the mathematics in Ger's paper Long Short-Term Memory in Recurrent Neural Networks. I have found the document clear and readable so far. On pg. 21 of the ...
9
votes
4answers
568 views

Beyond neural networks?

Are there possible algorithms that have the potential to replace neural nets in the near future? And do we need that? What is the worst thing of using neural networks in terms of efficiency?
5
votes
2answers
913 views

Structure of LSTM RNNs

I have some very basic questions here. This is probably because I didn't read the relevant documents closely enough. If I used some terminology incorrectly, please point them out. Thank you! For ...
1
vote
0answers
132 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
3
votes
2answers
154 views

How to build my own dataset and model for an LSTM neural network

I have a sort of mathematical problem and I'm not sure which model I should choose to make an LSTM neural network. Currently in my country, there is a system in which certain groups of researchers ...
1
vote
0answers
53 views

How to adapt RNNs to variable frequency / framerate of inputs?

Say I have an application where the frequency of the input is known but can vary widely across sequences. For example, they may be audio recordings acquired at different frequency, or videos that come ...
2
votes
1answer
38 views

Why are GRU and LSTM better than old types of RNN?

Seems older RNNs have a limitation for their use cases and have been outperformed by other architectures for specific tasks e.g GRUS and CNNs
3
votes
2answers
218 views

Is there an alternative to RNNs that doesn't require knowing input history?

In an RNN to train it, you need to roll it out, and enter in the history of inputs and the history of expected outcomes. This doesn't seem like a realistic picture of the brain since this would ...
-1
votes
1answer
48 views

Teaching a NN to manipulate pseudoRNG over a long time scale?

For speedrunning purposes, I am trying to train a neural network to identify human-executable ways to manipulate pseudo-RNG (in Pokemon Red, for the interested). The game runs at sixty frames per ...
4
votes
1answer
347 views

Over- and underestimations of the lowest and highest values in LSTM network

I'm training a LSTM network with multiple inputs and several LSTM layers in order to setup a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation on the ...
2
votes
2answers
243 views

What is the significance of this Stanford University “Financial Market Time Series Prediction with RNN's” paper?

Researchers at Stanford University released this paper in 2012: http://cs229.stanford.edu/proj2012/BernalFokPidaparthi-FinancialMarketTimeSeriesPredictionwithRecurrentNeural.pdf It goes on to ...
1
vote
0answers
26 views

Whats advantages does a Loop Network have over a Feed Forward Network?

I am interested to see what advantages a Loop Network (Feed Forward Network that takes its output as input, I think it's called an RNN, not sure). The only result I found was that it was extremely ...
1
vote
1answer
46 views

How to manage high numbers of input layer data points

Not sure if this is the correct forum, but I have been working with a large (non-image) dataset that will eventually be used to train a neural network. I have been puzzling over how to manage wide ...
2
votes
0answers
91 views

Combine two embeddding inputs to increase more performance in LSTM model

The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second ...
1
vote
2answers
105 views

Time Series: LSTM or Augmented Vector Space?

In time Series prediction, we have a stream of vectors. There are different approaches for accounting for the temporal patterns between these vectors. There's two that I'm considering. An LSTM or ...
1
vote
0answers
79 views

Best practices to classify recurring patterns using an LSTM or GRU

I'm working with acoustic data (filterbank features) and I want to build a neural network to detect claps using an LSTM (or a GRU) with a binary output (present/abscent), and I'm wondering about how I ...
1
vote
1answer
1k views

How to calculate the output of this neural network?

What is the output value of the network for these inputs respectively, and why? (Linear activation function is fine.) [2, 3][-1, 2][1, 0][3, 4] My main question is how you take the 'backwards' ...
3
votes
2answers
99 views

What is the relation between Convolutional Neural Networks and Recurrent Neural Networks?

I asked my self this simple question while reading "Comment Abuse Classification with Deep Learning" by Chu and Jue. Indeed, they say at the end of the that It is clear that RNNs, specifically ...
3
votes
1answer
859 views

Seq2Seq dialogs predicts only most common words like `you` after couple of epoches

I'm training Seq2Seq model on OpenSubtitles dialogs - Cornell-Movie-Dialogs-Corpus. My work based on the following papers (but currently I'm not implemented Attention yet): Sequence to Sequence ...
4
votes
3answers
145 views

What kinds of systems have so far failed to be modeled via supervised artificial network training?

Artificial networks model systems with a set of inputs and outputs and expected behavior. To train a network for modeling such systems, hundreds, thousands, or millions of example inputs-output pairs ...
1
vote
0answers
143 views

Trajectory classification using RNN

The problem: I want to classify a trajectory if it has some properties, for example I want to create a simple 0/1 classifier for circular trajectories. If a target is moving in a circular trajectory ...
1
vote
0answers
30 views

How to train a recurrent neural network with multiple series

I am new to neural networks. I am trying to model the run-off vs. time in a water channel after a storm event given that I know the permeability of the material in the channel, total precipitation, ...
1
vote
1answer
70 views

How is the word embedding represented in the paper “Recurrent neural network based language model”?

I'm reading "Recurrent neural network based language model" of Mikolov et al. (2010). Although the article is straight forward, I'm not sure how word embedding $w(t)$ is obtained: The reason I wonder ...
1
vote
0answers
249 views

Tensorflow: Can't overfit training data with batch size > 1

I coded a small RNN network with Tensorflow to return the total energy consumption given some parameters. There seem to be a problem in my code. It can't overfit the training data when I use a batch ...
-4
votes
1answer
114 views

How to train recurrent neural network? [closed]

I need to have a full brief on recurrent neural network. With the explanation how to train recurrent neural network??
-2
votes
1answer
334 views

How to train a simple neural network to create source code

I am thinking of making a simple and interesting research on training a RNN, to create source codes. The objective is to have a set of simple REPL programs in java, and create a source code for a ...
7
votes
4answers
4k views

What is the fundamental difference between CNN and RNN?

What is the fundamental difference between convolutional neural networks and recurrent neural networks? Where are they applied?
0
votes
1answer
126 views

What's wrong with getting a dynamic RNN's output at the end of a padded sequence?

Would it not even be more helpful? It will have more time to transform the input with the extra zeros padded to the end.
3
votes
0answers
116 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that when training RNN's on text for something like language modeling, the text is usually featurized character-by-...
7
votes
1answer
131 views

Neural network design when amount of input neurons vary

I'm looking to design a neural network that can predict which runner wins in a sports game, where the amount of runners varies between 2-10. In each case, specific data about the individual runners ...
2
votes
0answers
35 views

Detecting symmetry in small images with RNN

My network works on 32x32 normalized (translationally) but noisy images. Its task it to determine whether image has simple symmetry (horizontal/vertical). It needs to be reasonably robust to rotation (...