Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [recurrent-neural-networks]

Use for questions about Recurrent Neural Networks

0
votes
0answers
9 views

How my LSTM is making association between the columns?

I have created LSTM model using the following Tensorflow code: ...
0
votes
2answers
28 views

Do I need an encoder-decoder architecture to predict the next item of a sequence?

I am trying to understand how RNNs are used for sequence modelling. On a tutorial here, it mentions that if you want to translate say a sentence from English to French you can use an encoder-decoder ...
2
votes
0answers
63 views

What is the intuition behind the calculation of the similarity between encoder and decoder states?

Suppose that we are doing machine translation. We have a conditional language model with attention where we are are trying to predict a sequence $y_1, y_2, \dots, y_J$ from $x_1, x_2, \dots x_I$: $$P(...
-1
votes
0answers
10 views

Slow moving auto parts data (public) from a US automobile company: where to find it? [migrated]

I have found multiple papers which use this dataset (because it's apparently public) to set benchmarks in forecasting but they don't mention where it can be found. I have done some searching on my own ...
1
vote
1answer
34 views

Hidden state of the GRU

I'm having a hard time understanding how does the size of the hidden state affects GRU. For example in a concrete example lets say I want to lean a GRU to count. I'm gonna feed it fx 3 timestamps the ...
0
votes
1answer
56 views

BERT - What are the segment and position embeddings used in here?

They only reference in the paper that the position embeddings are learned, which is different from what was done in ELMo. ELMo paper - https://arxiv.org/pdf/1802.05365.pdf BERT paper - https://arxiv....
0
votes
1answer
54 views

how to predict the stock of food items using ai

Actually, I want to make an AI model which tells the seller about the maintaining stock of food items as a parameter of time and eventually learns by itself with the customer buying data (problem - ...
1
vote
0answers
18 views

Under What Conditions Will a Hopfield Network Tend to Converge to Spurious States?

I'm relatively new to neural networks, and I've been trying to program my own Hopfield network. I got it to the point where it can reliably reproduce a single pattern from a completely scrambled ...
2
votes
1answer
82 views

Additive Attention in Convolutional Networks

Attention has been used widely in recurrent networks to weight feature representations learned by the model. This is not a trivial task since recurrent networks have a hidden state that captures ...
1
vote
2answers
55 views

How to use RNN With Attention Mechanism on Non Textual Data?

Recurrent Neural Networks (RNN) With Attention Mechanism is generally used for Machine Translation and Natural Language Processing. In Python, implementation of RNN With Attention Mechanism is ...
0
votes
1answer
19 views

LSTM language model not working

I am trying to use a Keras LSTM neural network for character level language modelling. As the input, I give it the last 50 characters and it has to output the next one. It has 3 layers of 400 neurons ...
-2
votes
1answer
100 views

RNN LSTM not converging with Adam

I am trying to train a RNN with text from wikipedia but I having having trouble getting the RNN to converge. I have tried increasing the batch size but it doesn't seem to be helping. All data is one ...
1
vote
0answers
61 views

Number of nodes in hidden layer for LSTM

I am starting to learn LSTM by understanding how it is used for creating a char-RNN and had a fundamental question. Does the number of nodes in the hidden layer need to be the same as that of the ...
1
vote
1answer
43 views

Active Learning and RNN

We have a series of data which want to label part of each series. As we do not have any training data, we try using Active Learning as a solution. But, the problem is our classifier is something like ...
3
votes
1answer
38 views

Does an advanced Dialogue state tracking eliminate the need of intent classifier and slot filling models in dialogue systems/ chatbots?

I am learning to create a dialogue system. The various parts of such a system are Intent classifier, slot filling, Dialogue state tracking (DST), dialogue policy optimization and NLG. While reading ...
1
vote
0answers
41 views

Why are all the actions converging to the same index?

I am using PPO with an LSTM agent. My agent is performing 10 actions for each episode, one action is corresponding to one LSTM timestep and the action space is discrete. I have only one reward per ...
0
votes
0answers
22 views

Train a recurrent neural network by concatenating time series. Is it safe?

As the title says, I want to train a Jordan network (i.e. a particular kind of recurrent neural network) using a certain number of time series. Let's say that $x_1, x_2, \ldots x_N$ are $N$ input ...
2
votes
0answers
81 views

Update of weights in Recurrent Neural Network through back propagation

How does Recurrent Neural Network updates its weights and bias through backpropagation? Is time taken into account while updating the weights of a RNN using Backpropagation through time(BPTT)?"
-1
votes
1answer
22 views

Getting better results in improving the configuration

Currently, I found the right recipe for a time series regression problem to finally get acceptable to good results. Here is the config file ...
1
vote
2answers
131 views

What should I do when I have a variable-length sequence when instantiating an LSTM in Keras?

In keras, when we use an LSTM/RNN model, we need to specify the node [i.e., LSTM(128)]. I have a doubt regarding how it actually works. From the LSTM/RNN unfolding image or description, I found that ...
3
votes
1answer
164 views

Fourier Transform inputs (Frequency) for RNN

Can the recurrent neural network input come from short time fourier transform in MATLAB? I mean the input is not from time series domain.
3
votes
1answer
186 views

Where can I find the original paper that introduced RNNs?

I was able to find the original paper on LSTM, I was not able to find the paper that introduced "vanilla" RNNs. Where can I find it?
1
vote
1answer
54 views

predict waste generation

I am starting a project to predict the generation of urban waste. I have found very little information on this topic on the internet. I would be very useful advice on how to approach this topic, and ...
2
votes
1answer
63 views

Why would giving my AI more data make it perform worse?

So I trained an AI to generate shakespeare, which it did somewhat well. I used this 10,000 character sample. Next I tried to get it to generate limericks using these 100,000 limericks. It generated ...
2
votes
1answer
48 views

Is it possible to use an RNN to predict a feature that is not an input feature?

I came across RNN's a few minutes ago, which might solve a problem with sequenced data I've had for a while now. Let's say I have a set of input features, generated every second. Corresponding with ...
3
votes
2answers
127 views

Where can I find pre-trained language models in English and German?

Where can I find (more) pre-trained language models? I am especially interested in neural network based models for English and German. And I specifically mean language model in its standard sense. I ...
1
vote
0answers
36 views

How recurrent neural network work when predict many days?

I use recurrent neural network, RNNs have to get input one value per step and it will show one value output. If I have daily sale demand time series data. I want to predict sale demand for three ...
4
votes
1answer
208 views

Will attention based networks prevail over RNN and LSTM?

There is no point in picking one of the growing number of articles that come up in a web search for, "Deep learning attention networks," however the bold claims in Attention Is All You Need, Ashish ...
1
vote
1answer
187 views

Backward Pass for LSTMs

TL;DR I am currently trying to understand the mathematics in Ger's paper Long Short-Term Memory in Recurrent Neural Networks. I have found the document clear and readable so far. On pg. 21 of the ...
8
votes
4answers
359 views

Beyond neural networks?

Are there possible algorithms that have the potential to replace neural nets in the near future? And do we need that? What is the worst thing of using neural networks in terms of efficiency?
5
votes
2answers
375 views

Structure of LSTM RNNs

I have some very basic questions here. This is probably because I didn't read the relevant documents closely enough. If I used some terminology incorrectly, please point them out. Thank you! For ...
1
vote
0answers
100 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
3
votes
2answers
103 views

How to build my own dataset and model for an LSTM neural network

I have a sort of mathematical problem and I'm not sure which model I should choose to make an LSTM neural network. Currently in my country, there is a system in which certain groups of researchers ...
1
vote
0answers
22 views

How to adapt RNNs to variable frequency / framerate of inputs?

Say I have an application where the frequency of the input is known but can vary widely across sequences. For example, they may be audio recordings acquired at different frequency, or videos that come ...
2
votes
1answer
30 views

Why are GRU and LSTM better than old types of RNN?

Seems older RNNs have a limitation for their use cases and have been outperformed by other architectures for specific tasks e.g GRUS and CNNs
3
votes
2answers
120 views

Is there an alternative to RNNs that doesn't require knowing input history?

In an RNN to train it, you need to roll it out, and enter in the history of inputs and the history of expected outcomes. This doesn't seem like a realistic picture of the brain since this would ...
-1
votes
1answer
40 views

Teaching a NN to manipulate pseudoRNG over a long time scale?

For speedrunning purposes, I am trying to train a neural network to identify human-executable ways to manipulate pseudo-RNG (in Pokemon Red, for the interested). The game runs at sixty frames per ...
3
votes
1answer
209 views

Over- and underestimations of the lowest and highest values in LSTM network

I'm training a LSTM network with multiple inputs and several LSTM layers in order to setup a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation on the ...
2
votes
2answers
176 views

What is the significance of this Stanford University “Financial Market Time Series Prediction with RNN's” paper?

Researchers at Stanford University released this paper in 2012: http://cs229.stanford.edu/proj2012/BernalFokPidaparthi-FinancialMarketTimeSeriesPredictionwithRecurrentNeural.pdf It goes on to ...
1
vote
0answers
24 views

Whats advantages does a Loop Network have over a Feed Forward Network?

I am interested to see what advantages a Loop Network (Feed Forward Network that takes its output as input, I think it's called an RNN, not sure). The only result I found was that it was extremely ...
1
vote
1answer
36 views

How to manage high numbers of input layer data points

Not sure if this is the correct forum, but I have been working with a large (non-image) dataset that will eventually be used to train a neural network. I have been puzzling over how to manage wide ...
1
vote
0answers
71 views

Combine two embeddding inputs to increase more performance in LSTM model

The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second ...
1
vote
2answers
78 views

Time Series: LSTM or Augmented Vector Space?

In time Series prediction, we have a stream of vectors. There are different approaches for accounting for the temporal patterns between these vectors. There's two that I'm considering. An LSTM or ...
1
vote
0answers
67 views

Best practices to classify recurring patterns using an LSTM or GRU

I'm working with acoustic data (filterbank features) and I want to build a neural network to detect claps using an LSTM (or a GRU) with a binary output (present/abscent), and I'm wondering about how I ...
1
vote
1answer
783 views

How to calculate the output of this neural network?

What is the output value of the network for these inputs respectively, and why? (Linear activation function is fine.) [2, 3][-1, 2][1, 0][3, 4] My main question is how you take the 'backwards' ...
3
votes
2answers
67 views

Are Convolutional Neural Networks Recurrent Neural Networks?

I asked my self this simple question while reading "Comment Abuse Classification with Deep Learning" by Chu and Jue. Indeed, they say at the end of the that It is clear that RNNs, specifically ...
3
votes
1answer
463 views

Seq2Seq dialogs predicts only most common words like `you` after couple of epoches

I'm training Seq2Seq model on OpenSubtitles dialogs - Cornell-Movie-Dialogs-Corpus. My work based on the following papers (but currently I'm not implemented Attention yet): Sequence to Sequence ...
4
votes
3answers
132 views

What kinds of systems have so far failed to be modeled via supervised artificial network training?

Artificial networks model systems with a set of inputs and outputs and expected behavior. To train a network for modeling such systems, hundreds, thousands, or millions of example inputs-output pairs ...
1
vote
0answers
107 views

Trajectory classification using RNN

The problem: I want to classify a trajectory if it has some properties, for example I want to create a simple 0/1 classifier for circular trajectories. If a target is moving in a circular trajectory ...
1
vote
0answers
24 views

How to train a recurrent neural network with multiple series

I am new to neural networks. I am trying to model the run-off vs. time in a water channel after a storm event given that I know the permeability of the material in the channel, total precipitation, ...