Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [recurrent-neural-networks]

Use for questions about Recurrent Neural Networks

0
votes
0answers
5 views

Interpretation of cost behaviour in ensemble method

I am working on a problem in NLP, in mapping a question to a target passage. To solve this problem, I am using a fairly complicated model including attention mechanism. The dataset is quite large and ...
0
votes
0answers
11 views

Can you train a deep recurrent neural network layer by layer?

Specifically for Gated Recurrent Unit, and say GRU is "layered" via but suppose it's only 2 layers deep for simplicity, and suppose the "total loss" = $L$ = $\sum l_{t} = \sum error(y^{2}_{t})$ for ...
1
vote
1answer
21 views

Active Learning and RNN

We have a series of data which want to label part of each series. As we do not have any training data, we try using Active Learning as a solution. But, the problem is our classifier is something like ...
0
votes
0answers
5 views

LSTM output dimensionality

I am new to LSTMs. When reading the papers and websites about LSTM architecture, there is something I do not get. As I understand it, a single LSTM layer can have multiple LSTM cells (just like a ...
0
votes
0answers
12 views

Did anyone try topic modelling with neural nets?

I constantly see Latent Dirichlet Allocation (LDA) as a go to technique for topic modelling. It performs okay-ish, but ignores word context and (subjectively) seems outdated. Has anyone implemented ...
3
votes
1answer
33 views

Does an advanced Dialogue state tracking eliminate the need of intent classifier and slot filling models in dialogue systems/ chatbots?

I am learning to create a dialogue system. The various parts of such a system are Intent classifier, slot filling, Dialogue state tracking (DST), dialogue policy optimization and NLG. While reading ...
0
votes
0answers
17 views

Language Models RNN

I am relatively new to recurrent neural networks and it seems like a vast domain. So I want to get my initial footing right. There seems to be a whole lot of applications in this field, but the first ...
0
votes
0answers
7 views

Why does an RNN for text generation benefit from having more nodes than sequence length

If I have an RNN for text generation and want the RNN to learn characterwise, I partition the text to learn into sequences of equal length. Now my logic would be: If I have a sequence length x, then I ...
1
vote
0answers
26 views

Why are all the actions converging to the same index?

I am using PPO with an LSTM agent. My agent is performing 10 actions for each episode, one action is corresponding to one LSTM timestep and the action space is discrete. I have only one reward per ...
0
votes
0answers
17 views

Train a recurrent neural network by concatenating time series. Is it safe?

As the title says, I want to train a Jordan network (i.e. a particular kind of recurrent neural network) using a certain number of time series. Let's say that $x_1, x_2, \ldots x_N$ are $N$ input ...
2
votes
0answers
72 views

Update of weights in Recurrent Neural Network through back propagation

How does Recurrent Neural Network updates its weights and bias through backpropagation? Is time taken into account while updating the weights of a RNN using Backpropagation through time(BPTT)?"
-1
votes
1answer
18 views

Getting better results in improving the configuration

Currently, I found the right recipe for a time series regression problem to finally get acceptable to good results. Here is the config file ...
3
votes
1answer
75 views

Fourier Transform inputs (Frequency) for RNN

Can the recurrent neural network input come from short time fourier transform in MATLAB? I mean the input is not from time series domain.
0
votes
0answers
18 views

Sequentia data with static data (gender, group, Major) in RNN model (LSTM)

I have a sequential dataset and I built an LSTM model using Keras for prediction accuracy. I would like to add static data ( i.e. gender, blood group, major) to the sequential data. I found a paper (...
3
votes
1answer
49 views

Where can I find the original paper that introduced RNNs?

I was able to find the original paper on LSTM, I was not able to find the paper that introduced "vanilla" RNNs. Where can I find it?
1
vote
1answer
47 views

predict waste generation

I am starting a project to predict the generation of urban waste. I have found very little information on this topic on the internet. I would be very useful advice on how to approach this topic, and ...
0
votes
0answers
16 views

Can an RNN be said to be a special case of a message passing neural network?

Can an RNN be said to be a special case of a simple message passing neural network, where one element passes on a message to the next element in the sequence? Or is there more to the definition of a "...
2
votes
1answer
60 views

Why would giving my AI more data make it perform worse?

So I trained an AI to generate shakespeare, which it did somewhat well. I used this 10,000 character sample. Next I tried to get it to generate limericks using these 100,000 limericks. It generated ...
2
votes
1answer
42 views

Is it possible to use an RNN to predict a feature that is not an input feature?

I came across RNN's a few minutes ago, which might solve a problem with sequenced data I've had for a while now. Let's say I have a set of input features, generated every second. Corresponding with ...
3
votes
2answers
78 views

Where can I find pre-trained language models in English and German?

Where can I find (more) pre-trained language models? I am especially interested in neural network based models for English and German. And I specifically mean language model in its standard sense. I ...
1
vote
0answers
32 views

How recurrent neural network work when predict many days?

I use recurrent neural network, RNNs have to get input one value per step and it will show one value output. If I have daily sale demand time series data. I want to predict sale demand for three ...
3
votes
1answer
57 views

Will attention based networks prevail over RNN and LSTM?

There is no point in picking one of the growing number of articles that come up in a web search for, "Deep learning attention networks," however the bold claims in Attention Is All You Need, Ashish ...
1
vote
1answer
133 views

Backward Pass for LSTMs

TL;DR I am currently trying to understand the mathematics in Ger's paper Long Short-Term Memory in Recurrent Neural Networks. I have found the document clear and readable so far. On pg. 21 of the ...
8
votes
4answers
256 views

Beyond neural networks?

Are there possible algorithms that have the potential to replace neural nets in the near future? And do we need that? What is the worst thing of using neural networks in terms of efficiency?
3
votes
1answer
206 views

Structure of LSTM RNNs

I have some very basic questions here. This is probably because I didn't read the relevant documents closely enough. If I used some terminology incorrectly, please point them out. Thank you! For ...
1
vote
0answers
84 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
3
votes
2answers
85 views

How to build my own dataset and model for an LSTM neural network

I have a sort of mathematical problem and I'm not sure which model I should choose to make an LSTM neural network. Currently in my country, there is a system in which certain groups of researchers ...
1
vote
0answers
15 views

How to adapt RNNs to variable frequency / framerate of inputs?

Say I have an application where the frequency of the input is known but can vary widely across sequences. For example, they may be audio recordings acquired at different frequency, or videos that come ...
0
votes
0answers
54 views

Listen, Attend and Spell

I was reading the Listen, Attend and Spell paper and did not understand some terms/concepts. I need help with the following: what is pyramid Bidirectional LSTM and how it helps to reduce the time ...
2
votes
1answer
24 views

Why are GRU and LSTM better than old types of RNN?

Seems older RNNs have a limitation for their use cases and have been outperformed by other architectures for specific tasks e.g GRUS and CNNs
3
votes
2answers
82 views

Is there an alternative to RNNs that doesn't require knowing input history?

In an RNN to train it, you need to roll it out, and enter in the history of inputs and the history of expected outcomes. This doesn't seem like a realistic picture of the brain since this would ...
-1
votes
1answer
36 views

Teaching a NN to manipulate pseudoRNG over a long time scale?

For speedrunning purposes, I am trying to train a neural network to identify human-executable ways to manipulate pseudo-RNG (in Pokemon Red, for the interested). The game runs at sixty frames per ...
3
votes
1answer
152 views

Over- and underestimations of the lowest and highest values in LSTM network

I'm training a LSTM network with multiple inputs and several LSTM layers in order to setup a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation on the ...
2
votes
2answers
142 views

What is the significance of this Stanford University “Financial Market Time Series Prediction with RNN's” paper?

Researchers at Stanford University released this paper in 2012: http://cs229.stanford.edu/proj2012/BernalFokPidaparthi-FinancialMarketTimeSeriesPredictionwithRecurrentNeural.pdf It goes on to ...
1
vote
0answers
23 views

Whats advantages does a Loop Network have over a Feed Forward Network?

I am interested to see what advantages a Loop Network (Feed Forward Network that takes its output as input, I think it's called an RNN, not sure). The only result I found was that it was extremely ...
1
vote
1answer
28 views

How to manage high numbers of input layer data points

Not sure if this is the correct forum, but I have been working with a large (non-image) dataset that will eventually be used to train a neural network. I have been puzzling over how to manage wide ...
1
vote
0answers
58 views

Combine two embeddding inputs to increase more performance in LSTM model

The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second ...
1
vote
2answers
62 views

Time Series: LSTM or Augmented Vector Space?

In time Series prediction, we have a stream of vectors. There are different approaches for accounting for the temporal patterns between these vectors. There's two that I'm considering. An LSTM or ...
0
votes
0answers
20 views

Any opensource nnet for video transcription?

I am interested in AI that describes what's happening in video. Are there any open source nnets available on gihtub, etc? If not, what's the best way to find papers that implemented these types of ...
1
vote
0answers
58 views

Best practices to classify recurring patterns using an LSTM or GRU

I'm working with acoustic data (filterbank features) and I want to build a neural network to detect claps using an LSTM (or a GRU) with a binary output (present/abscent), and I'm wondering about how I ...
1
vote
1answer
578 views

How to calculate the output of this neural network?

What is the output value of the network for these inputs respectively, and why? (Linear activation function is fine.) [2, 3][-1, 2][1, 0][3, 4] My main question is how you take the 'backwards' ...
3
votes
2answers
62 views

Are Convolutional Neural Networks Recurrent Neural Networks?

I asked my self this simple question while reading "Comment Abuse Classification with Deep Learning" by Chu and Jue. Indeed, they say at the end of the that It is clear that RNNs, specifically ...
3
votes
1answer
292 views

Seq2Seq dialogs predicts only most common words like `you` after couple of epoches

I'm training Seq2Seq model on OpenSubtitles dialogs - Cornell-Movie-Dialogs-Corpus. My work based on the following papers (but currently I'm not implemented Attention yet): Sequence to Sequence ...
4
votes
3answers
131 views

What kinds of systems have so far failed to be modeled via supervised artificial network training?

Artificial networks model systems with a set of inputs and outputs and expected behavior. To train a network for modeling such systems, hundreds, thousands, or millions of example inputs-output pairs ...
1
vote
0answers
93 views

Trajectory classification using RNN

The problem: I want to classify a trajectory if it has some properties, for example I want to create a simple 0/1 classifier for circular trajectories. If a target is moving in a circular trajectory ...
1
vote
0answers
23 views

How to train a recurrent neural network with multiple series

I am new to neural networks. I am trying to model the run-off vs. time in a water channel after a storm event given that I know the permeability of the material in the channel, total precipitation, ...
1
vote
1answer
54 views

Recurrent neural network based language model - how word-vector w(t) is represented in this approach?

I'm reading "Recurrent neural network based language model" of Mikolov et al. (2010). Although the article is straight forward, I'm not sure how word vector w(t) is obtained (printscreen from PDF ...
2
votes
0answers
226 views

Tensorflow: Can't overfit training data with batch size > 1

I coded a small RNN network with Tensorflow to return the total energy consumption given some parameters. There seem to be a problem in my code. It can't overfit the training data when I use a batch ...
-3
votes
1answer
60 views

How to train recurrent neural network? [closed]

I need to have a full brief on recurrent neural network. With the explanation how to train recurrent neural network??
-2
votes
1answer
290 views

How to train a simple neural network to create source code

I am thinking of making a simple and interesting research on training a RNN, to create source codes. The objective is to have a set of simple REPL programs in java, and create a source code for a ...