Questions tagged [recurrent-neural-networks]

For questions related to recurrent neural networks (RNNs), artificial neural networks that contain backward or self-connections, as opposed to just having forward connections, like in a feed-forward neural network. An RNN can be trained using back-propagation through time, such that these backward connections "memorize" previously seen inputs. Consequentially, RNNs are well suited to sequence prediction and similar tasks.

Filter by
Sorted by
Tagged with
1
vote
0answers
13 views

Do RNN solves the need for LSTM and/or multiple states in Deep Q-Learning?

Introduction I am trying to setup a Deep Q-Learning agent. I have looked that the papers Playing Atari with Deep Reinforcement Learning as well as Deep Recurrent Q-Learning for Partially Observable ...
1
vote
0answers
18 views

How do we minimize loss for a single neuron with a feedback?

Suppose we had a series of single-dimensional data points $X = \{x_1, x_2, \dots, x_n \}$, where $n$ is the number of data points and there corresponding output values $T = \{t_1, t_2, \dots, t_n \}$. ...
3
votes
0answers
17 views

Is there any way of generating fixed-length sequences with RNNs?

Is there any way of generating fixed-length sequences with RNNs? I want to tell my character level RNN to generate a name of length 3, 4, 5 and so on. I haven't found anything online like this, but my ...
2
votes
0answers
27 views

How to predict an event (or action) based on a window of time-series measurements?

I have an input vector $X$, which contains a series of measurements within a period, e.g. 100 measurements in 1 sec. The goal is to predict an event, let's say, moving forward, backward or static. I ...
0
votes
1answer
22 views

Which NLP application that supports recurrent neural network?

Some of the NLP applications taken from this link NLP Applications: Machine Translation Speech Recognition Sentiment Analysis Question Answering Automatic Summarization Chatbots Market Intelligence ...
1
vote
0answers
49 views

Why do momentum techniques not work well for RNNs?

AFAIK, momentum is quite useful when training CNNs, and can speed-up the training substantially without any drop in validation accuracy. I've recently learned that it is not as helpful for RNNs ...
1
vote
1answer
17 views

Transformer for speech recognition?

(1) Are there examples that transformer have better accuracy than RNN end-to-end model like RNN-transducer for speech recognition? (2) Can transformer be used for online speech recognition which ...
0
votes
1answer
12 views

Pros and Cons of Seq2Seq vs Bidirectional RNN

It seems to me that Seq2Seq models and Bidirectional RNNs try to do the same thing. Is that true? Also, when would you recommend one setup over another? Thanks!
0
votes
0answers
9 views

My CTC loss model's loss stagnates and then outputs only blank characters

I am trying to implement BaiDu's DeepSpeech1 in keras using CTC loss, my code is below: ...
1
vote
0answers
11 views

How to pad sequences during training for an encoder decoder model

I've got an encoder-decoder model for character level English language spelling correction, it is pretty basic stuff with a two LSTM encoder and another LSTM decoder. However, up until now, I have ...
1
vote
0answers
14 views

Recurrent neural Network for survival analyses: Dealing with forecast data as feature which can exceed the number of days untill a event occurs

I am building a Recurrent Neural network (LSTM) for predicting the number of days until a Pollen season starts (when the cumulative of the year exceeds X). One of the features I am including in my ...
1
vote
1answer
170 views

What’s the difference between LSTM and RNN?

What's the difference between LSTM and RNN? I know that RNN is a network layer used in neural networks, but what exactly is an LSTM? Is it also a network layer with the same characteristics?
1
vote
0answers
15 views

When stacking LSTM's, should the hidden units increase?

I'm using Weights and Biases to do some hyperparameter sweeping for a supervised sequence-to-sequence problem I'm working on. One thing I noticed is that the sweeps with a gradually increasing number ...
1
vote
0answers
23 views

What is the difference between an generalised estimating equation and a recurrent neural network?

What is the difference between a generalised estimating equation (GEE) model and a recurrent neural network (RNN) model, in terms of what these two models are doing? Apart from the differences in the ...
1
vote
0answers
13 views

Simple sequential model with LSTM which doesn't converge

I'm actually trying to create a sequential neural network in order to translate a "human" sentence in a "machine" sentence understandable by an algorithm. Like It didn't work, I've try to create a NN ...
1
vote
1answer
13 views

how to produce documents like factset blackline?

factset blackline reports essentially can compare two 10-Q SEC filings and show you the difference between the two documents. It highlights added items in green and removed items in red + ...
0
votes
0answers
10 views

What will be the sequence of steps in a human activity recognition model using LSTM?

In the context of these steps detection, tracking, action classification and activity recognition. Which step will be first and further sequence?
0
votes
0answers
14 views

RNN/LSTM with a large amounts of data

I have sequence data that's quite large - 4x65k per sample. I'm interested in doing classification problems. The number of classes is moderate - ~27 or so What is the suggestion for dealing with this ...
1
vote
1answer
41 views

What is teacher forcing?

In the paper Neural Programmer-Interpreters, the authors use the teacher forcing technique, but what exactly is it?
1
vote
1answer
47 views

Are there names for neural networks with a well-defined layer or neuron characteristics?

Are there names for neural networks with a well-defined layer or neuron characteristics? For example, a matrix that has the same number of rows and columns is called a square matrix. Is there an ...
1
vote
0answers
32 views

Can GraphRNN be used with very large graphs?

In the GraphRNN paper, the authors only implement the algorithm up to a graph size of 2k nodes. Would this still work on much larger graphs (on the order of $10^7$)? Or would the computation just ...
0
votes
0answers
21 views

Web stream requests prediction architecture

What's in your opinion the best possible architecture for the following problem ? If you have any code that can be used it would be great . Dataset : 400.000 samples given in hex format in an .xlsx ...
2
votes
0answers
17 views

How do I train a multiple-speaker model (speech synthesis) based on Tacotron 2 and espnet?

I'm new to Speech Synthesis & Deep Learning. Recently, I got a task as described below: I have problem in training a multi-speaker model which should be created by Tacotron2. And I was told I can ...
1
vote
1answer
19 views

What is the appropriate RNN structure to do Sentiment Analysis with multiple dependent ratings?

Suppose we are doing sentiment analysis for a restaurant. Customers can rate the restaurant by #1: how expensive the restaurant is, #2:how good is the food and #3: how likely they will come again. The ...
0
votes
1answer
18 views

Convert input dataset given in hex addresses to int

I have created an LSTM Neural Network which take as input the following format in an .csv file ...
1
vote
0answers
15 views

Is there a way to use RNN (in tensorflow) to do something like a batch Kalman with the weight dynamics specified in the loss?

Or would you simply do this as a time series of models. Basically I think you can think of time series of weights as the hidden states and the dynamics driving the weight time series as the RNN ...
1
vote
0answers
31 views

How to exploit translational symmetry for extrapolation in video generation using machine learning

I'll try to rephrase my problem in the context of video processing. Imagine that initial frame of video has some translational symmetry. The frame evolves according to an update rule. I generate a ...
0
votes
1answer
43 views

How to process data in a data stream for a LSTM

How can a data stream for a RNN (LSTM) be handled, when the stream contains data sets belonging to different prediction classes? Training phase: I have trained a LSTM to predict a class out of a ...
1
vote
0answers
10 views

How do I build a multi RNN network with keras?

I have 2 (independently long) sequences (a and b) of feature vectors that I want to use as input for a neural network. The idea was to build 2 GRU based encoders (one for each sequence). I would than ...
2
votes
0answers
27 views

How do I determine the best neural network architecture for a problem with 3 inputs and 12 outputs?

This post continues the topic in the following post: Is it possible to train a neural network with 3 inputs and 12 outputs?. I conducted several experiments in MATLAB and selected those neural ...
4
votes
0answers
55 views

RNN models displays upper limit on predictions

I have trained a RNN, GRU, and LSTM on the same dataset, and looking at their respective predictions I have observed, that they all display an upper limit on the value they can predict. I have ...
4
votes
1answer
43 views

Training an RNN to answer simple quesitons

I would like to train an RNN to follow the sentences: "Would you like some cheese"? with "Yes, I would like some cheese." So whenever the template "Would you like some ____?" appears then RNN ...
3
votes
0answers
16 views

How to back propagate for implementation of Sequence-to-Sequence with Multi Decoders

I am proposing a modified version of Sequence-to-Sequence model with dual decoders. The problem that I am trying to solve is Neural Machine Translation into two languages at once. This is the ...
2
votes
0answers
45 views

Is there a recurrent neural network where the output becomes a partial input?

I am aware of the way RNN works (finite and infinite impulse) and I have seen a lot of use (e.g. in speech recognition). I have understood it is used to "store" value and/or re-use them. But I am ...
1
vote
0answers
21 views

What is the term for an RNN that is a completely connected directed graph?

There seems to be a severe problem with the taxonomy of neural network topologies. What I'd like to know is the term I should use to search for the most general topology: completely connected ...
0
votes
0answers
26 views

Outliers detection problem in neural networks

Assuming we have big m x n input dataset with m x 1 output vector. It's a classification problem with only two possible values: either 1 or 0. Now the problem is that almost all elements of the output ...
0
votes
0answers
24 views

How can I write out the Real-TIme Recurrent Learning Gradient equations for a network?

This question is about Real-Time Recurrent Learning Gradient on a Recurrent neural network . How can I write out the RTRL equations for a network ? Before present an example give let's introduce ...
2
votes
0answers
22 views

How are batch statistics computed in Recurrent Batch Normalization?

I'm implementing recurrent BN per this paper in Keras, but looking at it and those citing it, a detail remains unclear to me: how are batch statistics computed? Authors omit explicit clarification, ...
1
vote
0answers
25 views

Imposing contraints on sequence of image classifications

Are there example implementations of networks that apply constraints across sequences of image classifications where class labels are ordinal numbers? For example, to cause the output of a CNN to ...
1
vote
0answers
33 views

What is the correct input shape for my LSTM network?

My professor gave us a workshop where we have to do classification of a dataset of ECG signals between healthy and unhealthy types using LSTM. Each signal consists of 1,285 time steps. What my prof ...
1
vote
1answer
22 views

Can a character-level Seq2Seq setup learn to perfectly reconstruct structured data like name strings?

If not perfect, how well can they do? For example, if I give the Seq2Seq setup a name it did not see in the training process, can it output the same name without error? Example ...
2
votes
0answers
35 views

Difference in the code structure of RNN and CNN

I understand that in general RNN is good for time series data and CNN image data, and have noticed many blogs explaining the ...
2
votes
0answers
62 views

What is a location-based addressing in a neural Turing machine?

In the neural Turing machine (NTM), the content-based addressing and location-based addressing is used for memory addressing. Content-based addressing is similar to the attention-based model, ...
0
votes
1answer
45 views

Do we have anything like accuracy and loss in RNN models?

I have a paper about trading which has been implemented with RNN on Tensorflow. We have about 2 years of data from trading. Here are some samples : Date, Open, High, Low, Last, Close, Total Trade ...
4
votes
0answers
41 views

What evaluation metric are used for sequence-to-sequence prediction problems?

I am solving many sequence-to-sequence prediction problems using RNN/LSTM. What type of evaluation metrics can be used for sequence prediction problems? One metric is the mean squared error (MSE) ...
3
votes
0answers
44 views

Ideas on a network that can translate image differences into motor commands?

I'd like to design a network that gets two images (an image under construction, and an ideal image), and has to come up with an action vector for a simple motor command which would augment the image ...
2
votes
1answer
82 views

Why can't LSTMs tell a long story?

There is a recent trend in people using LSTMs to write novels. I haven’t attempted this myself. From what I’m hearing, they can tell a story, but it seems they lose the context of the story rather ...
8
votes
2answers
708 views

Why use a recurrent neural network over a feedforward neural network for sequence prediction?

If recurrent neural networks (RNNs) are used to capture prior information, couldn't the same thing be achieved by a feedforward neural network (FFNN) or multi-layer perceptron (MLP) where the inputs ...
5
votes
1answer
65 views

Why do small datasets require more samples, while big datasets require fewer samples in negative sampling?

In the deep learning specialization course by Andrew Ng, in the video Sequence Models (minute 4:13), he says that in negative sampling we have to choose a sample of words from the corpus to train ...
2
votes
2answers
66 views

What is hidden state exactly in LSTM and RNN?

I'm working on research rn using LSTM as an encoder decoder in hopes to make inferences. The reason we are using encoder decoder for this is because there is hopes that the hidden state given by the ...