Questions tagged [recurrent-neural-networks]

For questions related to recurrent neural networks (RNNs), artificial neural networks that contain backward or self-connections, as opposed to just having forward connections, like in a feed-forward neural network. An RNN can be trained using back-propagation through time, such that these backward connections "memorize" previously seen inputs. Consequentially, RNNs are well suited to sequence prediction and similar tasks.

Filter by
Sorted by
Tagged with
2
votes
0answers
54 views

How do I train a multiple-speaker model (speech synthesis) based on Tacotron 2 and espnet?

I'm new to Speech Synthesis & Deep Learning. Recently, I got a task as described below: I have problem in training a multi-speaker model which should be created by Tacotron2. And I was told I can ...
1
vote
1answer
20 views

What is the appropriate RNN structure to do Sentiment Analysis with multiple dependent ratings?

Suppose we are doing sentiment analysis for a restaurant. Customers can rate the restaurant by #1: how expensive the restaurant is, #2:how good is the food and #3: how likely they will come again. The ...
1
vote
1answer
21 views

Convert input dataset given in hex addresses to int

I have created an LSTM Neural Network which take as input the following format in an .csv file ...
1
vote
0answers
15 views

Is there a way to use RNN (in tensorflow) to do something like a batch Kalman with the weight dynamics specified in the loss?

Or would you simply do this as a time series of models. Basically I think you can think of time series of weights as the hidden states and the dynamics driving the weight time series as the RNN ...
1
vote
0answers
32 views

How to exploit translational symmetry for extrapolation in video generation using machine learning

I'll try to rephrase my problem in the context of video processing. Imagine that initial frame of video has some translational symmetry. The frame evolves according to an update rule. I generate a ...
0
votes
1answer
96 views

How to process data in a data stream for a LSTM

How can a data stream for a RNN (LSTM) be handled, when the stream contains data sets belonging to different prediction classes? Training phase: I have trained a LSTM to predict a class out of a ...
1
vote
0answers
10 views

How do I build a multi RNN network with keras?

I have 2 (independently long) sequences (a and b) of feature vectors that I want to use as input for a neural network. The idea was to build 2 GRU based encoders (one for each sequence). I would than ...
2
votes
0answers
27 views

How do I determine the best neural network architecture for a problem with 3 inputs and 12 outputs?

This post continues the topic in the following post: Is it possible to train a neural network with 3 inputs and 12 outputs?. I conducted several experiments in MATLAB and selected those neural ...
4
votes
0answers
60 views

RNN models displays upper limit on predictions

I have trained a RNN, GRU, and LSTM on the same dataset, and looking at their respective predictions I have observed, that they all display an upper limit on the value they can predict. I have ...
4
votes
1answer
44 views

Training an RNN to answer simple quesitons

I would like to train an RNN to follow the sentences: "Would you like some cheese"? with "Yes, I would like some cheese." So whenever the template "Would you like some ____?" appears then RNN ...
3
votes
0answers
16 views

How to back propagate for implementation of Sequence-to-Sequence with Multi Decoders

I am proposing a modified version of Sequence-to-Sequence model with dual decoders. The problem that I am trying to solve is Neural Machine Translation into two languages at once. This is the ...
2
votes
0answers
48 views

Is there a recurrent neural network where the output becomes a partial input?

I am aware of the way RNN works (finite and infinite impulse) and I have seen a lot of use (e.g. in speech recognition). I have understood it is used to "store" value and/or re-use them. But I am ...
1
vote
0answers
21 views

What is the term for an RNN that is a completely connected directed graph?

There seems to be a severe problem with the taxonomy of neural network topologies. What I'd like to know is the term I should use to search for the most general topology: completely connected ...
0
votes
0answers
29 views

Outliers detection problem in neural networks

Assuming we have big m x n input dataset with m x 1 output vector. It's a classification problem with only two possible values: either 1 or 0. Now the problem is that almost all elements of the output ...
0
votes
0answers
28 views

How can I write out the Real-TIme Recurrent Learning Gradient equations for a network?

This question is about Real-Time Recurrent Learning Gradient on a Recurrent neural network . How can I write out the RTRL equations for a network ? Before present an example give let's introduce ...
2
votes
0answers
22 views

How are batch statistics computed in Recurrent Batch Normalization?

I'm implementing recurrent BN per this paper in Keras, but looking at it and those citing it, a detail remains unclear to me: how are batch statistics computed? Authors omit explicit clarification, ...
1
vote
0answers
25 views

Imposing contraints on sequence of image classifications

Are there example implementations of networks that apply constraints across sequences of image classifications where class labels are ordinal numbers? For example, to cause the output of a CNN to ...
1
vote
0answers
35 views

What is the correct input shape for my LSTM network?

My professor gave us a workshop where we have to do classification of a dataset of ECG signals between healthy and unhealthy types using LSTM. Each signal consists of 1,285 time steps. What my prof ...
1
vote
1answer
22 views

Can a character-level Seq2Seq setup learn to perfectly reconstruct structured data like name strings?

If not perfect, how well can they do? For example, if I give the Seq2Seq setup a name it did not see in the training process, can it output the same name without error? Example ...
2
votes
0answers
36 views

Difference in the code structure of RNN and CNN

I understand that in general RNN is good for time series data and CNN image data, and have noticed many blogs explaining the ...
2
votes
0answers
76 views

What is a location-based addressing in a neural Turing machine?

In the neural Turing machine (NTM), the content-based addressing and location-based addressing is used for memory addressing. Content-based addressing is similar to the attention-based model, ...
0
votes
1answer
47 views

Do we have anything like accuracy and loss in RNN models?

I have a paper about trading which has been implemented with RNN on Tensorflow. We have about 2 years of data from trading. Here are some samples : Date, Open, High, Low, Last, Close, Total Trade ...
4
votes
0answers
94 views

What evaluation metric are used for sequence-to-sequence prediction problems?

I am solving many sequence-to-sequence prediction problems using RNN/LSTM. What type of evaluation metrics can be used for sequence prediction problems? One metric is the mean squared error (MSE) ...
3
votes
0answers
49 views

Ideas on a network that can translate image differences into motor commands?

I'd like to design a network that gets two images (an image under construction, and an ideal image), and has to come up with an action vector for a simple motor command which would augment the image ...
2
votes
1answer
84 views

Why can't LSTMs tell a long story?

There is a recent trend in people using LSTMs to write novels. I haven’t attempted this myself. From what I’m hearing, they can tell a story, but it seems they lose the context of the story rather ...
8
votes
2answers
755 views

Why use a recurrent neural network over a feedforward neural network for sequence prediction?

If recurrent neural networks (RNNs) are used to capture prior information, couldn't the same thing be achieved by a feedforward neural network (FFNN) or multi-layer perceptron (MLP) where the inputs ...
5
votes
1answer
70 views

Why do small datasets require more samples, while big datasets require fewer samples in negative sampling?

In the deep learning specialization course by Andrew Ng, in the video Sequence Models (minute 4:13), he says that in negative sampling we have to choose a sample of words from the corpus to train ...
2
votes
2answers
72 views

What is hidden state exactly in LSTM and RNN?

I'm working on research rn using LSTM as an encoder decoder in hopes to make inferences. The reason we are using encoder decoder for this is because there is hopes that the hidden state given by the ...
3
votes
1answer
51 views

Why RNNs often use just one hidden layer?

Did I get it right, that RNNs most often have just one hidden neuron layer? Is there a reason for that? Will RNNs with several hidden layers in each cell work worse? Thank you!!
0
votes
0answers
53 views

Why is the value range of my LSTM model's prediction different from my test labels?

I am using LSTM to do time-series anomaly detection. The data is an hourly sensor input across multiple years (i.e. the global_active_energy attribute of the dataset from https://www.kaggle.com/uciml/...
1
vote
2answers
97 views

Necessity of GPUs and hardware minimum specs for Deep Learning?

I’m doing some research into what hardware I need and what hardware I have available in college for a final year project. The project is designing a self driving car/computer vision system inside a ...
2
votes
2answers
99 views

How can neural networks be used to generate rather than classify?

In my experience with Neural Nets, I have only used them to take input vectors and return binary output. But, here in a video, https://youtu.be/ajGgd9Ld-Wc?t=214, Kai Fu Lee, renowned AI Expert shows ...
0
votes
1answer
72 views

Emotional Speech Synthesis

We are a team of computer science our graduation project about EmotionalSpeech Synthesis. We've found valuable information like research papers and WaveNet, Tacotron. A website (https://www.voicery....
1
vote
1answer
65 views

Recurrent Neural Network to track distance from origin

I have a game/simulation that takes a vector of encoded sequences of moves (up, down, left, right). Let's say that these are sequential step taken by an ant moving in a 2D space, starting from the ...
0
votes
0answers
11 views

Why word embedding such as word2vec is not used as the output layer of a seq2seq decoder?

It would make sense to make the decoder predict a smaller embedding vector instead of softmax over a large dictionary. The word having the most cosine similarity with the output embedding could be ...
5
votes
1answer
539 views

What is the relationship between the size of the hidden layer and the size of the cell state layer in an LSTM?

I was following some examples to get familiar with TensorFlow's LSTM API, but noticed that all LSTM initialization functions require only the num_units parameter, ...
0
votes
0answers
27 views

Better versions of LSTM

I know recurrent neural networks for example LSTM is one of them. After LSTM different variants of LSTM have come for example GRU. I don't know about newer RNN like LSTM and GRU, so I decided to ask ...
0
votes
1answer
21 views

How to change this RNN text classification code to become text generation code?

I can do text classification with RNN, in which the last output of RNN (rnn_outputs[-1]) is used to matmul with output layer weight and plus bias. That is getting a word (class name) after the last T ...
0
votes
1answer
32 views

How to map X to Y for TensorFlow RNN training data

Usually for DNN, I have the training data of matching X (2D) to Y (2D), for example, XOR data: X = [[0,0],[0,1],[1,0],[1,1]]; Y = [[0], [1], [1], [0] ]; ...
3
votes
1answer
83 views

Issue at training simple RNN for word generation

After completing Coursera course from Andrew Ng, I wanted to implement again simple RNN for generating dinosaurs name based on a text file containing around 800 dinosaurs name. This is done with ...
2
votes
2answers
402 views

What are some examples of LSTM architectures?

I've been doing some class assignments recently on building various neural networks. For convolutional networks, there are several well-known architectures such as LeNet, VGG etc. Such "classic" ...
3
votes
2answers
74 views

Conferences for Human Activity Recognition

What are some conferences for publishing papers on Deep Learning for Human Activity recognition? Do any of the major conferences have specific tracks for Human Activity Recognition?
1
vote
0answers
23 views

Is there an standard algorithm for giving options from an RNN?

You can feed books to an RNN and it learn how to produce text. What I'm interested in is an algorithm that, given say 20 letters it suggest, say the best 10 options for the next 10 letters. So for ...
1
vote
0answers
21 views

Literature on Sequence Regresssion

I have some rated time-sequential data and I would like to test if an ANN can learn a correlation between my measurements and ratings. I suspect I could just try a CNN where 1 Dimension is time or an ...
1
vote
1answer
216 views

What is an identity recurrent neural network?

What is an identity recurrent neural network (IRNN)? What is the difference between an IRNN and RNN?
3
votes
1answer
56 views

How are the observations stored in the RNN that encodes the state?

I am a bit confused about observations in RL systems which use RNN to encode the state. I read a few papers like this and this. If I were to use a sequence of raw observations (or features) as an ...
1
vote
0answers
15 views

A gated neural network for internal thought?

I have an idea for an RNN which has no separate internal memory state only an output. But there is a gate in which tells the neural network whether the output will be acted out in the physical world ...
2
votes
1answer
67 views

How can I keep context in my chatbot

I have created a chatbot by Keras based on movie dialog. I used RNN more specifically GRU . My bot can reply well. But the problem is , it can't hold the context . As an example if I say ...
4
votes
3answers
109 views

Can we optimize an optimization algorithm?

In this answer to the question Is an optimization algorithm equivalent to a neural network?, the author stated that, in theory, there is some recurrent neural network that implements a given ...
1
vote
1answer
76 views

Is there a rigorous proof for finding Hopfield minima?

I am looking for a rigorous mathematical proof for finding the several local minima of the Hopfield networks. I am searching for something rigorous, a demonstration, not just let the network keep ...