Questions tagged [recurrent-neural-networks]

Use for questions about Recurrent Neural Networks

Filter by
Sorted by
Tagged with
0
votes
2answers
20 views

How can neural networks be used to generate multiple outputs?

As far I've been using Neural Nets, I've used them to take input vectors and return yes/no like the output. But, here in a video, https://youtu.be/ajGgd9Ld-Wc?t=214, Kai Fu Lee, renowned AI Expert ...
0
votes
1answer
39 views

Emotional Speech Synthesis

We are a team of computer science our graduation project about EmotionalSpeech Synthesis. We've found valuable information like research papers and WaveNet, Tacotron. A website (https://www.voicery....
1
vote
1answer
48 views

Recurrent Neural Network to track distance from origin

I have a game/simulation that takes a vector of encoded sequences of moves (up, down, left, right). Let's say that these are sequential step taken by an ant moving in a 2D space, starting from the ...
2
votes
1answer
54 views

Issue at training simple RNN for word generation

After completing Coursera course from Andrew Ng, I wanted to implement again simple RNN for generating dinosaurs name based on a text file containing around 800 dinosaurs name. This is done with ...
0
votes
0answers
7 views

Why word embedding such as word2vec is not used as the output layer of a seq2seq decoder?

It would make sense to make the decoder predict a smaller embedding vector instead of softmax over a large dictionary. The word having the most cosine similarity with the output embedding could be ...
4
votes
1answer
30 views

dimensions of hidden layer and cell state layer in LSTM

I was following some examples to get familiar with tensorflow LSTM related api, but noticed that all LSTM initialization functions require only num_units parameter which denotes number of hidden units ...
0
votes
1answer
19 views

How to change this RNN text classification code to become text generation code?

I can do text classification with RNN, in which the last output of RNN (rnn_outputs[-1]) is used to matmul with output layer weight and plus bias. That is getting a word (class name) after the last T ...
3
votes
2answers
298 views

Where can I find pre-trained language models in English and German?

Where can I find (more) pre-trained language models? I am especially interested in neural network based models for English and German. And I specifically mean language model in its standard sense. I ...
0
votes
0answers
17 views

Better versions of LSTM

I know recurrent neural networks for example LSTM is one of them. After LSTM different variants of LSTM have come for example GRU. I don't know about newer RNN like LSTM and GRU, so I decided to ask ...
0
votes
1answer
29 views

How to map X to Y for TensorFlow RNN training data

Usually for DNN, I have the training data of matching X (2D) to Y (2D), for example, XOR data: X = [[0,0],[0,1],[1,0],[1,1]]; Y = [[0], [1], [1], [0] ]; ...
1
vote
2answers
75 views

Which algorithm should I use to map an input sentence to an output sentence?

I am new to NLP realm. If you have an input text "The price of orange has increased" and output text "Increase the production of orange". Can we make our RNN model to predict the output text? Or what ...
1
vote
2answers
84 views

What are some examples of LSTM architectures?

I've been doing some class assignments recently on building various neural networks. For convolutional networks, there are several well-known architectures such as LeNet, VGG etc. Such "classic" ...
2
votes
1answer
32 views

How are the observations stored in the RNN that encodes the state?

I am a bit confused about observations in RL systems which use RNN to encode the state. I read a few papers like this and this. If I were to use a sequence of raw observations (or features) as an ...
2
votes
2answers
243 views

What is the significance of this Stanford University “Financial Market Time Series Prediction with RNN's” paper?

Researchers at Stanford University released this paper in 2012: http://cs229.stanford.edu/proj2012/BernalFokPidaparthi-FinancialMarketTimeSeriesPredictionwithRecurrentNeural.pdf It goes on to ...
2
votes
2answers
35 views

Conferences for Human Activity Recognition

What are some conferences for publishing papers on Deep Learning for Human Activity recognition? Do any of the major conferences have specific tracks for Human Activity Recognition?
1
vote
1answer
44 views

Is there a rigorous proof for finding Hopfield minima?

I am looking for a rigorous mathematical proof for finding the several local minima of the Hopfield networks. I am searching for something rigorous, a demonstration, not just let the network keep ...
1
vote
0answers
23 views

Is there an standard algorithm for giving options from an RNN?

You can feed books to an RNN and it learn how to produce text. What I'm interested in is an algorithm that, given say 20 letters it suggest, say the best 10 options for the next 10 letters. So for ...
1
vote
0answers
20 views

Literature on Sequence Regresssion

I have some rated time-sequential data and I would like to test if an ANN can learn a correlation between my measurements and ratings. I suspect I could just try a CNN where 1 Dimension is time or an ...
1
vote
1answer
45 views

What is an identity recurrent neural network?

What is an identity recurrent neural network (IRNN)? What is the difference between an IRNN and RNN?
1
vote
0answers
14 views

A gated neural network for internal thought?

I have an idea for an RNN which has no separate internal memory state only an output. But there is a gate in which tells the neural network whether the output will be acted out in the physical world ...
2
votes
1answer
40 views

How can I keep context in my chatbot

I have created a chatbot by Keras based on movie dialog. I used RNN more specifically GRU . My bot can reply well. But the problem is , it can't hold the context . As an example if I say ...
0
votes
1answer
60 views

How can I stabilise a recurrent neural network used for binary classification?

I’m looking for some help with my neural network. I’m working on a binary classification on a recurrent neural network that predicts stock movements (up and down) Let’s say I’m studying Eur/Usd, I’m ...
4
votes
2answers
90 views

Can we optimize an optimization algorithm?

In this answer to the question Is an optimization algorithm equivalent to a neural network?, the author stated that, in theory, there is some recurrent neural network that implements a given ...
3
votes
2answers
123 views

How do layers in an artificial neural network transform inputs to outputs?

To me, most ANN/RNN related articles don't tell me actually how the network is implemented. I know that in the ANN you'll have multiple neurons, activation function, weights, etc. But, how do you, ...
1
vote
1answer
44 views

Are there neural networks that accept graphs or trees as inputs?

As far I know, the RNN accepts a sequence as input and can produce as a sequence as output. Are there neural networks that accept graphs or trees as inputs, so that to represent the relationships ...
0
votes
0answers
44 views

Why doesnt my lstm model for time series prediction improve after certain level of performance?

I created an lstm model which predicts multioutput sequeances. It takes variable length sequences as input. These sequences are padded with zero to obtain equal length. Note that the time series are ...
6
votes
3answers
71 views

In sequence-to-sequence, why is the output of the decoder used as its input?

The basic seq-2-seq model consists of 2 parts: a recurrent encoder that compresses a sequence to a vector and decoder that unrolls the vector into the output sequence: Why is the output, ...
0
votes
0answers
17 views

Dynamic frames processing with CNN LSTM combination or otherwise

I have a unique implementation where I have to process videos with dynamic frame rates (that is the number of frames is different for each video in a batch). I am stacking all the frames in a single ...
2
votes
0answers
140 views

Preprocessing of training dataset for machine learning

I'm developing a log analyzer to predict and find errors in an equipment. Each logged data contains the following format: ...
2
votes
1answer
188 views

Additive Attention in Convolutional Networks

Attention has been used widely in recurrent networks to weight feature representations learned by the model. This is not a trivial task since recurrent networks have a hidden state that captures ...
0
votes
1answer
73 views

how to predict the stock of food items using ai

Actually, I want to make an AI model which tells the seller about the maintaining stock of food items as a parameter of time and eventually learns by itself with the customer buying data (problem - ...
3
votes
2answers
99 views

What is the relation between Convolutional Neural Networks and Recurrent Neural Networks?

I asked my self this simple question while reading "Comment Abuse Classification with Deep Learning" by Chu and Jue. Indeed, they say at the end of the that It is clear that RNNs, specifically ...
0
votes
1answer
15 views

How to handle proper names or variable names in word2vec?

The input in word2vec is known word (spellings), each tagged by its ID. But if you process real text, there can be not only dictionary words but also proper nouns like human names, trade marks, file ...
2
votes
1answer
26 views

Changes in flow detection neural network?

Do you have any advice, what architecture of neural network is the best for following task? Let input be some (complex function), the neural network gains a flow of its values, so I guess there will ...
1
vote
1answer
30 views

Train and Test Accuracy of GRU network not increasing after 2nd epoch

So I´m currently implementing my first neural network using GRUs as a model and Keras as an implementation since it´s pretty highlevel. My problem is about the classification of 8 hour long timeseries ...
1
vote
1answer
107 views

Do you need to store prevous values of weights and layers on recurrent layer while BPTT?

The Back propagation through time on recurrent layer is defined similar to normal one, means somethin like ...
1
vote
0answers
39 views

RNN weights when varying the input size

I have a time-varying input size vector for a RNN. However, I am facing some difficulties understanding how to deal with my network weights when the input changes. Say we have a set of natural ...
0
votes
0answers
66 views

How Seq2Seq with Bidirectional RNN works?

First of all the scope of the question is as follows - we have Sequence2Sequence architecture with: Decoder: Bidirectional LSTM Encoder: regular (single directional) LSTM What I know: When you ...
7
votes
4answers
4k views

What is the fundamental difference between CNN and RNN?

What is the fundamental difference between convolutional neural networks and recurrent neural networks? Where are they applied?
12
votes
2answers
2k views

What is a Recurrent Neural Network?

Surprisingly this wasn't asked before - at least I didn't find anything besides some vaguely related questions. So, what is a recurrent neural network, and what are their advantages over regular NNs?
0
votes
2answers
101 views

Time series RNN vs DNN

Understandably RNNs are very good at solving problems involving audio, video and text processing due to arbitrary input length of this sort of data. What I don't understand is why RNNs are also ...
28
votes
4answers
38k views

How to select number of hidden layers and number of memory cells in an LSTM?

I am trying to find some existing research on how to select the number of hidden layers and the size of these of an LSTM-based RNN. Is there an article where this problem is being investigated, i.e., ...
3
votes
1answer
198 views

A mathematical explanation of Attention Mechanism

I am trying to understand why attention models are different than just using neural networks. Essentially the optimization of weights or using gates for protecting and controlling cell state (in ...
4
votes
1answer
347 views

Over- and underestimations of the lowest and highest values in LSTM network

I'm training a LSTM network with multiple inputs and several LSTM layers in order to setup a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation on the ...
2
votes
1answer
198 views

What is a state in a recurrent neural network?

I am Reading "Supervised Sequence Labelling with Recurrent Neural Networks" written by Alex Graves to try to understand LSTM networks and I am a bit confused about the equations. Specifically, what I ...
1
vote
2answers
234 views

How to use RNN With Attention Mechanism on Non Textual Data?

Recurrent Neural Networks (RNN) With Attention Mechanism is generally used for Machine Translation and Natural Language Processing. In Python, implementation of RNN With Attention Mechanism is ...
1
vote
0answers
38 views

What is the feasible neural network structure that can learn to identify types of trajectory of moving dots?

I have multiple image sequences, each of which contains an animation of two moving dots. The trajectory of the dots in a sequence is always cyclic (not necessarily circular). There are two types of ...
1
vote
1answer
70 views

How is the word embedding represented in the paper “Recurrent neural network based language model”?

I'm reading "Recurrent neural network based language model" of Mikolov et al. (2010). Although the article is straight forward, I'm not sure how word embedding $w(t)$ is obtained: The reason I wonder ...
1
vote
0answers
70 views

How does bidirectional encoding allow the predicted word to indirectly “see itself”?

Before the release of BERT we used to say that it is not possible to train bidirectional models by simply conditioning each word on its previous and next words, since this would allow the word that’s ...
0
votes
0answers
15 views

Using the cloud service to trasform a picture using a neural algorithm?

yesterday I tried to transform a picture in the artistic style using CNNs based on A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge using a recent Torch ...