Questions tagged [recurrent-neural-networks]

For questions related to recurrent neural networks (RNNs), artificial neural networks that contain backward or self-connections, as opposed to just having forward connections, like in a feed-forward neural network. An RNN can be trained using back-propagation through time, such that these backward connections "memorize" previously seen inputs. Consequentially, RNNs are well suited to sequence prediction and similar tasks.

Filter by
Sorted by
Tagged with
4
votes
1answer
545 views

Over- and underestimations of the lowest and highest values in LSTM network

I'm training a LSTM network with multiple inputs and several LSTM layers in order to setup a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation on the ...
3
votes
1answer
345 views

What is the significance of this Stanford University “Financial Market Time Series Prediction with RNN's” paper?

Researchers at Stanford University released, in 2012, the paper Financial Market Time Series Prediction with Recurrent Neural Networks. It goes on to discuss how they used echo state networks to ...
1
vote
0answers
28 views

Whats advantages does a Loop Network have over a Feed Forward Network?

I am interested to see what advantages a Loop Network (Feed Forward Network that takes its output as input, I think it's called an RNN, not sure). The only result I found was that it was extremely ...
1
vote
1answer
57 views

How to manage high numbers of input layer data points

Not sure if this is the correct forum, but I have been working with a large (non-image) dataset that will eventually be used to train a neural network. I have been puzzling over how to manage wide ...
2
votes
0answers
101 views

Combine two embeddding inputs to increase more performance in LSTM model

The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second ...
1
vote
2answers
126 views

Time Series: LSTM or Augmented Vector Space?

In time Series prediction, we have a stream of vectors. There are different approaches for accounting for the temporal patterns between these vectors. There's two that I'm considering. An LSTM or ...
1
vote
0answers
121 views

Best practices to classify recurring patterns using an LSTM or GRU

I'm working with acoustic data (filterbank features) and I want to build a neural network to detect claps using an LSTM (or a GRU) with a binary output (present/abscent), and I'm wondering about how I ...
1
vote
1answer
1k views

How to calculate the output of this neural network?

What is the output value of the network for these inputs respectively, and why? (Linear activation function is fine.) [2, 3][-1, 2][1, 0][3, 4] My main question is how you take the 'backwards' ...
4
votes
2answers
111 views

Do convolutional neural networks also have recurrent connections? [duplicate]

I asked my self this simple question while reading "Comment Abuse Classification with Deep Learning" by Chu and Jue. Indeed, they say at the end of the that It is clear that RNNs, specifically ...
3
votes
1answer
1k views

Seq2Seq dialogs predicts only most common words like `you` after couple of epoches

I'm training Seq2Seq model on OpenSubtitles dialogs - Cornell-Movie-Dialogs-Corpus. My work based on the following papers (but currently I'm not implemented Attention yet): Sequence to Sequence ...
4
votes
3answers
148 views

What kinds of systems have so far failed to be modeled via supervised artificial network training?

Artificial networks model systems with a set of inputs and outputs and expected behavior. To train a network for modeling such systems, hundreds, thousands, or millions of example inputs-output pairs ...
1
vote
0answers
169 views

Trajectory classification using RNN

The problem: I want to classify a trajectory if it has some properties, for example I want to create a simple 0/1 classifier for circular trajectories. If a target is moving in a circular trajectory ...
1
vote
0answers
32 views

How to train a recurrent neural network with multiple series

I am new to neural networks. I am trying to model the run-off vs. time in a water channel after a storm event given that I know the permeability of the material in the channel, total precipitation, ...
1
vote
1answer
93 views

How is the word embedding represented in the paper “Recurrent neural network based language model”?

I'm reading "Recurrent neural network based language model" of Mikolov et al. (2010). Although the article is straight forward, I'm not sure how word embedding $w(t)$ is obtained: The reason I wonder ...
1
vote
0answers
263 views

Tensorflow: Can't overfit training data with batch size > 1

I coded a small RNN network with Tensorflow to return the total energy consumption given some parameters. There seem to be a problem in my code. It can't overfit the training data when I use a batch ...
-2
votes
1answer
132 views

How to train recurrent neural network? [closed]

I need to have a full brief on recurrent neural network. With the explanation how to train recurrent neural network??
11
votes
5answers
7k views

What is the fundamental difference between CNN and RNN?

What is the fundamental difference between convolutional neural networks and recurrent neural networks? Where are they applied?
0
votes
1answer
168 views

What's wrong with getting a dynamic RNN's output at the end of a padded sequence?

Would it not even be more helpful? It will have more time to transform the input with the extra zeros padded to the end.
3
votes
0answers
166 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in The Unreasonable Effectiveness of Recurrent Neural Networks) that when training RNN on text for something like language modeling, the text is usually featurized character-by-...
7
votes
1answer
195 views

Neural network design when amount of input neurons vary

I'm looking to design a neural network that can predict which runner wins in a sports game, where the amount of runners varies between 2-10. In each case, specific data about the individual runners ...
2
votes
0answers
43 views

Detecting symmetry in small images with RNN

My network works on 32x32 normalized (translationally) but noisy images. Its task it to determine whether image has simple symmetry (horizontal/vertical). It needs to be reasonably robust to rotation (...
2
votes
0answers
47 views

Recommendations on which architecture to use to guess appointment

I'm currently developping an application which allows psychologists to manage their schedule and budget. As a proof of concept, I would like to create an intelligent appointment service. There can be ...
1
vote
0answers
25 views

Can one use a trained recurrent neural network to find the most probable sequence of a set?

Let's say I train an RNN on the sequence of characters in War and Peace. After doing so is there a way I can take an arbitrary set of letters, e.g. HNTASK, and have it (ideally) tell me that the most ...
2
votes
0answers
54 views

Are gradients of weights in RNNs dependent on the gradient of every neuron in that layer?

I am writing my own recurrent neural network in Java to understand the inner workings better. While working through the math, I found that in timesteps later than 2 the gradient of weight w of neuron ...
2
votes
0answers
70 views

seq2seq vector to letters model

I'm looking to build a sequence-to-sequence model that takes in a 2048-long vector of 1s and 0s as my input and translating it to my known output of (a variable length) 1-20 long characters (ex. ...
5
votes
2answers
4k views

Spam Detection using Recurrent Neural Networks

I am working on this code for spam detection using recurrent neural networks. Question 1. I am wondering whether this field (using RNNs for email spam detection) worths more researches or it is a ...
2
votes
0answers
148 views

Preprocessing of training dataset for machine learning

I'm developing a log analyzer to predict and find errors in an equipment. Each logged data contains the following format: ...
3
votes
1answer
876 views

What is a state in a recurrent neural network?

I am Reading "Supervised Sequence Labelling with Recurrent Neural Networks" written by Alex Graves to try to understand LSTM networks and I am a bit confused about the equations. Specifically, what I ...
43
votes
4answers
64k views

How to select number of hidden layers and number of memory cells in an LSTM?

I am trying to find some existing research on how to select the number of hidden layers and the size of these of an LSTM-based RNN. Is there an article where this problem is being investigated, i.e., ...
1
vote
1answer
3k views

Neural network algorythms without any libraries [closed]

I am a php developer learning python for one reason, i wanna learn ai and i think that python would be better than php at that. I tried finding tutorials on how to build a neural network but they all ...
1
vote
0answers
98 views

Infer dependent variables to produce output aligned to trained data

Hypothetical example, say I wanted: P(gender,ethnicity|age,hair); so that the input would aligned to a trained dataset of: ...
5
votes
1answer
103 views

Are neurons instantly feed forward when input arrives?

Let's say I have a neural network with 5 layers, including the input and output layer. Each layer has 5 nodes. Assume the layers are fully connected, but the 3rd node in the 2nd layer is connected to ...
2
votes
1answer
412 views

Good books to read on Artificial/Recurrent Neural Networks? [closed]

A professor and I have been learning about artificial neural networks. We have a pretty good idea of the basics- backpropagation, convolutional networks, and all that jazz. We finished one book and ...
11
votes
2answers
4k views

How to train a chatbot

I wanted to started experimenting with neural network and as a toy problem I wished to train one to chat, i.e. implement a chatting bot like cleverbot. Not that clever anyway. I looked around for ...
9
votes
4answers
750 views

Are we technically able to make, in hardware, arbitrarily large neural networks with current technology?

If neurons and synapses can be implemented using transistors, what prevents us from creating arbitrarily large neural networks using the same methods with which GPUs are made? In essence, we have ...
4
votes
1answer
1k views

How can I understand this statement about RNNs and hidden layers?

In the lecture, there was a statement: Recurrent neural networks with multiple hidden layers are just a special case that has some of the hidden to hidden connections missing. I understand ...
17
votes
1answer
395 views

Could a Boltzmann machine store more patterns than a Hopfield net?

This is from a closed beta for AI, with this question being posted by user number 47. All credit to them. According to Wikipedia, Boltzmann machines can be seen as the stochastic, generative ...
-4
votes
2answers
526 views

What are the approaches to predict sequence of π numbers? [closed]

Given list of fixed numbers from a mathematical constant such as Pi, is it is possible to train AI to attempt to predict the next numbers? Which AI or neural network would be more suitable for this ...
16
votes
2answers
272 views

How do I decide the optimal number of layers for a neural network?

How do I decide the optimal number of layers for a neural network (feedforward or recurrent)?
2
votes
2answers
260 views

Can we ever achieve hypercomputation using recurrent neural networks?

It is proved that a recurrent neural net with rational weights can be a super-Turing machine. Can we achieve this in practice ?

1 2 3 4
5