Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [neural-networks]

For questions about a neural networks, such as MLPs, CNNs, RNNs, LSTM networks, their variants or any other machine learning components that qualify as a neural networks in that they simulate a complexity handling aspect of biological neural networks in vertebrates.

0
votes
0answers
5 views

Training Joint Embeddings

Im looking for somebody to help me on the right path or in the direction of some research that may be related to a task I'm trying to solve. So lets say we had two labeled datasets, and we want some ...
-1
votes
0answers
19 views

Refactoring tensorflow tf.while loop

Thanks in advance for considering my question. TL;DR: I have a while loop in tensorflow, that I think is causing extreme slowness during training, and would need to refactor. Much appreciated if any ...
0
votes
0answers
5 views

Interpretation of cost behaviour in ensemble method

I am working on a problem in NLP, in mapping a question to a target passage. To solve this problem, I am using a fairly complicated model including attention mechanism. The dataset is quite large and ...
-1
votes
0answers
7 views

Character Embeddings with CNN

I've read about character embeddings as suggested by Zhang, Zhao, and LeCun. These approaches takes a character stream as input. Would an encoder trained on just words, not complete texts, still be ...
1
vote
1answer
26 views

Maximum number of neurons in a layer given number of neurons in previous layer

Consider an extremely complicated feed-forward neural network training example but with no need of computational efficiency or limiting of processing time. What is the maximum number of hidden ...
3
votes
3answers
66 views

Why can't the XOR linear inseparability problem be solved with one perceptron - like this?

Consider a perceptron where $w_0=1$ and $w_1=1$: Now, say we use an activation function $f(x)=1,~for~x=1$$~~~~~~~~~~~~~0, otherwise$ The output is then summarised as: $x_0~~~~~x_1~~~~~w_0*x_0 + ...
0
votes
1answer
14 views

Can a LFSR be approximated by a Neural Network?

I was wondering whether a LFSR could be approximated by a NN (output or current state). We know that a LFSR is called linear in some sort of mathematical sense, but is that true? Considering it ...
1
vote
0answers
28 views

How to encode Azul game state as NN input

Question to NN practicioners. I'd like to encode Azul board game state as an input to NN, let's focus on 2-player variant for a while. There are 5 round "Factories" on the table (7 on picture, ignore ...
1
vote
1answer
8 views

Using an 'operation ID' as a neural network input

Sorry if this is basic or covered elsewhere, I am just starting here and I wasn't able to find an answer, but I might have not been searching for the right thing. So: I am training a neural network ...
2
votes
1answer
35 views

How do randomly initialized neural networks behave?

I am wondering how the output of randomly initialized MLPs and ConvNets behave with respect to their inputs. Can anyone point to some analysis or explanation of this? I am curious about this because ...
0
votes
0answers
11 views

Can you train a deep recurrent neural network layer by layer?

Specifically for Gated Recurrent Unit, and say GRU is "layered" via but suppose it's only 2 layers deep for simplicity, and suppose the "total loss" = $L$ = $\sum l_{t} = \sum error(y^{2}_{t})$ for ...
1
vote
0answers
12 views

Change parameter in Karaboga's code of ABC algorithm

I'm working on a problem and need to use Karaboga's code of the ABC algorithm but I have some questions... Does this formula for calculating a parameter have to be changed: ...
0
votes
0answers
22 views

Attention mechanism

I'm having problems implementing Bahdanau attention mechanism in a custom seq2seq (encoder-decoder) architecture. So far I reached this point: ...
0
votes
0answers
3 views

What is the difference between set of transition samples D and pattern set P in algorithm implementation of NFQ?

Neural Fitted Q iteration Algorithm : What is the difference between the set of transition samples D and pattern set P in algorithm implementation of NFQ in Paper "Neural Fitted Q Iteration - First ...
0
votes
2answers
32 views

Neural network architecture for function approximation

We have convolutional neural networks and recurrent neural networks for analysing images and sequential data, respectively. What is the main architecture used for function approximation? (e.g. a ...
6
votes
1answer
41 views

Are Modular Neural Networks more effective than large, monolithic networks at any tasks?

Modular/Multiple Neural networks (MNNs) revolve around training smaller, independent networks that can feed into each other or another higher network. In principle, the hierarchical organization ...
2
votes
2answers
47 views

Can the same input for a plain neural network be used for a convolutional neural network?

Can the same input for a plain neural network be used for CNNs? Or does the input matrix need to be structured in a different way for CNNs compared to regular NNs?
0
votes
0answers
5 views

LSTM output dimensionality

I am new to LSTMs. When reading the papers and websites about LSTM architecture, there is something I do not get. As I understand it, a single LSTM layer can have multiple LSTM cells (just like a ...
1
vote
0answers
17 views

Implementation of back propagation in python [migrated]

I am trying to implement a back propagation algorithm in python but I am finding that when I check this gradient against an approximated gradient the calculated gradient is wrong. I also found that ...
1
vote
1answer
33 views

Genetic Algorithm vs Particle Swarm Optimization

Which one gives better optimization results? Genetic Algorithm or Particle Swarm Optimization? Can I use them for online tuning problems? Thanks in advance!
0
votes
0answers
14 views

Best machine learning method for system identification

I am currently developing a machine learning for system identification Overall, the steps are: Collecting data from a plant (non-linear plant) Train the ML model to replicate the system The model ...
2
votes
0answers
30 views

How a game playing agent could identify potential objects and proximity?

Most implementations I'm seeing for playing games like Atari (usually similar to DeepMind's work using DQN) have 4 graphical frames of input fed into 3 convolutional layers which are then fed into a ...
1
vote
1answer
20 views

Steps for final Logistic Regression Modal

I am new for machine learning and I am tried to understand basic steps to get final modal of Logistic Regression. I know Logistic Regression is supervisory learning technique. Therefore we want to ...
0
votes
0answers
12 views

Did anyone try topic modelling with neural nets?

I constantly see Latent Dirichlet Allocation (LDA) as a go to technique for topic modelling. It performs okay-ish, but ignores word context and (subjectively) seems outdated. Has anyone implemented ...
4
votes
1answer
33 views

Am I able to visualize the differentiation in backprop as follows?

I'm wondering if I can visualize the backprop process as follows (please excuse me if I have written something terrible wrong). If the loss function $L$ on a neural network represents the function has ...
3
votes
1answer
33 views

Does an advanced Dialogue state tracking eliminate the need of intent classifier and slot filling models in dialogue systems/ chatbots?

I am learning to create a dialogue system. The various parts of such a system are Intent classifier, slot filling, Dialogue state tracking (DST), dialogue policy optimization and NLG. While reading ...
3
votes
1answer
42 views

Neural network architecture for comparison

When someone wants to compare 2 inputs, the most widespread idea is to use a Siamese architecture. Siamese architecture is a very high level idea, and can be customized based on the problem we are ...
-1
votes
0answers
27 views

How to build an AI that suggests source texts automatically when given a debate argument as input?

Take a site like this: http://www.thesquabbler.com/design/ (just an approximate design to explain the concept), where users add debate topics and then for and against arguments to them. The key is ...
0
votes
1answer
58 views

Can BERT be used for sentence generating tasks?

I am a new learner in NLP. I am interested in the sentence generating task. As far as I am concerned, one state-of-the-art method is the CharRNN, which uses RNN to generate a sequence of words. ...
1
vote
0answers
9 views

What does it essentially mean if the neural network has convex error surface?

Suppose if I am building a Linear Regression model with one fully connected layer and a sigmoid with minimizing mean squared error as objective. Why would the error surface be convex? Does finding ...
0
votes
0answers
17 views

Language Models RNN

I am relatively new to recurrent neural networks and it seems like a vast domain. So I want to get my initial footing right. There seems to be a whole lot of applications in this field, but the first ...
0
votes
0answers
7 views

Why does an RNN for text generation benefit from having more nodes than sequence length

If I have an RNN for text generation and want the RNN to learn characterwise, I partition the text to learn into sequences of equal length. Now my logic would be: If I have a sequence length x, then I ...
5
votes
5answers
179 views

Where can I discuss with deep learning beginners?

I did some self-study to learn Neural network, object detection, and deep learning. Now I started implementing YOLOv3. I am looking for some website that I can communicate with people and make friends ...
0
votes
0answers
29 views

How to train a stock trading neural network so that the 'profit' parameter is maximized?

I am watching some beginner level video training on neural networks using Tensorflow / Keras to get a better understanding of how they work and how to best implement them. I have some questions on ...
0
votes
1answer
7 views

Image prediction model when data-set classes have visual similarity

Lets say we have a data-set of all cats and we have to identify the cat breed based on given test image. As, the two different cat breeds have visual similarity can we use existing networks (VGG, ...
1
vote
0answers
19 views

Machine learning approach to facial recognition

First of all i'm very new to the field. maybe my question is a bit too naive of even trivial.. I'm currently trying to understand how can i go about recognizing different faces. Here is what i tried ...
1
vote
1answer
18 views

Neural machine translation, that outputs multiple alternative, ambiguous translations?

Is there neural machine translation methods, that for one input sentence outputs multiple alternative output sentences in that target language. It is quite possible, that sentence in source language ...
0
votes
0answers
17 views

Train a recurrent neural network by concatenating time series. Is it safe?

As the title says, I want to train a Jordan network (i.e. a particular kind of recurrent neural network) using a certain number of time series. Let's say that $x_1, x_2, \ldots x_N$ are $N$ input ...
0
votes
1answer
26 views

How does the internal convolution Neural Network work?

I found the below image of how a CNN works but I don't really understand it. I do understand CNNs (I think) but I find this diagram very confusing. My summarized, simplified understanding : -...
1
vote
0answers
24 views

Use of backpropagation for weight updates in a combination of 2 neural networks

Every neural network updates its weights through back-propagation. How is back-propagation used for updating weights in a combination of 2 or more neural networks (e.g.:CNN-LSTM, GAN-CNN, etc.). For ...
-1
votes
0answers
31 views

Why my initial loss is bigger than the expected?

I am trying to perform a simple binary classification with a neural network on the make_moons dataset. Because of random initialization, I expect the first values to be equally splited between ...
1
vote
3answers
50 views

Features Map convolutional neural network

I have a question about convolutional neural newtork. Consider this image: conv example We have a part of an input matrix and a filter. Ok, now we can do the convolution and the result is a scalar, ...
-2
votes
3answers
93 views

How are artificial neural networks different from normal computers?

I have been given this question that how neural networks are different from normal computers? But after searching alot i still couldnt get the best reason on it which makes my point clear, so how ...
2
votes
1answer
84 views

Significance of depth of a deep neural network

How is a feed-forward neural network with few hidden layers and lots of nodes in those hidden layers different from a network with a lot of hidden layers but relatively lesser nodes in those hidden ...
0
votes
0answers
15 views

How to train a multi inputs deep learning model using every combination of inputs?

I am beginner in deep learning. I want to create a multi inputs CNN model in Keras. The model takes two inputs of images to give the two images class. The two images from differnt datasets that have ...
1
vote
0answers
13 views

Learning Tree Paths when Given Vectors

The problem statement: Mapping from a vector space representation onto a tree structure. Possible solution: Given a word vector as input, produce a path in the tree from the root down to the node ...
4
votes
2answers
51 views

Do GAN's come under Supervised Learning or Unsupervised Learning?

My guess is that they come under supervised learning, as we have labelled dataset of images, but I am not sure as there maybe other aspects in GANs which might come into play in the determination of ...
0
votes
1answer
23 views

Can an artificial network create a rule from rule components?

If an antecedent in a rule involves $m$ two-state features and results in consequences from a set of $n$ possible ones, we have $2^{m+n}$ permutations, which are, in a sense, be categories. If ...
0
votes
0answers
13 views

what should be the chain rule to calculate the weight change for the input layer

for example to calculate the weight change in the hidden layer i am deriving it by the following : - THE FOREWARD PASS: \begin{equation*} \mathtt{\begin{array}{ l } \ \ \ \ \ \ p1_{in}\text{=\ i1\...
2
votes
3answers
69 views

Do genetic algorithm and neural networks really think?

I'm aware of those AI programmes which can play games and neural networks which can identify pictures. But are they really thinking. Do they think like humans? Do they have consciousness? Or are they ...