Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
1
vote
0answers
4 views

Computational complexity of the forward pass of a CONVOLUTIONAL neural network?

How do I determine the computational complexity (big-O notation) of the forward pass of a CONVOLUTIONAL neural network? Let's assume for simplicity that we use zero-padding such that the input size ...
0
votes
1answer
46 views

Input tensor shape order for RNN (PyTorch)

I am confused as to why the sequence length is the first dimension of the input tensor for an RNN when batch size is the first dimension for any other kind of network (Linear/CNN/etc.). This makes me ...
1
vote
2answers
62 views

Why the cost/loss starts to increase for some iterations during the training phase?

I am trying to build a recurrent neural network from scratch. It's a very simple model. I am trying to train it to predict two words (dogs and gods). While training, the value of cost function starts ...
1
vote
0answers
28 views

Why would the loss increase on a single fixed input?

I'm training a neural network on some input data. I know that loss increasing may be related to: overfitting, if the loss increases on test data (while still decreases on training data) oscillations ...
0
votes
0answers
12 views

Limbs for PAFs in OpenPose

What limbs are used in openpose for the PAF? If you look at the skeletal reconstruction one would assume it is $ears - eyes$, $eyes - nose$, $nose-neck$, $neck - hips$, $shoulders - elbows$, $elbows- ...
0
votes
2answers
32 views

Is there a neural network that accepts both the current input and previous output?

I am quite new to neural networks. I am trying to implement in Python a neural network having only one hidden layer with $N$ neurons and $1$ output layer. The point is that I am analyzing time series ...
2
votes
1answer
53 views

How to use text as an input for a neural network - regression problem? How many likes/claps an article will get

I am trying to predict the number of likes an article or a post will get using a NN. I have a dataframe with ~70,000 rows and 2 columns: "text" (predictor - strings of text) and "likes&...
0
votes
1answer
30 views

How to input dataset with multi-value properties?

I'm trying to learn to use AI, and so I've followed some basic tutorials like training an MLP to predict the price of a car given properties like its age and manufacturer. Now I want to see if I can ...
0
votes
0answers
16 views

Image classification on SVG format

To best of my knowledge, images are usually fed in pixel format to ML models. Is there any work that does image classification where the image format is SVG?
-1
votes
0answers
23 views

Training a Neural Network with Hexadecimal input

I currently started working in the field of Machine Learning and have been stuck on a problem where I have to train a Neural Network with a dataset containing hexadecimal inputs. I found that we can ...
0
votes
1answer
134 views

What is the best neural network architecture for this problem?

I built a three-layer neural network (first is 1D convolutional and the remaining two are linear). It takes an input of 5 angles in radians, and outputs two numbers from 0 to 1, which are respectively ...
2
votes
1answer
145 views

Kohonen clustering of flowers

I have a question about output of my SOM network. I have trained my network with diffrent size, learning rate and epochs, but my output always can recognise two big clusters. Iris-setosa and Iris-...
2
votes
0answers
55 views

How can I find a specific word in an audio file?

I'm trying to train and use a neural network to detect a specific word in an audio file. The input of the neural network is an audio of 2-3 seconds duration, and the neural network must determine ...
2
votes
1answer
91 views

Why do the inputs and outputs of a convolutional layer usually have the same depth?

Here's the famous VGG-16 model. Do the inputs and outputs of a convolutional layer, before pooling, usually have the same depth? What's the reason for that? Is there a theory or paper trying to ...
0
votes
0answers
18 views

What are some solutions for dealing with time series data that are recorded at uneven intervals?

Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic ...
1
vote
1answer
72 views

Why is my validation/test accuracy higher than my training accuracy

Is this due to my dropout layers being disabled during evaluation? I'm classifying the CIFAR-10 dataset with a CNN using the Keras library. There are 50000 samples in the training set; I'm using a ...
3
votes
1answer
63 views

Is a basic neural network architecture better with small datasets?

I'm currently trying to predict 1 output value with 52 input values. The problem is that I only have around 100 rows of data that I can use. Will I get more accurate results when I use a small ...
0
votes
0answers
19 views

How are the step size and covariance matrix updated in CMA-ES?

I've been following the tutorial The CMA Evolution Strategy: A Tutorial to try and understand the CMA-ES, but I'm having trouble understanding how the step size and the covariance matrix are been ...
1
vote
1answer
41 views

What are some applications where tree models perform better than neural networks?

Neural networks are known to be better modeling techniques as compared to machine learning tree-based algorithms. Are there any exceptions to this?
0
votes
0answers
20 views

How to interpret the final neural network generated in NEAT-Python?

I made a simple obstacle collision game in Python and made a neural network which will play the game using the NEAT (NeuroEvolution of Augmenting Topologies) and I got a final network as follows -> ...
7
votes
3answers
1k views

Do GANs come under supervised learning or unsupervised learning?

Do GANs come under supervised learning or unsupervised learning? My guess is that they come under supervised learning, as we have labeled dataset of images, but I am not sure as there might be other ...
0
votes
0answers
19 views

Finding the energy function given update rule of a single layer non-linear neural network

Consider the network with N neurons, each of which takes a $2 \times k$ input specified by the tuple $(\vec c_t, \vec \theta_t)$ to produce output $\vec{R}_t$ through an update rule on the pairwise ...
1
vote
1answer
219 views

How to encode card game state into neural network input

I'm trying create neural network to predict moves in a card game. I am looking for recommendations on encoding the game state to my input layer. It's a complex turn based collectible card game (...
1
vote
2answers
195 views

TensorFlow 2.0 - Normalizing input to DNN (on structured data)

I have a structured dataset of around 100 gigs, and I am using DNN for classification in TF 2.0. Because of this huge dataset, I cannot load entire data in memory for training. So, I'll be reading ...
0
votes
0answers
33 views

Is it a good idea to train a neural network to classify images without base-hypothesis?

I'm a relative beginner in deep-learning (understand by that, I'm doing my first kaggle competition right now, and I have loads to learn still) and I was just wondering something. Let's say you have ...
4
votes
1answer
85 views

What is eager learning and lazy learning?

What is the difference between eager learning and lazy learning? How does eager learning or lazy learning help me build a neural network system? And how can I use it for any target function?
0
votes
1answer
60 views

Why is non-linearity desirable in a neural network?

Why is non-linearity desirable in a neural network? I couldn't find satisfactory answers to this question on the web. I typically get answers like "real-world problems require non-linear ...
3
votes
1answer
68 views

Is batch normalization not suitable for non-gaussian input?

I generate some non-Gaussian data, and use two kinds of DNN models, one with BN and the other without BN. I find that the model DNN with BN can't predict well. The codes is shown as follow: <...
1
vote
3answers
89 views

Why does a neuron in a multi-layer network need several input connections?

For example, if I have the following architecture: Each neuron in the hidden layer has a connection from each one in the input layer. 3 x 1 Input Matrix and a 4 x 3 weight matrix (for the ...
1
vote
2answers
62 views

In what situations ELUs should be used instead of RELUs?

I always use RELUs actication functions when I need to and I understand limitations of ELUs. So in what situation do I need to consider ELUs over RELUs?
1
vote
0answers
73 views

How can a de-noising auto-encoder act as an anomaly detection model?

In some research papers, I have seen that, for training the autoencoders, instead of giving the non-anomalous input images, they add some anomalies to the normal input images, and train the auto-...
1
vote
0answers
59 views

Is there a way to reduce the RMSE error when training a neural network to recognise MNIST digits using ANFIS?

I wanted to build a digit recognition neural network using MATLAB ANFIS kit. I started by using the MNIST database and I figured out it's almost impossible to classify 784 dimension data using ANFIS. ...
1
vote
1answer
46 views

Recommendations or resources for neural network/deep learning for time series application?

I know there are quite a few good deep learning books out there, but most explain neural networks and deep learning via application on images. If there are examples/code, they are often done on the ...
2
votes
1answer
63 views

why the sigmoid function will be 1 and 0 if we use a fully connected layer that produce a big enough positive(res negative )output

I am using a fully connected neural network that uses sigmoid activation function. If we feed a big enough weights the sigmoid function will finally become 1 or 0, is there any solution to avoid this? ...
2
votes
1answer
89 views

Price difference predictions curve almost vanished

With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well ...
1
vote
1answer
68 views

What is the difference between artificial neural network (ANN) and deep learning?

I have read many mixed definitions around these two terms. For example, is it right to say deep learning is any ANN with more than two hidden layers? What are formal definitions for these two?
0
votes
0answers
50 views

What are some scalable approaches to perform anomaly detection (for images with small cracks) with unsupervised learning?

I have some images with anomalies, like small cracks, but it's an imbalanced dataset. Please, suggest some effective scalable approaches. Should I consider convolutional auto-encoders? It's supposed ...
2
votes
1answer
83 views

Why L2 loss is more commonly used in Neural Networks than other loss functions?

Why L2 loss is more commonly used in Neural Networks than other loss functions? What is the reason to L2 being a default choice in Neural Networks?
5
votes
3answers
148 views

Why is symbolic AI not so popular as ANN but used by IBM's Deep Blue?

Everybody is implementing and using DNN with, for example, TensorFlow or PyTorch. I thought IBM's Deep Blue was an ANN-based AI system, but this article says that IBM's Deep Blue was symbolic AI. Are ...
1
vote
0answers
45 views

Reference for Transfer Learning via Final Layers of a Neural Network

Problem (Sketch): I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $f$ seeks to learn a new network $g$ whose last few layers behave very ...
0
votes
1answer
60 views

Is it necessary to standardise the expected output

Normalisation transform data into a range: $$X_i = \dfrac{X_i - Min}{Max-Min}$$ Practically, I found out that the model doesn't generalise well when using normalisation of input data, instead of ...
3
votes
1answer
75 views

What activation functions are currently popular?

I am not asking what activation function is better. I want to know what activation functions are more used in research or deployment. Also, are they used in combination? e.g. ReLU, ELUs, etc. I'd ...
2
votes
1answer
80 views

What kind of neural network can be trained to recognise patterns?

Is there a type of neural network that can be fed patterns to train itself on to complete new patterns that it has not seen before? What I'm trying to do is train a neural network to transform an ...
1
vote
2answers
74 views

How to create neural network that predicates result of exam?

Actually, I am "fresh-water", and I've never known what is neural network. Now I am trying understand how to design simple neuronetwork for this problem: I'd like to make up such neural network that ...
2
votes
0answers
50 views

What is the reason for mode collapse in GAN as opposed to WGAN?

In this article I am reading: $D_{KL}$ gives us inifity when two distributions are disjoint. The value of $D_{JS}$ has sudden jump, not differentiable at $\theta=0$. Only Wasserstein metric provides ...
0
votes
0answers
55 views

Backpropagation implementation with Java

I've been trying to implement a Multilayer Perceptron Network using java language with the ultimate goal of creating and teaching a neural network to recognize handwritten digits. Pretty simple and ...
1
vote
1answer
62 views

What should the output of a neural network that needs to classify in an unsupervised fashion XOR data be?

XOR data, without labels: [[0,0],[0,1],[1,0],[1,1]] I'm using this network for auto-classifying XOR data: ...
0
votes
0answers
50 views

Can I train a neural network with multiple datasets (e.g. 25)?

I want to create a neural network that I can train with many datasets (e.g. 20 - 25 datasets). Can I use transfer learning for this? Or is there a better approach than this?
2
votes
0answers
46 views

What is meant by “arranging the final features of CNN in a grid” and how to do it?

In the paper What You Get Is What You See: A Visual Markup Decompiler, the authors have proposed a method to extract the features from the CNN and then arrange those extracted features in a grid to ...
1
vote
1answer
43 views

How does the regression layer in the localization network of a spatial transformer work?

I am trying to understand the spatial transformer network mentioned in this paper https://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf. I am clear about the last two stages of the ...

1
2 3 4 5
35