Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
1
vote
2answers
185 views

Is a VGG-based CNN model sometimes better for image classfication than a modern architecture?

I have an image classification task to solve, but based on quite simple/good terms: There are only two classes (either good or not good) The images always show the same kind of piece (either with or ...
7
votes
5answers
1k views

Can prior knowledge be encoded in deep neural networks?

I was reading Gary Marcus's a Critical Appraisal of Deep Learning. One of his criticisms is that neural networks don't incorporate prior knowledge in tackling a problem. My question is: have there ...
25
votes
4answers
4k views

Can neural networks be used to prove conjectures?

Imagine I have a list (in a computer-readable form) of all problems (or statements) and proofs that math relies on. Could I train a neural network in such a way that, for example, I enter a problem ...
0
votes
0answers
3 views

More inputted variables mean more powerful ANN model?

what is the impact of the inputted variables number on the ANN model (In general)? does increasing variables numbers mean a more powerful prediction model (for approximation purpose)? these two ...
0
votes
0answers
9 views

Is there a ReLU implementation where the slope can be specified for positive values

I am testing the effects of different activations on my model and was wondering if there already exists a variant of ReLU where one can set the slope for positive values. PReLU learns a parameter for ...
4
votes
2answers
89 views

Why neural networks tend to be trained to recognize multiple things instead of just one?

I was watching this series: https://www.youtube.com/watch?v=aircAruvnKk The series demonstrates neural networks by building a simple number recognizing network. It got me thinking: Why neural networks ...
2
votes
1answer
155 views

Is it normal that SOM clusters the instances with the “versicolor” class into multiple different BMUs?

I have trained (with different sizes, learning rates, and epochs) a SOM network to cluster the Iris dataset. The instances associated with the class setosa have ...
0
votes
0answers
9 views

Training a model to identify certain differences between images?

Newbie to CV here so sorry of this is basic. Here's the deal, I have a program that I run many times. and each run I produce a screenshot. I need to compare screenshots from N-1 and N runs and make ...
0
votes
1answer
147 views

What is the best neural network architecture for this problem?

I built a three-layer neural network (first is 1D convolutional and the remaining two are linear). It takes an input of 5 angles in radians, and outputs two numbers from 0 to 1, which are respectively ...
4
votes
1answer
443 views

Is there a way to perform pattern recognition without a labeled training set?

I have a 10GB file of a time series 1D signal. I want to find some patterns within this signal, I know CNN's are great for this but the problem is I don't have any training data. Now, I could, of ...
2
votes
1answer
109 views

Why do the inputs and outputs of a convolutional layer usually have the same depth?

Here's the famous VGG-16 model. Do the inputs and outputs of a convolutional layer, before pooling, usually have the same depth? What's the reason for that? Is there a theory or paper trying to ...
0
votes
0answers
25 views

Why using Aux() to implement VAE_1?

I have a bit of difficulties working on the article :https://arxiv.org/pdf/2009.07047v1.pdf. I want to implement the VAE_1, but it used VAE-GAN. I am familiar with ...
2
votes
1answer
33 views

What do echo state networks give us over a generic RNN resevoir?

Slightly generalizing the definition in Jaeger 2001, let's define a reservoir system to be any system of the form $$h_{t}=f(h_{t-1}, x_t)$$ $$y_t=g(Wh_t)$$ where $f$ and $g$ are fixed and $W$ is a ...
1
vote
0answers
15 views

Why should variance(output) equal variance(input) in Xavier Initialisation?

In a lot of explanations online for Xavier Initialization, I see the following: With each passing layer, we want the variance to remain the same. This helps us keep the signal from exploding to a ...
12
votes
7answers
12k views

How to classify data which is spiral in shape?

I have been messing around in tensorflow playground. One of the input data sets is a spiral. No matter what input parameters I choose, no matter how wide and deep the neural network I make, I cannot ...
1
vote
0answers
13 views

Looking for a good approach for building an automated director for a racing game spectator mode

I'm building a tool that should assist a director to broadcast a racing game. I want this tool to suggest the human director which car to focus on and with which camera (among the available ones). I ...
1
vote
1answer
106 views

Why is my validation/test accuracy higher than my training accuracy

Is this due to my dropout layers being disabled during evaluation? I'm classifying the CIFAR-10 dataset with a CNN using the Keras library. There are 50000 samples in the training set; I'm using a ...
0
votes
0answers
12 views

Why does 0.8:0.2 divided dataset have a much greater AUROC than its 5-fold cross validated counterpart's mean AUROC?

I trained a dataset with a 5-fold cross validation to search for hyper-parameters by an AUROC metric. For splitting I used ...
3
votes
1answer
73 views

Is a basic neural network architecture better with small datasets?

I'm currently trying to predict 1 output value with 52 input values. The problem is that I only have around 100 rows of data that I can use. Will I get more accurate results when I use a small ...
1
vote
1answer
48 views

Computer vision - Can you put more weight on a specific part of the object?

Let's say I'm looking for any item that has a certain shape (outline) in a photo. but I can further classify it only according to particular features, that most of them are expected to be shown only ...
2
votes
0answers
12 views

Image classification - Need method to classify “unknown” objects as “trash” (3D objects)

We have an image classifier that was built using CNN with faster R-CNN and Yolov5. It is designated to run on 3D objects. All of those objects have similar "features" structure, but the ...
9
votes
2answers
887 views

Is neural networks training done one-by-one? [duplicate]

I'm trying to learn neural networks by watching this series of videos and implementing a simple neural network in Python. Here's one of the things I'm wondering about: I'm training the neural network ...
2
votes
1answer
705 views

Is back-propagation applied for each data point or for a batch of data points?

I am new to deep learning and trying to understand the concept of back-propagation. I have a doubt about when the back-propagation is applied. Assume that I have a training data set of 1000 images ...
0
votes
0answers
26 views

Deep Q-learning network not working

I have this code: ...
3
votes
1answer
98 views

In GradCAM, why is activation strength considered an indicator of relevant regions?

In the GradCAM paper section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification ...
1
vote
1answer
248 views

How to encode card game state into neural network input

I'm trying create neural network to predict moves in a card game. I am looking for recommendations on encoding the game state to my input layer. It's a complex turn based collectible card game (...
0
votes
0answers
11 views

Add Additional Positional Information to Image Classification Neural Network

I am trying to find the best way to provide a neural network with both an image and some annotations about the image. Specifically, I'm creating a network to calculate an approximate 'cost' to go from ...
0
votes
1answer
65 views

Why is non-linearity desirable in a neural network?

Why is non-linearity desirable in a neural network? I couldn't find satisfactory answers to this question on the web. I typically get answers like "real-world problems require non-linear ...
3
votes
1answer
85 views

Is batch normalization not suitable for non-gaussian input?

I generate some non-Gaussian data, and use two kinds of DNN models, one with BN and the other without BN. I find that the model DNN with BN can't predict well. The codes is shown as follow: <...
2
votes
1answer
94 views

Price difference predictions curve almost vanished

With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well ...
1
vote
0answers
20 views

Neural network architecture with inputs and outputs being an unkown function each

I am trying to set up a neural network architecture that is able to learn the points of one function (blue curves) from the points of an other one (red curves). I think that it could be somehow ...
0
votes
0answers
19 views

What would be the best criteria for determining the best generation on a maze training by mutation?

I'm training a neural network to solve a maze. My process is the following: Randomly generate a small maze Spawn hundreds of cars at the start, that will go through the same one maze Assign the same ...
-1
votes
1answer
74 views

How to extract the main text from a formated text file?

My idea is to model and train a neural network that receives a text version of a PDF file as the input and gives the content text as output. Take the scenario: One prints a PDF file to a text file (...
1
vote
2answers
75 views

Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter?

Background From my understanding (and following along with this blog post), (deep) neural networks apply transformations to the data such that the data's representation to the next layer (or ...
2
votes
1answer
22 views

Does this diagram represent several LSTMs, or one through several timesteps?

I'm trying to read this paper describing Google's LSTM architecture for machine translation. It features this diagram on page 4: I'm interested in the encoder block, on the left. Apparently, the pink ...
4
votes
0answers
62 views

Has the logistic map ever been used as an activation function?

I find the logistic map absolutely fascinating. Both in itself (because I love fractal) and because it is observed in nature (see: https://www.youtube.com/watch?v=ovJcsL7vyrk). I'm wondering if anyone ...
-1
votes
0answers
27 views

computational complexity of attention in dynamic-convolution

I'm reading this paper , Dynamic convolution-Attention over Convolution kernels. I couldn't understand the complexity of attention i.e., How to calculate O(π(...
1
vote
0answers
22 views

How does Google's 2016 GNMT architecture work?

I'm trying to read this paper describing Google's LSTM architecture for machine translation from 2016. However, I'm getting stuck as certain things are described too vaguely for me. This is a picture ...
1
vote
1answer
34 views

Why are shallow networks so prevalent in RL?

In deep learning, using more layers in a neural network adds the capacity to capture more features. In most RL papers, their experiments use a 2 layer neural network. Learning to Reset, Constrained ...
1
vote
0answers
13 views

React on train-validation curve after trening

I have a regression task that I tray to solve with AI. I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance) I understand basic ...
1
vote
1answer
51 views

What kind of problems is DQN algorithm good and bad for?

I know this is a general question, but I'm just looking for intuition. What are the characteristics of problems (in terms of state-space, action-space, environment, or anything else you can think of) ...
1
vote
0answers
45 views

How does NN follows law of energy conservation?

Communication requires energy, and using energy requires communication. According to Shannon, the entropy value of a piece of information provides an absolute limit on the shortest possible average ...
3
votes
1answer
112 views

Should I prefer the model with the lowest validation loss or the highest validation accuracy to deploy?

I trained a ResNet20 on Cifar10 and obtained the following learning curves. From the figures, I see at epoch 52, my validation loss is 0.323 (the lowest), and my validation accuracy is 89.7%. On the ...
4
votes
1answer
266 views

Is there a continuous conditional variational auto-encoder?

The Conditional Variational Autoencoder (CVAE), introduced in the paper Learning Structured Output Representation using Deep Conditional Generative Models (2015), is an extension of Variational ...
0
votes
0answers
4 views

How to find rotation x/y/z of chessboard diagram with what network architecture?

I want to recognize pieces of chessboard diagram (not real 3d pieces but just diagram). I split this task in some operation like rotation/cutting/segmentation. First of all I want to detect chessboard ...
19
votes
3answers
25k views

How can we process the data from both the true distribution and the generator?

I'm struggling to understand the GAN loss function as provided in Understanding Generative Adversarial Networks (a blog post written by Daniel Seita). In the standard cross-entropy loss, we have an ...
13
votes
2answers
2k views

How are generative adversarial networks trained?

I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative ($...
0
votes
0answers
25 views

What is the purpose of soft orthogonal regularization in training deep neural network?

I'm reading papers regarding soft orthogonal regularization, $\frac \lambda 4||WW^\intercal - I||_F^2$, over a deep neural network whose activation function is ReLU and weight matrix $W$ is ...
1
vote
1answer
52 views

How does the regression layer in the localization network of a spatial transformer work?

I am trying to understand the spatial transformer network mentioned in this paper https://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf. I am clear about the last two stages of the ...
2
votes
2answers
66 views

How do neural networks weigh multiple inputs/features of different dimensionality?

I am confused about how neural networks weigh different features or inputs. Consider this example. I have 3 features/inputs: an image, a dollar amount, and a rating. However, since one feature is an ...

1
2 3 4 5
38