Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

2,221 questions
Filter by
Sorted by
Tagged with
1 vote
9 views

How to identify important features in data?

I have a couple opportunities to write a paper, or papers over some of the neural networks I have made. I was wondering if there are anyways to figure out why the neural network classifies the data I ...
14 views

How can I demonstrate my novel machine learning classification model has value in publication?

I designed a machine learning classification algorithm that's simple enough that a comp sci 101 student could code it, relies on very few assumptions, extremely fast and efficient, and surprisingly ...
300 views

Is there a standardized method to train a reinforcement learning NN by demonstration?

I'm less familiar with reinforcement learning compared to other neural network learning approaches, so I'm unaware of anything exactly like what I want for an approach. I'm wondering if there are any ...
20 views

How to tackle the Human Error made in labeling datasets for Classifcation Tasks like Facial Expression Recognition?

I am working on the Facial Expression Recognition Task. One of the most challenging tasks that I faced was Human Error in labeling the datasets (ex: let's say FER2013). Are there anyways to Handle ...
1 vote
19 views

Can the output layer be connected to multiple layers?

Normaly, the output layer is only connected to the second last layer. Is there any model that the output layer is connected to multiple layers (For example, the second last layer AND the layer before ...
• 11
7 views

Why do we subtract logsumexp from the outputs of this neural network?

I'm trying to understand this tutorial for Jax. Here's an excerpt. It's for a neural net that is designed to classify MNIST images: ...
• 121
1 vote
28 views

Do NNs suffer from lack of efficiency in network structure and suggesting training parameters?

I am working on dynamical systems using Optimal Control theory and trying to find the connection between this field and Machine Learning. Consider a simple 2-layer Neural Network (NN) where the ...
11 views

Learn parameters for function to get normal distributed results

The problem is that I have a function which takes some parameters and returns a single value as result and I want to determine how the parameters have to be set in order to get approximately normal ...
1 vote
19 views

How Can We Create Neural Networks with Different Depths and Widths But Same Number of Parameters?

Right now I am doing a research project investigating how the depth of a Neural Network affects its capacity to learn. In order to do this, I wanted to test different Networks with the same number of ...
• 11
1 vote
22 views

Why are SVMs / Softmax classifiers considered linear while neural networks are non-linear?

My understanding is that neural networks are definitely not linear classifiers, as the point of functions like ReLU is to introduce non-linearity. However, here's where my understanding starts to ...
• 121
19 views

CNN image classification [closed]

While training CNN with a fully connected layer for image classification, isn't training everything at once the problem? For example, we want to classify dogs. Somehow in the first epoch feature ...
• 1
47 views

Why are embeddings added, not concatenated?

Let's consider the following example from BERT I cannot understand why "the input embeddings are the sum of the token embeddings, the segmentation embeddings, and the position embeddings". ...
• 161
8 views

Can neutral networks memorize labels if they are the same, and ordered, for each batch?

I am trying to reproduce research results that I found somewhere, and I just can't seem to match their high results when replicating it. Upon further investigation, I found that they do not shuffle ...
• 146
8 views

is there any variation in the results if you resize a image with black lines?

Hello I need to resize a lot of images each of these has its own random size, for example, I have photos with the following size 100x200 102x200 202x201 ... in general, the resolution of each photo of ...
• 113
22 views

Hidden layer size <= input size? [closed]

My network has 160 input features and 32 outputs and consists of 2 fully connected hidden layers. My idea was to cap the size of those hidden layers to a maximum of 160 (like the input features). Is ...
• 11
16 views

What type of NN should I use to find a function of 4 real variables? [closed]

I have an unknown function of 4 real variables, all of them between 0 and 1. The output is always between 0 and 1 as well. I got the values for over 3000 thousand different combinations of these 4 ...
• 101
24 views

How to setup and train a network with same structure but varying input and output size?

I am dealing with a problem of asset allocation. Asset price is input into the NN and the corresponding weight is output. Let's (to) be simple, assume the NN consists of three layers, the input layer ...
1 vote
26 views

Make an NN utilize other NNs as part of its decision process

Suppose I have a NN that learns to predict the time it takes a robot to move between two jobs. That's three inputs (for starters): robot, job A, job B. Not all robots travel at the same speed, and ...
• 111
8 views

What is the Best Neural Network architecture to estimate a Linear convex function?

I am currently working on a Q learning algorithm for multi-agent systems and sub-classes of Dec-POMDPs .. It has been shown before that the Q value at any time step can be reduced to a piecewise ...
13 views

Visualizing the loss landscape in deep NN to compare optimization methods

I'm comparing 2 optimization algorithms for deep neural nets through visualizing the loss landscape. The visualization method is described here. Besides the qualitative observation that how trajectory ...
• 141
14 views

How do we call a transformer having N encoders and M decoders and a learnable cross-connectivity between encoders and decoders?

How do we call a transformer having N encoders and M decoders and a learnable cross-connectivity between encoders and decoders? I am interested particularly in the case when M=1, but I imagine that it ...
• 51
12 views

Calculation of the out-of distribution (OOD) and in-distribution (ID) performance (in distrbutional shifts)

Within my work I have collected a lot of data with a question on how to evaluate my datasets properly to give a more data-centric view. What I have come across is the 'Wilds' paper which can be found ...
1 vote
33 views

What is the advantage of adding CNN to LSTM for forecasting sequential data?

I am working with simulated sequential data and the goal is to forecast that data. Long-short-term-memory (LSTM) is one of the most advanced models to forecast time series according to this post. I ...
• 111
45 views

What method to use when optimizing an array of data

Say I have an array of data, where each element describes a shape made of points, in vector form (each vector has several hundred dimensions). Each element also has a rating that gets higher, the ...
23 views

How to justify the chosen neural architecture?

I had a task to implement a neural network that would carry out multiclass classification of traffic by several parameters. On the advice of colleagues, I chose the "Multilayer Perceptron" ...
25 views

How to configure a neural network to selectively change only certain characters in a string?

I'm trying to figure out how to train a neural network to macronize Latin text. Essentially, in Latin, vowels can either be long or short, and length is indicated with a macronized character: i.e. o ...
50 views

What exactly is the AI explainability problem?

I am pretty new to AI and have recently been paying attention to AI explainability and the fact that it remains a hurdle within the path of commercializing certain AI systems in health for instance. I ...
1 vote
32 views

How to identify and diferentiate several edge lines of an object?

I want to create an AI to detect and identify certain edge lines on my image. The input image is a locker key, and I want to know the exact position of certain edges. Sample input image: Sample ...
• 111
66 views

How to use a trained neural network to find optimal function inputs? [closed]

I have a deep neural network with 4 input nodes, 4 hidden layers with 4 nodes each and 1 output layer with one node (TRUE, FALSE). I have already trained the NN using backpropagation because I have ...
137 views

Why are only neural networks (and not SVMs, for example) used for reinforcement learning?

I know that neural networks are the "universal function approximator", but they also have a huge number of trainable parameters and are extremely prone to overfitting. So my question is: Why ...
52 views

Why aren't neural networks contractions?

I'm not sure I understand why neural networks aren't considered contractions, as Geoffrey J. Gordon says in his paper: Stable Function Approximation in Dynamic Programming: "Our theorems in the ...
37 views

Do I need to create one or many neural networks to play Risk? [closed]

I have a school project to develop an AI model that plays the Risk board game as optimally as possible. Now, I have made the environment of Risk in Python and I narrowed down my possible machine ...
49 views

YearPrediction dataset for a regression task: is it possible to evaluate a fair comparison between standard loss and a quadratic one?

We are trying to evaluate a loss function on the Year Prediction (Million Songs) data set. The problem is that we don't know how to configure an experiment in order to test if one loss (the standard ...
30 views

Is it possible to use attention in non sequential data in neural networks?

I'm still trying to understand the attention mechanism. It is not clear to me what query, key, and value mean yet, for example. However, my main issue is regarding how to apply attention in my use ...
• 101
14 views

Why even-sized kernels are used in upscaling layers?

I have noticed that UNet and many GANs uses even-sized kernels in the upscale part of the model. I have read that at least in the GAN situation one of the reasons why we use even-sized kernels is that ...
36 views

Is early stopping a collection of techniques or a single technique?

Early stopping is a regularization technique in neural network training. It avoids overfitting. I have doubts about early stopping that whether it refers to a single technique (sense #1) or a family ...
• 3,099
39 views

How can a convnet learn with a 3x3 output layer?

I was studying the "Deep Learning with Python" book, I came across this MNIST example and this is how the last conv2d layer looks like: ...
20 views

Mathematically speaking, what does the target networks in DDPG compute? [duplicate]

In general, to implement DDPG, we use four networks instead of two. They are actor, critic, target actor, and target critic. I am writing the mathematical formulation of the first two networks I ...
• 3,099
10 views

How to model a set-to-set mapping with graph neural networks?

I have a task on a heterogeneous graph where a set of nodes is given as input and some of the nodes are acceptable outputs. The dataset essentially consists of pairs (X, Y) where X is a set of nodes ...
• 101
38 views

How should you reshape data before feeding it to LSTM layers?

I was curious if anyone had any advice on how to reshape data for a recurrent neural network. What I've been doing is array.reshape(len(X_train), # of points in time, # of features) And then in the ...
26 views

Predicting using time-series data and static data?

I have recently been working on predicting the final value of articles on Steemit.com using downloaded data. I have a large variety of features which divide into two types. Features which change over ...
14 views

What happens in tensorflow/pytorch under the hood when doing a convolution2d() with x filters when num input feature maps/channels > x? [duplicate]

I was wondering about what happens in tensorflow/pytorch under the hood when doing a convolution2d() with x filters when num input feature maps/channels > x? e.g. my input shape with 129 feature ...
17 views

If two functions are close apart can I proof the difference of their empirical loss is also small?

I am trying to understand the proof of Theorem 3 in the paper A Universal Law of Robustness via isoperimetry by Bubeck and Sellke. Basically, there exist at least one $w_{L,e}$ in $\mathcal{W}_{L,e}$ ...
• 101
29 views

Why gradients are used in Layer-wise Relevance Propagation (LRP)?

To give you an overview, Layer-wise Relevance Propagation is a technique by which we can get relevance values at each node of the neural network. These calculated relevance values (per node) are ...
• 111
42 views

How to decode P bits that represent a random weight generator?

So I've been tasked by my neural network professor at university to replicate the following research: Intelligent Breast Cancer Diagnosis Using Hybrid GA-ANN. Each chromosome represents a possible net,...
1 vote
49 views

Can you make a Neural Network drunk or high?

We know that the human brain can become sozzled by various substances that are released into the brain, but can you make an artificial neural network drunk or high? For example on a RL Agent that ...
11 views

Is it possible to add identities into existed face recognition system without re-train the NN?

I have followed some tutorials and I find out all of them could not add new faces. If a new face is added, the system would have to be retrained. This doesn't sound logical because it would waste too ...
197 views

How does learning the moves of chess show up in a neural network?

Is learning the moves a special case or just the same sort of thing that happens as the AI learns strategy? If you take two different neural networks and teach them each how the pieces move, what ...
• 141