Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
1
vote
0answers
7 views

Does this diagram represent several LSTMs, or one through several timesteps?

I'm trying to read this paper describing Google's LSTM architecture for machine translation. It features this diagram on page 4: I'm interested in the encoder block, on the left. Apparently, the pink ...
2
votes
0answers
29 views

Logistic Map as activation function

I find the logistic map absolutely fascinating. Both in itself (because I love fractal) and because it is observed in nature (see: https://www.youtube.com/watch?v=ovJcsL7vyrk). I'm wondering if anyone ...
0
votes
0answers
6 views

React on train-validation curve after trening

I have a regression task that I tray to solve with AI. I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance) I understand basic ...
0
votes
0answers
4 views

How to find rotation x/y/z of chessboard diagram with what network architecture?

I want to recognize pieces of chessboard diagram (not real 3d pieces but just diagram). I split this task in some operation like rotation/cutting/segmentation. First of all I want to detect chessboard ...
0
votes
0answers
17 views

What is the purpose of soft orthogonal regularization in training deep neural network?

I'm reading papers regarding soft orthogonal regularization, $\frac \lambda 4||WW^\intercal - I||_F^2$, over a deep neural network whose activation function is ReLU and weight matrix $W$ is ...
1
vote
1answer
28 views

Why are shallow networks so prevalent in RL?

In deep learning, using more layers in a neural network adds the capacity to capture more features. In most RL papers, their experiments use a 2 layer neural network. Learning to Reset, Constrained ...
0
votes
0answers
39 views

How does NN follows law of energy conservation?

Communication requires energy, and using energy requires communication. According to Shannon, the entropy value of a piece of information provides an absolute limit on the shortest possible average ...
0
votes
0answers
18 views

How does Google's 2016 GNMT architecture work?

I'm trying to read this paper describing Google's LSTM architecture for machine translation from 2016. However, I'm getting stuck as certain things are described too vaguely for me. This is a picture ...
0
votes
0answers
23 views

Which models can I use for supervised learning with images?

I have to do a project that detects fabric surface errors and I will use machine learning methods to deal with it. I have a dataset that includes around six thousand fabric surface images with the ...
3
votes
0answers
53 views
+50

In GradCAM, why is activation strength considered an indicator of relevant regions?

In the GradCAM paper section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification ...
1
vote
2answers
56 views

How to use sigmoid as transfer function when input is not (0,1) range in ANN?

I am building my first ANN from scratch. I know that I need a transfer function and I want to use the sigmoid function as my teacher recommended that. That function can be between 0 and 1, but my ...
1
vote
0answers
30 views

How to use unmodified input in neural network?

My question may be a bit hard to explain... My neural network learns a categorical distribution, which serves as an index. This index will look up the value (= action_mean) in Input 2. From this ...
1
vote
1answer
45 views

What kind of problems is DQN algorithm good and bad for?

I know this is a general question, but I'm just looking for intuition. What are the characteristics of problems (in terms of state-space, action-space, environment, or anything else you can think of) ...
1
vote
1answer
48 views

Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter?

Background From my understanding (and following along with this blog post), (deep) neural networks apply transformations to the data such that the data's representation to the next layer (or ...
1
vote
1answer
31 views

Is there some known pattern for selecting a batch of candidates for the next generation?

I'm a beginner with a classic "racing car" sandbox and a homemade simple neural network. My pattern: Copy the "top car" (without mutation) to the next generation If there are ...
0
votes
1answer
17 views

Which machine learning technique can I use to match one set of data points to another?

I have two measuring devices. Both measure the same thing. One is accurate, the other is not, but does correlate with a non-fixed offset, some outliers, and some noise. I won't always be using the ...
-1
votes
1answer
51 views

How to extract the main text from a formated text file?

My idea is to model and train a neural network that receives a text version of a PDF file as the input and gives the content text as output. Take the scenario: One prints a PDF file to a text file (...
0
votes
0answers
6 views

Best strategy for Classification of Science Subjects. Phy, Chem , Maths and Bio? BERT, Transformers, Attention+SLTM, Self-Attention+LSTM?

I am working on a project where I have to first classify the Subjects of the given question and then the respective Chapter and then the sub-topic. In a nutshell, I have to predict the Subject, Grade ...
2
votes
0answers
16 views

Understanding neural network achitectures in policy gradient reinforcement learning for continuous state and action space

I am trying to train a neural network using reinforcement learning / policy gradient methods. The states, i.e. the inputs, as well as the actions I am trying to sample are vectors with each element ...
2
votes
0answers
43 views

Why are “Transformers” called this way?

What is the reason behind the name "Transformers", for Multi Head Self-Attention-based neural networks from Attention is All You Need? I have been googling this question for a long time, and ...
2
votes
0answers
22 views

What is the appropriate way of passing a list of integers that represents the environment to a neural network's dense layer?

I'm training an RL agent using the DQN algorithm to do a specific task. The environment is represented by a list of $10$ integer numbers from $0$ to $20$. An example would be $[5, 15, 8, 8, 0, \dots]$....
1
vote
0answers
35 views

What are the use-cases of self-replicating neural networks?

I started researching the subject of self-replication in neural networks, and unexpectedly I saw that there is not much research on this subject. I should mention I am new in the field of NNs. This ...
0
votes
1answer
30 views

Difference between Neural Compute Stick 2 and Google Coral USB for edge computing [closed]

I am trying understand machine learning inferece, and i would like to know what exactly is the difference between Google Coral USB and Movidius Intel Neural Compute Stick 2. From what i could gather ...
1
vote
1answer
24 views

Is average pooling equivalent to a strided convolution with a specific constant kernel?

It seems to me that average pooling can be replaced by a strided convolution with a constant kernel. For instance, a 3x3 pooling would be equivalent to a strided convolution (of stride $3$) with a $3 \...
0
votes
1answer
20 views

Are there any downsides of using a fixed seed for a neural network's weight initialization?

For example, if we set the random seed to be 0, will we run into any problems? For example, maybe for seed 0, we can only reach a certain training error, but other seeds will converge to a much lower ...
1
vote
0answers
15 views

Given a dataset and a neural network, is there some heuristic or theorem to determine whether this neural network has enough capacity? [duplicate]

What is the consensus regarding NN "capacity" or expressive power? I remember reading somewhere that expressive power grows exponentially with depth, but I cannot seem to find that exact ...
0
votes
0answers
18 views

Am I using transfer learning when I use SSD ResNet 50 model architecture?

Using Label-img, I have successfully labeled my images (dimensions 1100 x 1100 pixels), and am currently training the SSD ResNet50 model (from the TensorFlow 2 Detection Model Zoo). I downloaded the ...
0
votes
0answers
11 views

Does a Siamese Network need other trainable layers after the distance layer?

I'm approaching at Siamese Networks in order to use them for Image Similarity. I found that many people use famous models like VGG or ResNet to build the vectors that will go on the distance layer in ...
0
votes
0answers
31 views

Why aren't Self Replicating Neural Networks more popular? [closed]

I started researching the subject of Self Replicating Neural Networks recently and went all the from the depths of 1950s of the Von Neumann universal constructor to this paper. Now I am very curious ...
0
votes
0answers
8 views

Cyclic network's output converges to a constant

I have a problem. i have designed a network that takes in as input a segment of the output vector. The problem is, depending on the eigenvectors of the weight matrix, the series converges to a ...
0
votes
0answers
16 views

Need suggestion for Reinforcement Learning based visual landing system for quadcopters (UAVs)

I have deep interest in quadcopters. I need ideas for designing of an experiment. I have a programmable quadcopter. I can autonomously land it on a staitonary landing pad with a vision algorithm. ...
2
votes
0answers
52 views

Why should the weight updates be proportional to input?

I'm reading the book Grokking Deep Learning. Regarding weight updates during training, it has the following code and explanation: ...
0
votes
0answers
10 views

Effect of Batch Size on a Translation Model's Validation Scores

This is a general question. I am training a neural machine translation model with different batch sizes: 16, 32, and 64 for the same amount of epochs (10). I am getting very low validation scores for ...
1
vote
0answers
38 views

Where can I find pre-trained agents able to play games with multiple stages like exploration, dialog, combat?

My goal is to create an ML model to be able to classify different game stages, e.g., dialog with a non-player character, exploration, combat with enemy, in-game menu etc. In order to do that, I am ...
1
vote
0answers
29 views

In the MINE paper, why is $\hat{G}_B$ biased, and how does the exponential moving average reduce the bias?

While reading the Mutual Information Neural Estimation (MINE) paper [1] I came across section 3.2 Correcting the bias from the stochastic gradients. The proposed method requires the computation of the ...
1
vote
1answer
48 views

Can most of the basic machine learning models be easily represented as simple neural network architectures?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. In chapter 1.2.1 Single Computational Layer: The Perceptron, the author says the following: Different ...
0
votes
1answer
29 views

How is the performance of a model affected by adding a ReLU to fully connected layers?

How significant is adding a ReLU to fully connected (FC) layers? Is it necessary, or how is the performance of a model affected by adding ReLU to FC layers?
0
votes
0answers
6 views

Convert tensorflow model into a matrix [migrated]

Suppose we have trained a neural network with Tensorflow. I know that deep learning is like finding the best matrix of weights. So is there anyway to convert the trained model into the matrix (lets ...
0
votes
0answers
10 views

How does scheduled sampling for transformers work?

I was reading this paper which applies a modified version of the transformers for traffic forecasting. I am somewhat familiar with the transformer architecture and how it functions, but, in the paper, ...
0
votes
1answer
18 views

Will changing the dimension reduction size of a neural network (i.e. SSD ResNet-50) change the overall outcome and accuracy of the model?

I am training a convolutional neural network to detect objects (weeds amongst crops, in my case) using TensorFlow. The original dimensions of the raw training photos are 4000 x 3000 pixels, which must ...
1
vote
1answer
46 views

Why does the accuracy drop while the loss decrease, as the number of epochs increases?

I've been trying to find the optimal number of epochs that I should train my neural network (that I just implemented) for. The visualizations below show the neural network being run with a variable ...
-1
votes
1answer
22 views

Loss function definition

I have read what the loss function is but I am not sure if I have understood it. For each neuron in the output layer the loss function is equal most usually to the square of the difference value of ...
0
votes
0answers
8 views

Is there a framework or method that would help visualise inner workings of a feedforward neural network?

I wonder if there is some framework or method to help visualising inner workings of a feedforward deep neural network? What I mean by this is something similar to what is being done with CNNs where we ...
0
votes
1answer
37 views

If neurons performed the operation of an entire layer, would that make the neural network more effective?

(I have a very primitive understanding of neural networks, so please forgive the lack of technicality here.) I am used to seeing a neuron in a neural network as something that- Takes the inputs and ...
1
vote
0answers
11 views

Find object's location in an area using computer vision

I'm trying to see how to detect the location of a soccer ball in the field using the live camera. What are some ways to achieve this? 1- Assuming we have a fixed camera with a wide shot. How to find ...
1
vote
2answers
30 views

Get object's orientation or angle after object detection

I'm trying to get a detected car's orientation when object detection is applied. For instance, when we apply object detection on a car and get a bounding box, is there any ways or methods to calculate ...
0
votes
0answers
33 views

Is the “mlpconv” layer in the “Network in Network” paper computing $1 \times 1$ convolutions or not?

I am reading the Network in Network paper. This is the equation they introduce for their mlpconv layer (equation 2 in the paper, page 3): $$ \begin{aligned} f_{i, j,...
1
vote
1answer
53 views

Evolved networks fail to solve XOR

My implementation of NEAT consistently fails to solve XOR completely. The species converge on different sub-optimal networks which map all input examples but one correctly (most commonly (1,1,0)). Do ...
-1
votes
0answers
34 views

My Neural net accuracy is low. Could my fix be Bias?

If I normalize my inputs and output between 0 and 1, then do I really need to worry about a neural net bias? My 4 inputs and 4 outputs are uniformly distributed between 0 and 1. I am using: loss=tf....
0
votes
0answers
35 views

Unable to meet desired mean squared error

I wish to get MSE < 0.5 on test data (https://easyupload.io/zr7xf3) which is 20% of given data chosen randomly. But I am reaching 0.73 using both plain Ridge Regression as well as a neural network ...

1
2 3 4 5
38