Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
1
vote
0answers
25 views

Convolutional Neural Network (CNN) with Tree architecture to organize the number of classes

At the moment, I have around 1.000 classes with accuracy and loss that are acceptable. In the long term, there could be more than 100.000 classes. The main problem is that every time a new class is ...
1
vote
0answers
20 views

What is a Silhouette Neural Network

I was going through a study in which I found something called a dilated Silhouette Neural Network. I want to know what it is, what it can do, and how it is better from a CNN? Link to the journal: Link
0
votes
0answers
16 views

How to manually optimize Neural Networks the most systematical way?

Do you have any ideas or guidance on how to do manual neural network optimization in the most systematic way? Especially when models train longer and the effects of hyperparameter fitting are very ...
0
votes
0answers
33 views

Is input standardisation better than input normalisation?

Consider a network which takes samples of single values. And consider the training set of 5 samples: $$ inp = [5, 6, 7, 8, 9] $$ Input normalisation: $$ min = 5, max = 9, span = 9-5 = 4 \\ Input1 = [(...
0
votes
2answers
35 views

Is "Pruning" only applicable to CNNs?

What Is Neural Network Pruning And Why Is It Important Today? The above article only talks about Convolutional Neural Networks: One of the first methods of pruning is pruning entire convolutional ...
0
votes
0answers
23 views

Regress values inside the bounding boxes to predict a value in Object Detection

I am currently working on an object detection task. I have a dataset of Grayscale and Depth Images. The annotation format is x1, y1, x2, y2, class, depth. I have calculated this depth (of each object/...
0
votes
0answers
28 views

What type of neural network do I need?

I am working on protein structure prediction. Suppose, I am solving a problem using Neural Networks. I know how many inputs and outputs there will be in the model, as it directly depends on the ...
0
votes
0answers
12 views

Is it valid to implement hyper-paramter tuning and THEN cross validation

I have a multi-label classification task I am implementing. I have done a hyper-parameter tuning to determine the best configuration for my neural network. Is it valid to do this (determine the best ...
0
votes
0answers
19 views

What type of activation function do I need? [duplicate]

Suppose, I am solving a problem using Neural Networks. I know how many inputs and outputs there will be in the model, as it directly depends on the problem statement. However, how do I know: What ...
0
votes
0answers
21 views

The number of hidden layers and the number of neurons in each hidden layer [duplicate]

Suppose, I am solving a problem using Neural Networks. I know how many inputs and outputs there will be in the model, as it directly depends on the problem statement. However, how do I know: How many ...
0
votes
0answers
4 views

Evaluating a CNN -multi class model with two separate thresholds

I have a model that outputs three classes. But here instead of one threshold, it depends on a combination of two (user input threshold). One threshold varies from 0.1 to 1.0 and the other varies from ...
1
vote
1answer
21 views

What are the labels in figure1 in the Paper "The perceptron: A probabilistic model for information storage and organization in the brain"?

This figure comes from The perceptron: A probabilistic model for information storage and organization in the brain I guess the first circle (neuron) labels RETINA, the second labels perceptron area, ...
1
vote
0answers
29 views

Is the formula $\frac {1}{s}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|$ the correct form of 0-1 loss function, in the context of Perceptron?

Per page 7 of this MIT lecture notes, the original single-layer Perceptron uses 0-1 loss function. Wikipedia uses $${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|} \tag{1}$$ to denote ...
2
votes
1answer
46 views

When does a neural network have a single and when does it have multiple outputs?

What I understand is, each input in a neural network is a feature. However, what I don't understand is, when we need multiple outputs in a neural network. For example, say, if we are classifying cats ...
1
vote
0answers
15 views

More channels vs multiple inputs in neural network

Suppose I want to train the model for playing chess. I found that existing models use as input the grid with dimensions 8x8x20 (so we have 20 channels). Some channels may represent how different kind ...
1
vote
0answers
21 views

Recommended literature on layers for reinforcement learning

I was recommended to ask here after I posted on stack overflow wrongly. I was wondering if anyone had any recommended readings on layers used in neural networks for reinforcement learning? I've been ...
1
vote
1answer
69 views

How does the neural network learn when used in the REINFORCE algorithm?

As per my understanding, you run an entire episode, which contains many steps, and then back-propagate using just 1 loss value. How does the neural network learn to differentiate between good and bad ...
0
votes
0answers
34 views

AI model to predict/generate person's image

I want to make a model that predicts person's shape depending on his son's image. My plan is to create a dataset and each data point in it consists of two images; One for the father or mother and one ...
2
votes
1answer
66 views

What gets optimized in convolutional neural network?

In a convolutional neural network, the hyperparameters such as number of kernels and stride, kernel size, etc are determined. After some combination of convolutions, ReLU and pooling layer there is ...
1
vote
0answers
14 views

Are there guiding principles as to which activation functions suit a given RL algorithm?

Are there rules of thumb as to which activation functions work well (or which one would not) on the policy and value network of a class of RL algorithms? For hidden layers and for the output layer. ...
1
vote
0answers
15 views

How to choose proper normalization strategy for the activations?

I am reading a survey on various normalization techniques adopted in neural network architectures. The purpose of introducing normalization is understandable - to stabilize the training and avoid ...
0
votes
0answers
18 views

Is the compression function for SHA hash algorithms a hidden layer in a neural net?

Is the compression function for SHA-256 and SHA-512 a "hidden layer" in a neural net? If so, what type of neural net is it in? SHA-256 and SHA-512 compression function: source: NIST, “...
0
votes
0answers
23 views

Could an adaptive discriminator augmentation (ADA) be used for a discriminatory task?

Was wondering if I could use an adaptive discriminator augmentation (ADA) on a data set like MNIST (multi-class classification task). It seems that this is focused on generative modeling, so not sure ...
1
vote
0answers
39 views

In NEAT, how do node numbers work?

I have read a lot of debates about node ids and such. I'm not 100% sure how it works, but I am assuming the next node added to a network would be the next number in that specific networks list? For ...
2
votes
0answers
43 views

How does the paper implement NEAT without a global set tracking Innovations?

I have been reading this paper on NEAT and trying to implement the algorithm in C#. For the most part, I understand everything in the paper however, there are 2 things I don't understand that confuse ...
0
votes
3answers
61 views

Is there a recent book that covers the theoretical and philosophical aspects of artificial intelligence?

What are some recent books that introduce AI and neural networks while also discussing the related philosophical issues, like epistemology and whether AI is really thinking, etc.?
0
votes
0answers
19 views

How to obtain STD from Neural Network with 2 continuous action output

In my Environment, I have two continuous action space self.action_space = spaces.Box(low=np.array([0.,0.]), high=np.array([4.,0.02]), shape=(2,), dtype=np.float32) ...
0
votes
0answers
31 views

GAN performs worse after 50 epochs than after 2

I am training GAN on SVHN dataset (house numbers in Google Street View images, dimensions: 3x32x32 - 3 color channels). The problem is that it performs worse after some training (e.g. after 50 epochs) ...
0
votes
1answer
10 views

Can we modelize an RNN by an ANN that takes precedent output as a part of input?

Is it possible to consider an RNN as a classical feedforward neural network that just take the precedent output as a part of the input ?
0
votes
0answers
28 views

Is NEAT speciation really effective?

I tried implementing NEAT algorithm from scratch, and it successfully solves XOR problem. I followed the original NEAT paper. However, when I run XOR problem solving test and calculate average ...
3
votes
1answer
102 views

Are these visualisations the filters of the convolution layer or the convolved images with the filters?

There are several images related to convolutional networks on the Internet, an example of which I have given below My question is: are these images the weights/filters of the convolution layer (the ...
2
votes
0answers
21 views

Can Neural Networks using ReLU activation work without using the bias term in their neurons?

I created a super simple NN of 1 input, 2 hidden layers of 2 neurons each and 1 output neuron as shown below. All activations are ReLUs and neurons doesn't use the bias term. What I found is that the ...
1
vote
0answers
45 views

Backpropagation not working as expected

I'm new to neural networks and I try to make a model that is guessing if a point is below or above relative to a function output. The idea is inspired from this video https://youtu.be/DGxIcDjPzac . ...
1
vote
1answer
51 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
1
vote
1answer
43 views

What is the state of the art in melody generation?

Generative Adversarial Networks can generate realistic photos of people, such as thispersondoesnotexist.com. I wonder whether one can train an artificial intelligence on a batch of plain solo melodies ...
1
vote
0answers
39 views

Identifying rotating and resizing letters with background noise

I'm trying to complete a captcha, and here is what it looks like: Between captchas the calligraphy of the letters is the same, but the letters may be resized and rotated. And the background noise (...
0
votes
0answers
15 views

Metric Learning with l2 distance and contrastive loss is not working

I am trying metric learning with L2 distance and contrastive loss with a pre-trained language transformer as an embedding extractor. I ran my model for 20 epochs, and the loss is decreasing. But when ...
0
votes
0answers
10 views

Increased performance using monotonic constraints with neural networks

I see that with the xgboost library, we can tell the training process that some features are necessarily monotonic with the model's output - https://xgboost.readthedocs.io/en/latest/tutorials/...
2
votes
1answer
64 views

How do sigmoid functions make it so that the prediction $\hat{y}$ indicates the probability that the observed value, $y$, is $1$?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.3 Choice of Activation and Loss Functions says the following: The choice of activation ...
0
votes
0answers
49 views

Why Word2Vec is called a neural model if no neural network is used in it?

Word2Vec model does not use any neural network. It uses logistic regression only. Consider the following paragraph from p:18 of Vector Semantics and Embeddings We’ll see how to do neural ...
0
votes
1answer
70 views

Can you use a graph as input for a neural network?

We want to try and distinguish real voices from (deep)fake voices using the graphs generated by a discrete fourier transform (generated from .wav audio files). We know from each image if it is a real ...
0
votes
1answer
26 views

Why do I have better RMSE when I don't scale the target? [closed]

I use PyTorch for training a simple neural net for a regression task on a dataset with 12 numerical features + target (target is the 13th column) + 2 categorical features Before training, I execute <...
0
votes
0answers
27 views

Is it possible to use Neural Networks with Contextual Bandit to learn the probability distributions instead of providing them?

I want to ask you if it's possible by using neural networks jointly with the Contextual Bandit algorithm to learn the probability distributions by which the rewards are computed as a function of the ...
0
votes
0answers
50 views

How do autoregressive attention mechanism work in multi-headed attention?

[LONG POST!!] I am working on a DNN model that works as an improviser to generate music sequences. The idea of generating music is based on taking a sequence of music nodes (their index representation)...
-1
votes
1answer
33 views

how to go from mathematical problem to neural network (and back)?

I am a little confused on how, you can find online papers that describe complex Machine Learning formulas in a mathematical/probabilistic way, and, in the other hands, easy tutorials that teach you ...
0
votes
2answers
40 views

Best way to use/learn ML for board-game reinforcement learning

I am relatively new to Python but I taught myself enough to code a two-player board game that is similar to chess. It has a simple Tkinter UI. Now I am dipping into machine learning, and I want to ...
2
votes
0answers
44 views

Loss function to minimize the distance between sets

Are there references or links to examples about loss functions "Distance Metrics" which could be used to minimize the distance between two sets for a neural network. More precisely, this ...
1
vote
0answers
17 views

Neural Network Regression Experiment Going Wrong

I've been trying to get a simple regression experiment going with a neural network and I would like some help interpreting what is going wrong. My goal is to see what level of regression accuracy I ...
0
votes
1answer
43 views

Which solutions are there to the problem of having too large activations before the softmax (or sigmoid) layer?

I'm trying to build a neural network (NN) for classification using only N-bit integers for both the activations and weights, then I will train it with some heuristic algorithm, based only on the NN ...
0
votes
0answers
17 views

Neural Networks different architectures but similar training curves

I have a base neural network architecture for (3D) image sequences classification, made of conv layers followed by a LSTM and dense layers. I have 3 similar architectures : 3 Conv -> 1 LSTM -> ...

1 2 3
4
5
45