Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
2
votes
1answer
51 views

What does "differentiable architecture" mean?

I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term "Differentiable Architecture", ...
0
votes
0answers
15 views

Why can we compute mutual information in deep neural networks in information bottleneck context?

In the famous Information bottleneck paper by Tishby(https://arxiv.org/abs/1703.00810), the author proposed a framework that the neural network can compress information. And they computed the mutual ...
1
vote
0answers
34 views

Do the terms multi-task and multi-output refer to the same thing in the context of deep learning?

Do the terms multi-task and multi-output refer to the same thing in the context of deep learning (with neural networks)? For example, do neural networks for multi-task learning use multiple outputs? ...
2
votes
0answers
20 views

What are the practical problems where full bayesian treatment is affordable?

Suppose, I have a problem, where there is rather a small number of training samples, and transfer learning from ImageNet or some huge NLP dataset is not relevant for this task. Due to the small number ...
0
votes
0answers
108 views

RLLib - What exactly do the avail_action and action_embed_size represent? How do they work with the action_mask to phase out invalid actions?

So, I'm fairly new to reinforcement learning and I needed some help/explanations as to what the action_mask and avail_action fields alongside the action_embed_size actually mean in RLlib (the ...
0
votes
0answers
37 views

Conditional input deep neural network

I need to input data conditionally to my deep network. In order to explain cases, I'd like to give an example. Assume that I have a 50-attribute dataset. For some attributes, a specific part of hidden ...
1
vote
1answer
56 views

Why is the input layer of a neural network usually not counted?

I came across the following statement from the caption of figure 7.8 from the textbook Neural Networks and Neural Language Models the input layer is usually not counted when enumerating layers Why ...
2
votes
1answer
178 views

Why is tanh a "smoothly" differentiable function?

The sigmoid, tanh, and ReLU are popular and useful activation functions in the literature. The following excerpt taken from p4 of Neural Networks and Neural Language Models says that ...
0
votes
0answers
18 views

What is the significance of the RegLoss colum in Neuralprophet

I recently made a forecast with neuralprophet and after training, I got a table with three columns; "SmoothL1Loss", "MAE" and "RegLoss". Please, I need to know the ...
0
votes
0answers
35 views

How many MAC operations are executed in one inference/training cycle of Google BERT?

I wonder if there is any information about the amount of MACs are executed for one training/inference cycle of Google BERT. I only found information about the number of layers and parameters here. ...
2
votes
1answer
1k views

Should I continue training if the neural network attains 100% training accuracy?

I have a neural network where there are two hidden layers. Each hidden layer has 128 neurons. The input layer has 20 inputs, and the output layer has 3 outputs. I have 1 million records of data. 80% ...
1
vote
1answer
71 views

How much money is spent training neural networks each year by companies such as Google and Facebook?

I am wondering what order of magnitude estimates for the following are for companies Google and Facebook, as well as total globally. What is the rough amount of money spent to train neural networks? ...
0
votes
0answers
15 views

Why doesn't a neuron activation depend on number of input (presynaptic) neurons?

In an artificial neural network, we usually use the same activation function for all neurons, independently of the number of input (presynaptic) neurons. However, usually, the number of input neurons ...
1
vote
3answers
126 views

Can people use neural networks without providing the set of training data?

It seems that neural networks (NNs) can be applied to supervised learning, unsupervised learning and reinforcement learning. Some people even train neural networks without the set of training data. If ...
0
votes
0answers
13 views

How to scale Computer Vision? How to implement Emotion detection from live video feed of N different video simultaneously?

I have a pipeline based on Scaled Yolov4 detection algorithm for faces which extract faces of users and then uses a CNN to ...
0
votes
0answers
25 views

Can people set loss function of neural network by themselves instead of choosing cross entropy or mean square error?

I found people used deep neural network to get optimal policy by solving a nonconvex optimization problem. Moreover, they didn't use any set of training data and claimed that it's the difference ...
0
votes
2answers
102 views

How to design a neural network with arbitrary input and output length?

I am trying to build a neural network that has an input of $n$ pairs of integer values (where $n$ is random) and a corresponding output of a binary array with length $n$. The input will be a set of ...
1
vote
0answers
27 views

Convolutional Neural Network (CNN) with Tree architecture to organize the number of classes

At the moment, I have around 1.000 classes with accuracy and loss that are acceptable. In the long term, there could be more than 100.000 classes. The main problem is that every time a new class is ...
1
vote
0answers
30 views

What is a Silhouette Neural Network

I was going through a study in which I found something called a dilated Silhouette Neural Network. I want to know what it is, what it can do, and how it is better from a CNN? Link to the journal: Link
0
votes
0answers
17 views

How to manually optimize Neural Networks the most systematical way?

Do you have any ideas or guidance on how to do manual neural network optimization in the most systematic way? Especially when models train longer and the effects of hyperparameter fitting are very ...
0
votes
2answers
49 views

Is pruning only applicable to convolutional neural networks?

This article talks about pruning in the context of convolutional neural networks: One of the first methods of pruning is pruning entire convolutional filters. Using an L1 norm of the weight of all ...
0
votes
0answers
26 views

Regress values inside the bounding boxes to predict a value in Object Detection

I am currently working on an object detection task. I have a dataset of Grayscale and Depth Images. The annotation format is x1, y1, x2, y2, class, depth. I have calculated this depth (of each object/...
0
votes
0answers
29 views

What type of neural network do I need?

I am working on protein structure prediction. Suppose, I am solving a problem using Neural Networks. I know how many inputs and outputs there will be in the model, as it directly depends on the ...
0
votes
1answer
41 views

Is it valid to implement hyper-parameter tuning and THEN cross-validation?

I have a multi-label classification task I am solving. I have done hyperparameter tuning (with Keras Tuner) to determine the best configuration for my neural network. Is it valid to do this (determine ...
0
votes
0answers
5 views

Evaluating a CNN -multi class model with two separate thresholds

I have a model that outputs three classes. But here instead of one threshold, it depends on a combination of two (user input threshold). One threshold varies from 0.1 to 1.0 and the other varies from ...
1
vote
1answer
23 views

What are the labels in figure1 in the Paper "The perceptron: A probabilistic model for information storage and organization in the brain"?

This figure comes from The perceptron: A probabilistic model for information storage and organization in the brain I guess the first circle (neuron) labels RETINA, the second labels perceptron area, ...
1
vote
0answers
30 views

Is the formula $\frac {1}{s}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|$ the correct form of 0-1 loss function, in the context of Perceptron?

Per page 7 of this MIT lecture notes, the original single-layer Perceptron uses 0-1 loss function. Wikipedia uses $${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|} \tag{1}$$ to denote ...
2
votes
1answer
62 views

When does a neural network have a single and when does it have multiple outputs?

What I understand is, each input in a neural network is a feature. However, what I don't understand is, when we need multiple outputs in a neural network. For example, say, if we are classifying cats ...
1
vote
0answers
17 views

More channels vs multiple inputs in neural network

Suppose I want to train the model for playing chess. I found that existing models use as input the grid with dimensions 8x8x20 (so we have 20 channels). Some channels may represent how different kind ...
0
votes
0answers
25 views

Recommended literature on layers for reinforcement learning

I was recommended to ask here after I posted on stack overflow wrongly. I was wondering if anyone had any recommended readings on layers used in neural networks for reinforcement learning? I've been ...
1
vote
1answer
83 views

How does the neural network learn when used in the REINFORCE algorithm?

As per my understanding, you run an entire episode, which contains many steps, and then back-propagate using just a single loss value. How does the neural network learn to differentiate between good ...
0
votes
0answers
36 views

AI model to predict/generate person's image

I want to make a model that predicts person's shape depending on his son's image. My plan is to create a dataset and each data point in it consists of two images; One for the father or mother and one ...
2
votes
1answer
87 views

What gets optimized in convolutional neural network?

In a convolutional neural network, the hyperparameters such as number of kernels and stride, kernel size, etc are determined. After some combination of convolutions, ReLU and pooling layer there is ...
1
vote
0answers
16 views

Are there guiding principles as to which activation functions suit a given RL algorithm?

Are there rules of thumb as to which activation functions work well (or which one would not) on the policy and value network of a class of RL algorithms? For hidden layers and for the output layer. ...
1
vote
0answers
20 views

How to choose proper normalization strategy for the activations?

I am reading a survey on various normalization techniques adopted in neural network architectures. The purpose of introducing normalization is understandable - to stabilize the training and avoid ...
0
votes
0answers
21 views

Is the compression function for SHA hash algorithms a hidden layer in a neural net?

Is the compression function for SHA-256 and SHA-512 a "hidden layer" in a neural net? If so, what type of neural net is it in? SHA-256 and SHA-512 compression function: source: NIST, “...
2
votes
0answers
51 views

In NEAT, how do node numbers work?

I have read a lot of debates about node ids and such. I'm not 100% sure how it works, but I am assuming the next node added to a network would be the next number in that specific networks list? For ...
3
votes
0answers
49 views

How does the paper implement NEAT without a global set tracking Innovations?

I have been reading this paper on NEAT and trying to implement the algorithm in C#. For the most part, I understand everything in the paper however, there are 2 things I don't understand that confuse ...
1
vote
3answers
73 views

Is there a recent book that covers the theoretical and philosophical aspects of artificial intelligence?

What are some recent books that introduce AI and neural networks while also discussing the related philosophical issues, like epistemology and whether AI is really thinking, etc.?
0
votes
0answers
24 views

How to obtain STD from Neural Network with 2 continuous action output

In my Environment, I have two continuous action space self.action_space = spaces.Box(low=np.array([0.,0.]), high=np.array([4.,0.02]), shape=(2,), dtype=np.float32) ...
0
votes
0answers
49 views

GAN performs worse after 50 epochs than after 2

I am training GAN on SVHN dataset (house numbers in Google Street View images, dimensions: 3x32x32 - 3 color channels). The problem is that it performs worse after some training (e.g. after 50 epochs) ...
0
votes
1answer
11 views

Can we modelize an RNN by an ANN that takes precedent output as a part of input?

Is it possible to consider an RNN as a classical feedforward neural network that just take the precedent output as a part of the input ?
0
votes
0answers
38 views

Is NEAT speciation really effective?

I tried implementing NEAT algorithm from scratch, and it successfully solves XOR problem. I followed the original NEAT paper. However, when I run XOR problem solving test and calculate average ...
3
votes
1answer
124 views

Are these visualisations the filters of the convolution layer or the convolved images with the filters?

There are several images related to convolutional networks on the Internet, an example of which I have given below My question is: are these images the weights/filters of the convolution layer (the ...
2
votes
0answers
22 views

Can Neural Networks using ReLU activation work without using the bias term in their neurons?

I created a super simple NN of 1 input, 2 hidden layers of 2 neurons each and 1 output neuron as shown below. All activations are ReLUs and neurons doesn't use the bias term. What I found is that the ...
1
vote
0answers
47 views

Backpropagation not working as expected

I'm new to neural networks and I try to make a model that is guessing if a point is below or above relative to a function output. The idea is inspired from this video https://youtu.be/DGxIcDjPzac . ...
1
vote
1answer
63 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
2
votes
1answer
53 views

What is the state of the art in melody generation?

Generative Adversarial Networks can generate realistic photos of people, such as thispersondoesnotexist.com. I wonder whether one can train an artificial intelligence on a batch of plain solo melodies ...
1
vote
0answers
40 views

Identifying rotating and resizing letters with background noise

I'm trying to complete a captcha, and here is what it looks like: Between captchas the calligraphy of the letters is the same, but the letters may be resized and rotated. And the background noise (...
0
votes
0answers
27 views

Metric Learning with l2 distance and contrastive loss is not working

I am trying metric learning with L2 distance and contrastive loss with a pre-trained language transformer as an embedding extractor. I ran my model for 20 epochs, and the loss is decreasing. But when ...

1
3 4
5
6 7
43