Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0
votes
0answers
2 views

Deep NN architecture for predicting a matrix from a matrix and list of floats

I am trying to predict a matrix (size RxC) based on an input matrix (size RxC) and a list of floats ...
1
vote
0answers
5 views

ReLU function not behaving as per expectation

I implemented a simple neural network with 1 hidden layer. I used ReLU as activation function for the hidden layer and the output layer just uses the linear function. To check my implementation I ...
2
votes
1answer
54 views

How to create a neural network from a set of equations?

Say I have these equations: $$x_1 = x_2 + 2y_1 + b$$ $$x_2 = y_2 + c$$ $$y_1 = z + a$$ $$y_2 = y_3 + d$$ $$z = z_1 + e$$ $x_1$ depends on $x_2$ (depends on $y_2$ (depends on $y_3$)) and $y_1$ (depends ...
1
vote
1answer
17 views

Is reconciling shape discrepancies the only purpose of padding?

Padding is a technique used in some of the domains of artificial intelligence. Data is generally available in different shapes. But in order to pass the data as input to a model in deep learning, the ...
1
vote
0answers
26 views

Batch normalization for multiple datasets?

I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data. I was told that batch ...
0
votes
0answers
24 views

Why are not results reproducible when a model is loaded with tf.keras.models.load_model?

This is how my training script looks like, only the important parts. ...
1
vote
0answers
18 views

Is there any existing mechanism that allows us to pass input from randomly selected layers of neural network per iteration?

Consider the following neural network with $\ell$ layers. $$i_0 \rightarrow h_1 \rightarrow h_2 \rightarrow h_3 \cdots \rightarrow h_{\ell-1} \rightarrow o_{\ell} ,$$ where $i, h, o$ stands for ...
0
votes
0answers
27 views

Neural networks in models and in reality [closed]

I have recently read a modern book on neural systems in biology and found a lot of misconceptions between current models and real systems. At first, real neurons use both inhibitory (negative, -) and ...
1
vote
0answers
35 views

Why doesn't anyone use reinforcement learning to find the best possible alternative to backpropagation?

To be clear, I'm very uninformed on the topic of alternative learning algorithms to backprop, all my knowledge comes from articles like these: lets-not-stop-at-backprop backprop-alternatives we-need-a-...
2
votes
0answers
18 views

Methodologies for passing the best samples for a neural network to learn

Just an idea I am sure I read in a book some time ago, but I can't remember the name. Given a very large dataset and a neural network (or anything that can learn via something like stochastic gradient ...
0
votes
1answer
32 views

Using a Neural Network (LSTM) to approve/reject word-type sequences

I would like to train an LSTM neural network to either "approve" or "reject" a string based on the word-type sequence. For instance: "Mike's Airplane" would output "...
-1
votes
1answer
26 views

Regularize the weights of the neural networks to binary values?

I have multiple layers of neural network and I am trying to do an autoencoder network. I am interested to make the weights of the encoding layers of the neural networks to take only -1 and +1 and the ...
2
votes
0answers
34 views

How much research, approximately, is done in ANNs?

Does someone know where can I find information about how much research, nowadays, is done in ANNs? I've checked in this document Redes Neuronales: Conceptos básicos y aplicaciones, Universidad ...
0
votes
0answers
30 views

What are the input and output gradients in PyTorch?

Suppose I want to train a neural network with $m-$length inputs of form $x = [x_1, x_2, x_3, \cdots, x_m]$ and $n-$length outputs of form $y = [y_1, y_2, y_3, \cdots, y_n]$. Let the number of ...
0
votes
1answer
33 views

Where does batch normalization layers present in a neural network?

Batch normalization is a procedure widely used to train neural networks. Mean and standard deviation are calculated in this step of training. Since we train neural network by dividing training data ...
0
votes
1answer
32 views

Is precision of weights unimportant in neural networks?

While reading about Module in PyTorch, I came across a new data type called half datatype. half() method when calls on a Module ...
0
votes
0answers
29 views

What do buffers of a model in PyTorch store?

Consider the following method related to buffers in PyTorch ...
2
votes
1answer
223 views

Is the dropout technique specific only to neural networks?

In one Udemy course was mentioned that "dropout is unique to neural networks". However, I remember an example of decision trees where nodes that are not participating in the overall result ...
1
vote
1answer
39 views

What does it mean by “zeros the networks parameters gradients” in the context of training a neural network?

Consider the following PyTorch code ...
0
votes
0answers
15 views

How can implement a multi-task deep learning? [closed]

I have 3 classes (A, B, and C). I have 6 features: ...
1
vote
1answer
38 views

How is the convolution operation connected to neural networks?

I've been reading up on the convolution operation and neural networks. I understand that the convolution operation is defined as: $$(f * g)(t)=\int_{-\infty}^{\infty} f(\tau) g(t-\tau) d \tau$$ The ...
3
votes
2answers
77 views

Is there any difference between affine transformation and linear transformation?

Consider the following statements from A Simple Custom Module of PyTorch's documentation To get started, let’s look at a simpler, custom version of PyTorch’s Linear module. This module applies an ...
0
votes
0answers
15 views

What is the primary paper demonstrating that CNNs struggle with datasets containing ambiguities? [closed]

It is known that neural networks, such as convolutional neural networks, struggle with pattern recognition if training sets contain ambiguities (i.e. several labels can correspond to one and the same ...
0
votes
0answers
7 views

How to approach receipt interpretation with neural networks

I have never worked with neural networks before, but I want to learn and I have a specific use case that I would want to solve (or at least try to). I have developed software for multiple years, in a ...
0
votes
0answers
12 views

Comparing results of different image splicing methods on a part of the CASIA 2.0 dataset

So I am working on an image splicing detection algorithm using ResNet-50 model. I am using the CASIA 2.0 dataset which consists of 7491 Authentic images and 5123 Fake images. However out of the fake ...
0
votes
0answers
19 views

Does Keras or Tensorflow calculate backpropagation?

I am using tensorflow 2.1.0. Does Keras or Tensorflow calculate backpropagation? Where can I find the code for backpropagation in Keras and Tensorflow? Thanks, Jianqiao Huang
2
votes
1answer
31 views

What does “differentiable architecture” mean?

I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term "Differentiable Architecture", ...
0
votes
0answers
6 views

Why can we compute mutual information in deep neural networks in information bottleneck context?

In the famous Information bottleneck paper by Tishby(https://arxiv.org/abs/1703.00810), the author proposed a framework that the neural network can compress information. And they computed the mutual ...
1
vote
0answers
31 views

Do the terms multi-task and multi-output refer to the same thing in the context of deep learning?

Do the terms multi-task and multi-output refer to the same thing in the context of deep learning (with neural networks)? For example, do neural networks for multi-task learning use multiple outputs? ...
0
votes
0answers
32 views

Can a neural network be a universal Turing machine?

Can a neural network be configured to take another neural network as input and build a particular neural network? What I am curious about is whether it is possible in a neural network to receive other ...
0
votes
1answer
34 views

What are the labels of EMNIST?

I'm using pytorch and downloading the EMNIST dataset using ...
1
vote
0answers
12 views

What are the practical problems where full bayesian treatment is affordable?

Suppose, I have a problem, where there is rather a small number of training samples, and transfer learning from ImageNet or some huge NLP dataset is not relevant for this task. Due to the small number ...
1
vote
0answers
35 views

What causes high differences in neural network accuracy each run?

I trained a CNN using Keras in R to multi-dimensional image data for image classification of five classes. I realized that each run (I retrained the network on the same data for ten times), although I ...
0
votes
0answers
21 views

RLLib - What exactly do the avail_action and action_embed_size represent? How do they work with the action_mask to phase out invalid actions?

So, I'm fairly new to reinforcement learning and I needed some help/explanations as to what the action_mask and avail_action fields alongside the action_embed_size actually mean in RLlib (the ...
0
votes
0answers
16 views

Conditional input deep neural network

I need to input data conditionally to my deep network. In order to explain cases, I'd like to give an example. Assume that I have a 50-attribute dataset. For some attributes, a specific part of hidden ...
1
vote
1answer
40 views

Why is the input layer of a neural network usually not counted?

I came across the following statement from the caption of figure 7.8 from the textbook Neural Networks and Neural Language Models the input layer is usually not counted when enumerating layers Why ...
2
votes
1answer
49 views

Why is tanh a “smoothly” differentiable function?

The sigmoid, tanh, and ReLU are popular and useful activation functions in the literature. The following excerpt taken from p4 of Neural Networks and Neural Language Models says that ...
0
votes
0answers
7 views

What is the significance of the RegLoss colum in Neuralprophet

I recently made a forecast with neuralprophet and after training, I got a table with three columns; "SmoothL1Loss", "MAE" and "RegLoss". Please, I need to know the ...
0
votes
0answers
12 views

How many MAC operations are executed in one inference/training cycle of Google BERT?

I wonder if there is any information about the amount of MACs are executed for one training/inference cycle of Google BERT. I only found information about the number of layers and parameters here. ...
2
votes
1answer
1k views

Should I continue training if the neural network attains 100% training accuracy?

I have a neural network where there are two hidden layers. Each hidden layer has 128 neurons. The input layer has 20 inputs, and the output layer has 3 outputs. I have 1 million records of data. 80% ...
1
vote
1answer
52 views

How much money is spent training neural networks each year by companies such as Google and Facebook?

I am wondering what order of magnitude estimates for the following are for companies Google and Facebook, as well as total globally. What is the rough amount of money spent to train neural networks? ...
0
votes
0answers
14 views

Why doesn't a neuron activation depend on number of input (presynaptic) neurons?

In an artificial neural network, we usually use the same activation function for all neurons, independently of the number of input (presynaptic) neurons. However, usually, the number of input neurons ...
1
vote
3answers
113 views

Can people use neural networks without providing the set of training data?

It seems that neural networks (NNs) can be applied to supervised learning, unsupervised learning and reinforcement learning. Some people even train neural networks without the set of training data. If ...
0
votes
0answers
11 views

How to scale Computer Vision? How to implement Emotion detection from live video feed of N different video simultaneously?

I have a pipeline based on Scaled Yolov4 detection algorithm for faces which extract faces of users and then uses a CNN to ...
0
votes
0answers
22 views

Can people set loss function of neural network by themselves instead of choosing cross entropy or mean square error?

I found people used deep neural network to get optimal policy by solving a nonconvex optimization problem. Moreover, they didn't use any set of training data and claimed that it's the difference ...
0
votes
0answers
7 views

Image classification distributed inference (mobile/server)

I'd like to learn some stuff about distributed DNN inference and how it works in practice. So, let's consider the example of image classification and assume we have a mobile device which utilizes the ...
0
votes
2answers
46 views

How to design a neural network with arbitrary input and output length?

I am trying to build a neural network that has an input of $n$ pairs of integer values (where $n$ is random) and a corresponding output of a binary array with length $n$. The input will be a set of ...
1
vote
0answers
25 views

Convolutional Neural Network (CNN) with Tree architecture to organize the number of classes

At the moment, I have around 1.000 classes with accuracy and loss that are acceptable. In the long term, there could be more than 100.000 classes. The main problem is that every time a new class is ...
1
vote
0answers
18 views

What is a Silhouette Neural Network

I was going through a study in which I found something called a dilated Silhouette Neural Network. I want to know what it is, what it can do, and how it is better from a CNN? Link to the journal: Link
0
votes
0answers
16 views

How to manually optimize Neural Networks the most systematical way?

Do you have any ideas or guidance on how to do manual neural network optimization in the most systematic way? Especially when models train longer and the effects of hyperparameter fitting are very ...

1
2 3 4 5
43