Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
2
votes
2answers
50 views

How do we know that the neurons of an artificial neural network start by learning small features?

I'd like to ask you how do we know that neural networks start by learning small, basic features or "parts" of the data and then use them to build up more complex features as we go through ...
1
vote
1answer
44 views

What are examples of good free books that cover the back-propagation algorithm?

What are examples of good free books that cover the back-propagation used to train multilayer perceptrons? I've just started to learn about artificial neural networks, so I'm looking for books that ...
0
votes
0answers
14 views

Can an Autoencoder be used to create a simple text-completion plugin (code-completion for programming)?

Can an Autoencoder neural network be used to create a simple code completion plugin for a developer using a certain programming language ? The idea is that the training data will be generated from ...
2
votes
0answers
18 views

If I want to predict two unrelated values given the same sequence of data points, should I have a model with two outputs or two models?

I want to predict two separate y-values (not really logically connected) based on an input sequence of data (values x). Using LSTM cells. Should I train two models separately or should I just increase ...
2
votes
2answers
38 views

In classification, how does the number of classes affect the model size and amount of data needed to train?

When solving a classification problem with neural nets, be it text or images, how does the number of classes affect the model size and amount of data needed to train? Are there any soft or hard ...
4
votes
0answers
74 views

Why did the developement of neural networks stop between 50s and 80s?

In a video lecture on the development of neural networks and the history of deep learning (you can start from minute 13), the lecturer (Yann LeCunn) said that the development of neural networks ...
0
votes
1answer
33 views

Why Autoencoder Weights Are Not Always Tied

To me, tying weights in an autoencoder makes sense if we think of the auto encoder as doing PCA. Why in any situation would it make sense to not tie the weights? If we don't tie the weights, would it ...
2
votes
0answers
52 views

Bias gradient of layer before batch normalization always zero

From the original paper and this post we have that batch normalization backpropagation can be formulated as I'm interested in the derivative of the previous layer outputs $x_i=\sigma(w X_i+b)$ with ...
-1
votes
0answers
27 views

I'm designing a Neural Network in Matlab using the Sigmoid Function to complete a specific goal that should be trivial- and I am stuck

For a graduate school project, I'm crafting the Neural Netowrk that I breifly talked about in the title. I've been working on it for a while. This might sound contradictory, but I'm very happy with ...
6
votes
2answers
269 views

Is there an ideal range of learning rate which always gives a good result almost in all problems?

I once read somewhere that there is a range of learning rate within which learning is optimal in almost all the cases, but I can't find any literature about it. All I could get is the following graph ...
2
votes
0answers
29 views

Should I use batch gradient descent when I have a small sample size?

I have a dataset with an input size of 155x155, with the output being 155 x 1 with a 3-4 layer neural net being used for regression. With such a small sample size, should I use full batch gradient ...
0
votes
0answers
31 views

Is there any research work on known malware detection systems based on AI?

I'm working on writing an article about the possibilities of modern AI-based algorithms to produce invisible self-learning malware, that can distribute itself throughout the internet and create ...
3
votes
1answer
57 views

GAN Generator Output w/ Periodic Noise

I am training a Semi-Supervised GAN, using multivariate time-series with window of shape (180*80) with the generator and discriminator architecture below. My data is scaled using Robust Scaler, so I ...
1
vote
1answer
43 views

Is it possible to make a neural network to solve this “reaction time test”?

I'm thinking about writing an essay on the comparison between the human nervous system (reaction time) and a neural network that does the same reaction time test. I am very new in this area, so I was ...
0
votes
0answers
45 views

Is it possible to transform audio with neural networks to make it sound like 3d sound

so the idea is to feed neural network data like input: mono audio(extracted from existing 3d audio) output: 3d audio after training it should convert mono audio to 3d sound do you think it is possible?...
0
votes
1answer
79 views

Is it possible to predict $x^2$, $\log(x)$, or variable function of $x$ using RNN?

There were some posts that using RNN can predict the next point of the sine wave function with data history. However, I wondered if it also works on all the functions of $x$, such as $x^2$, $x^3$, $\...
2
votes
1answer
63 views

Would it be possible to determine the dataset a neural network was trained on?

Let's say we have a neural network that was trained with a dataset $D$ to solve some task. Would it be possible to "reverse-engineer" this neural network and get a vague idea of the dataset $...
1
vote
1answer
60 views

Is there a graph neural network algorithm that can deal with a different number of input and output nodes?

I am new to graph neural networks and their applications. I have an input graph $G = \{V, E\}$ and an output graph $G' = \{V', E'\}$ where the number of nodes $V$ and $V'$ are different. I am trying ...
1
vote
0answers
29 views

Is the framework provided by this paper for checking the constraints of AI systems really new?

The authors of this paper present a framework for checking the constraints of AI systems using formal argumentative logic between 2 agents: an interrogating agent and a suspect agent. The ...
1
vote
1answer
45 views

What is the definition of the hinge loss function?

I came across the hinge loss function for training a neural network model, but I did not know the analytical form for the same. I can write the mean squared error loss function (which is more often ...
0
votes
0answers
11 views

How to check if labels are multimodal for continuous data distribution

For example, If we have an object pointcloud which maps to two labels: -90 deg rotation and +90 deg rotation In general, if we have a non-functional mapping in the data (e.g. one input has two ...
0
votes
0answers
26 views

Understanding policies in helicopter control in the paper by Andrew Ng et al

I was going through this paper on helicopter flight control using reinforcement learning by Andrew Ng et al. It defines two policy classes to learn two policies, one for hovering the helicopter and ...
0
votes
0answers
28 views

Paillier scheme : Encoding floats into integers impact on accuracy in neural networks

In Privacy Preserving Processing Over Encrypted Images, I could understand that appropriate encoding of floats into integers (required in Paillier) only incur negligible error in computations. Can we ...
0
votes
1answer
53 views

Extracting information from RNA sequence

I am relatively new to machine learning, and I am trying to use a deep neural network to extract some information from sequences of RNA. A quick overview of RNA: there is both sequence and structure. ...
0
votes
1answer
39 views

classification of unseen classes of image in open set classification

I have a scanned image, and they need to be classified in one of the pre-defined image classes, so that it can be sorted. However, the problem is the open nature of the classes. At testing time, new ...
4
votes
0answers
46 views

Can the quality of randomness in neural network initialization affect model fitting?

This is a topic I have been arguing about for some time now with my colleagues, maybe you could also voice your opinion about it. Artificial neural networks use random weight initialization within a ...
1
vote
0answers
16 views

Dealing with bias in multi-channel auto encoders

The problem I have a multi-channel 1D signal I want to auto-encode. I am unable to resonstruct the input when the number of channels increases. Code I am using a convolutional encoder, and a ...
0
votes
0answers
33 views

Data Augmentation of store images using handwritten labels

I am new to AI and NN. I've started learning using Geron's book on Tensorflow. My first project ("Smart Shelf") is to determine which items in a store have been purchased and need refilled. ...
0
votes
0answers
22 views

How to change the number of input neurons in embedding layer?

I was building a recommender system using Tensorflow recommenders (TFRS) library . I was following the official tutorial for ranking model , where they have used two-tower model. The part where I have ...
1
vote
1answer
103 views

Is there a convention on the order of multiplication of the weights with the inputs in neural nets?

Is there a convention on how the input data and the weights are multiplied? The input data can be anything, including the result from the previous layers. There are two options: Option 1: $$\begin{...
4
votes
0answers
35 views

Visualizing the Loss Landscape of Neural Nets: Meaning of the word 'filter'?

I found myself scratching my head when I read the following phrase in the paper Visualizing the Loss Landscape of Neural Nets: To remove this scaling effect, we plot loss functions using filter-wise ...
-1
votes
0answers
20 views

How to detect and eliminate output bias in a neural network?

I am working on applying Neural networks for a regression problem on images and the outputs of the model that I trained are always of a lower value than the actual values. Does this certify as a bias ...
4
votes
1answer
118 views

Why do feedforward neural networks require the inputs to be of a fixed size, while RNNs can process variable-size inputs?

Why does a vanilla feedforward neural network only accept a fixed input size, while RNNs are capable of taking a series of inputs with no predetermined limit on the size? Can anyone elaborate on this ...
0
votes
0answers
23 views

What is the purpose of “alignment” in the self-attention mechanism of transformers?

I've been reading about transformers & have been having some difficulty understanding the concept of alignment. Based on this article Alignment means matching segments of original text with their ...
2
votes
1answer
34 views

Is a learned policy, for a deterministic problem, trained in a supervised process, a stochastic policy?

If I trained a neural network with 4 outputs (one for each action: move down, up, left, and right) to move an agent through a grid (deterministic problem). The output of the neural network is a ...
0
votes
0answers
24 views

Are all activation functions in a layer same? [duplicate]

I understand that for you can have multiple activation functions in different layers. CNN's usually have Relu followed by softmax for the classification. But what stops us in having multiple ...
1
vote
2answers
92 views

What is the target output for updating a Deep Q Network

I'm trying to implement Deep Q-Learning for a pet problem having a continuous state space and discretized action space. The algorithm for table-based Q-Learning updates a single entry of the Q table - ...
3
votes
0answers
37 views

Why does sigmoid saturation prevent signal flow through the neuron?

As per these slides on page 35: Sigmoids saturate and kill gradients. when the neuron's activation saturates at either tail of 0 or 1, the gradient at these regions is almost zero. the gradient and ...
2
votes
1answer
214 views

Are there any active areas of research in machine learning that do not involve neural networks at all?

So far, I have not been able to find many papers that do not involve neural networks, so I was hoping I can gain some insight here. Any references would be greatly appreciated.
1
vote
0answers
27 views

If one of the inputs to a neural network (that represents a policy) is noisy and degrades the performance, would this architecture solve the issue?

I'm using genetic algorithms to train deep reinforcement learning (DRL) agents, similarly to what was done in this paper. DRL policies are therefore represented by deep neural networks, which map ...
0
votes
1answer
48 views

Reasoning behind performance improvement with hopfield networks

In the paper Hopfield networks is all you need, the authors mention that their modern Hopfield network layers are a good replacement for pooling, GRU, LSTM, and attention layers, and tend to ...
4
votes
0answers
48 views

Why are most commonly used activation functions continuous?

I have come to notice that the most commonly used activation functions are continuous. Is there any specific reason behind this? Results such as this paper have worked on training networks with ...
6
votes
0answers
49 views

Why is my GAN more unstable with bigger networks?

I am working with generative adversarial networks (GANs) and one of my aims at the moment is to reproduce samples in two dimensions that are distributed according to a circle (see animation). When ...
0
votes
0answers
17 views

How does the embeddings work in vision transformer from paper?

I get the part from the paper where the image is split into P say 16x16 (smaller images) patches and then you have to ...
1
vote
1answer
54 views

What is the difference between out of distribution detection and anomaly detection?

I'm currently reading the paper Likelihood Ratios for Out-of-Distribution Detection, and it seems that their problem is very similar to the problem of anomaly detection. More precisely, given a neural ...
0
votes
1answer
41 views

Is the working of RNNs, LSTM and GRU sequential or parallel?

You take any blog or any example and all they tell you about is the given picture below. It has 4 different matrices and 3 of whose weights are shared. So, I'm wondering how is this achieved in ...
2
votes
0answers
44 views

Reinforcement learning and Graph Neural Networks: Entropy drops to zero

I am currently working on an experiment to link reinforcement learning with graph neural networks. This is my architecture: Feature Extraction with GCN: there is a fully meshed topology with ...
1
vote
1answer
42 views

Can cryptocurrency charts be estimated using neural networks?

If I were to make a neural network that predicts the value of e.g. Bitcoin tomorrow based on the chart of the last month, would that work? Of course, 100% accuracy cannot be reached, but a success ...
1
vote
2answers
77 views

How to recognize sequence of digits in an image

I am learning to program neural networks and others, and I would like to know how I can get the numbers that are in an image, for example if I pass an image that has 123 written, get with my model ...
2
votes
0answers
20 views

For binary classification learning problems, how should I label instances where I'm only 60% sure?

I've come across a few binary classification problems lately where the labelling was challenging even for an expert. I'm wondering what I should do with this. Here are some of my suggestions to get ...

1
2 3 4 5
40