Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
3
votes
0answers
22 views

Why does Batch Normalization work?

Adding BatchNorm layers improves training time and makes the whole deep model more stable. That's an experimental fact that is widely used in machine learning practice. My question is - why does it ...
1
vote
0answers
21 views

The MLP output of a neural network can be written as $\|x\|\|w_l\|\cos(\theta_l)$: why is the norm easier to maximize?

The MLP output of a neural network is a dot product between the weights and the input and therefore can be written as $\|x\|\|w_l\|\cos(\theta_l)$ (see this for more details), where $x$ is the input, $...
1
vote
0answers
31 views

What is the output of neuron $y_{2}$ at time step $t$?

In Fundamentals of Neural Networks: Architectures, Algorithms And Applications by Laurene V. Fausett on $\text{Page:32}$ it describes Hot and Cold perception modeling with McCulloch-Pitts Net the ...
-1
votes
0answers
14 views

Is there any use of having connections between nodes in the output layer of a neural network?

Is there any use of having connections between nodes in the output layer of a neural network? In some cases some outputs may depend on other outputs; by this logic, is it possible to have such neural ...
-2
votes
0answers
18 views

extract desire keyword/text pair

I am looking for extract keyword pair from text files. They might not be next to each other and do not have same pattern for each occurrence. And I would not think regex will works because there is no ...
0
votes
0answers
15 views

Is it possible to use RGB image with decimal values when feeding training data to CNN?

I am working with four grayscale images of float32 data type to perform regression using Keras. Three images are stacked using np.dstack to form a RGB data-set. The ...
0
votes
0answers
17 views

Design of the feed forward neural network with multiclass output and low number of inputs

I have a set of data items $X_{ij}$ each having $20$ columns (categorical data) and there are 30 different types of $X_{ij}$ As an input to NN, I want to provide a combination of these items ($8$ ...
1
vote
1answer
36 views

How to interpret this learning curve of my neural network?

How to interpret the following learning curves? Background: The accuracy starts at 50%, because the network has a binary output (0 or 1). I chose an exponentially decreasing learning rate of the ...
3
votes
3answers
40 views

Can predictions of a neural network using ReLU activation be non-linear (i.e. follow the pattern) outside of the scope of trained data?

Training on a quadratic function x = np.linspace(-10, 10, num=1000) np.random.shuffle(x) y = x**2 Will predict an expected quadratic curve between ...
0
votes
0answers
5 views

Feasibility f using Word2Vec embeddings as input for GANs

If I have a dataset of artwork and a list of say, 10-12 words that describe each piece of artwork, how feasible would it be to convert those words into a word2vec embedding and use that (plus lets say ...
2
votes
1answer
31 views

How to improve a trained model over time (i.e. with more predictions)?

I built a model using the tutorial on the TensorFlow site. It was a simple image classification neural network. I trained it and saved the model and weights together on a ...
0
votes
0answers
49 views

How to evaluate a Genome in NEAT

I am trying to implement NEAT from scratch by going through the original NEAT paper. I implemented a Genome class which consists of a list of Node Genes and ...
0
votes
0answers
10 views

Tiny yolo v4 dnn module opencv no detection [closed]

I have trained yolo-tiny-v4 on custom dataset on google colab and the detection works well . Then I've tried to load the yolo-tiny-v4 in other colab project with help of opencv's dnn module, no error ...
1
vote
0answers
16 views

Is it normal getting noise values in the error history along training iteration?

I'm giving my first steps in really learning machine learning. As an exercise in my online course, it was asked for me to code the Cost function of some neural network that should resolve the ...
0
votes
0answers
15 views

How do the trainable projection layer used in PRADO and pQRNN work?

Trainable projection layers are said to be a very powerful thing but after reading: https://www.aclweb.org/anthology/D19-1506.pdf https://arxiv.org/pdf/2101.08890.pdf I don't understand how it works....
-1
votes
1answer
27 views

Is a neural network an evolutionary algorithm? [closed]

Is a neural network not just an evolutionary algorithm with increased amount of parameters to represent, and optimize a problem in the world?
1
vote
0answers
13 views

In a convolutional neural network, how is the error delta propagated between convolutional layers?

I'm coding some stuff for CNNs, just relying on numpy (and scipy just for the convolution operation for pure performance reasons). I've coded a small network consisting of a convolutional layer with ...
1
vote
2answers
34 views

How can “any process you can imagine” be thought of as function computation?

I stumbled upon this passage when reading this guide. Universality theorems are a commonplace in computer science, so much so that we sometimes forget how astonishing they are. But it's worth ...
1
vote
0answers
26 views

What are some use cases of discrete optimization in Deep Learning?

When we talk of optimization, it usually boils down to gradient descent and its variants in the context of deep learning. However, I wonder if there are some works that use discrete optimization in ...
0
votes
1answer
34 views

How to transfer declarative knowledge into neural networks

Humans learn facts about the world like "most A are B" by own experience and by being told so (by other people or texts). The systems and mechanisms of storage and usage of such facts (by an ...
0
votes
0answers
16 views

What is the effect and behavior of using mixed weight instead of normal weight matrix?

Suppose I try to find appropriate matrix A in differential equation $\dot{X}=A X$ using RNN. Current state is $X=\begin{bmatrix} x_{1}\\ x_{2}\\ \end{bmatrix}$, and desired trajectory state is $X_d=\...
2
votes
0answers
23 views

How do I implement the cross-entropy-method for a RL environment with a continuous action space?

I found many tutorials and posts on how to solve RL environments with discrete action spaces using the cross entropy method (e.g., in this blog post for the OpenAI Gym frozen lake environment). ...
0
votes
0answers
17 views

MicroPython MicroMLP: How do I reward the program based on state?

I have been trying to use MicroMLP to teach a small neural network to converge to correct results. Ultimately, I want to have three outputs, one which is high priority (...
0
votes
0answers
24 views

Is it possible for an artificial neural network to be able to make predictions based on others made in previous moments?

I have an artificial intelligence project in my hands and my goal is to train an artificial neural network, a stacked hourglass network (SHN) in particular, to predict the jointed skeleton of dogs. ...
0
votes
0answers
14 views

Do dataset sizes matter in a Style GAN?

When working with classifiers, a class imbalance is a huge issue for our models. If we have too many images of class 1 and too few images from ...
0
votes
1answer
42 views

Is the final model scaling done on the full training set?

We have our training set and our test set. When we scale our data we "fit" the scaler transform to the training set and then we scale both the training set and test set using this scaler ...
0
votes
0answers
16 views

Multi dimensional LSTM modeling in KERAS

I have a database of time series signals with multiple features and Im trying to build a model to predict whether or not two samples are related to each other. For example : a database of 1000 sample ...
0
votes
1answer
40 views

XOR Neural Network gets stuck in training

I'm trying to create a neural network to simulate a XOR gate. Here's my dataset: ...
0
votes
0answers
9 views

Balancing two parallel imbalanced submodels while traning a deep neural network

My regression model has two parallel subnetworks A and B, which process a shared input into respective temporary outputs a and b. An input propagates along A and B independently, and a plane sum of a ...
0
votes
1answer
46 views

What are possible ways to combat overfitting or improve the test accuracy in my case?

I have asked a question here, and one of the comments suggested that this is a case of severe overfitting. I made a neural network, which uses residual boosting (which is done via a KNN), and I am ...
0
votes
2answers
74 views

Could I just choose the other (non-predicted) class when the accuracy is low?

I have a binary classification problem. My neural network is getting between 10% and 45% accuracy on the validation set and 80% on the training set. Now, if I have a 10% accuracy and I just take the ...
0
votes
0answers
15 views

How to have a DNN output a classification for each user at once?

I have a Reinforcement Learning environment with an agent that allocates power values to different users. To do so, I have thought of implementing a deep neural network like the one shown in the ...
0
votes
0answers
32 views

Is vectorizing backpropagation feasible?

Does it make sense to have the backpropagation of a neural network layer happen all at once if the learning rate is lowered? This would mean the new weights of that layer would be independent of each ...
2
votes
1answer
46 views

What is the intuition behind variational inference for Bayesian neural networks?

I'm trying to understand the concept of Variational Inference for BNNs. My source is this work. The aim is to minimize the divergence between the approx. distribution and the true posterior $$\text{KL}...
0
votes
1answer
30 views

How to train a sequence labeling model with annotations from three annotators?

I have a dataset of movie reviews annotated by 3 persons. The following example contains one sentence with corresponding annotations from 3 different persons. ...
1
vote
0answers
32 views

What are some most promising ways to approximate common sense and background knowledge?

I learned from this blog post Self-Supervised Learning: The Dark Matter of Intelligence that We believe that self-supervised learning (SSL) is one of the most promising ways to build such background ...
0
votes
0answers
11 views

How to predict when a number will occur again, give an array of integers?

I am new to machine learning and AI. I'm trying to create a program that adds a random number (1-5) to an array every 1 second and at a random point stops doing that. If that's the case, then it shall ...
0
votes
0answers
6 views

To find the accuracy between two patterns by comparing the similarity socre

I have faced problem with finding the similarity scores between two patterns. For example, I have normal ECG pattern and abnormal ECG pattern . Then I want to get find the accuracy of normal pattern ...
0
votes
0answers
33 views

What are the best optimizations I can add to my neural network?

I am making an artificial neural network from scratch (without nn libraries) in python. So, as you can guess, its extremely unoptimized and slow. For this neural ...
0
votes
2answers
80 views

Is my approach to building an RNN to predict the probability that the word is in English appropriate?

Goal To build an RNN which would receive a word as an input, and output the probability that the word is in English (or at least would be English sounding). Example ...
0
votes
2answers
31 views

How can I model any structure for a neural network?

Hello I am currently doing research on the effect of altering a neural network's structure. Particularly I am investigating what affect would putting a random DAG (directed acyclic graph) in the ...
2
votes
1answer
35 views

What is this algorithm? Is it a variant of Monte-Carlo Tree Search?

I'm using a Neural Network as an agent in a simple car racing game. My goal is to train the network to imitate a brute-force tree search to an arbitrary depth. My algorithm goes something like the ...
0
votes
1answer
57 views

Can we use Multiple data as Input in a NN for a single Output?

So I am new to NN and I'm trying to go deep and apply it to my subject. I would like to ask: the input of the NN can be 2 or more values for example-> the measurement of a value, distance, and time?...
0
votes
0answers
29 views

Are there regularisation methods related only to architecture of the CNNs?

Are there any methods of regularisation of deep neural networks, particularly CNNs (or generally ANN but that will also work on CNNs) that are related only to the network's architecture and not the ...
1
vote
0answers
14 views

Finding Specific Patterns in Data

I'm trying to research modeling that can help me find very specific patterns in data. I've done a fair amount of work about generalized predictions with machine learning, but I'm very confused about ...
1
vote
1answer
32 views

Preparing data set for the YOLO algorithm

Hi I am working on a project which requires the You Only Look Once algorithm in order to classify and localise objects within images. I have to prepare my dataset (which has 2 classes, and predicts 6 ...
0
votes
0answers
11 views

How to update all the weights in case only one data out of n signals is observable

If we have cost function as $$E_i = (D_i -Y_i)^T Q (D_i -Y_i)$$, where $$Q=\begin{bmatrix} 1 & 0 & 0\\ 0 & 0 & 0\\ 0 & 0 & 0 \end{bmatrix}$$( in case only one data signal can ...
0
votes
0answers
19 views

Should an increased learning rate for an adaptive linear neuron (ADALINE) reduce the square error at every epoch?

I am completely new to neural networks and therefore, my query may have some basic conceptual problem. I am following Fundamentals of Neural Networks by Laurene Fusett. In this book, the author ...
0
votes
1answer
23 views

Validation Accuracy remains constant while training VGG?

I posted this question on stackoverflow and got downvoted for unmentioned reason, so I'll repost it here, hoping to get some insights This is the plot This is the code: ...
2
votes
1answer
105 views

Why is no activation function needed for the output layer of a neural network for regression?

I'm a bit confused about the activation function in the output layer of a neural network trained for regression. In most tutorials, the output layer uses "sigmoid" to bring the results back ...

1
2 3 4 5
41