Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Unable to meet desired mean squared error

I wish to get MSE < 0.5 on test data (https://easyupload.io/zr7xf3) which is 20% of given data chosen randomly. But I am reaching 0.73 using both plain Ridge Regression as well as a neural network ...
4
votes
2answers
1k views

How to perform neural network with output constraint?

Imagine a "simple" feedforward, fully connected neural network, with some input size, some number of hidden layers, and some # of neurons....etc BUT with a fixed number of output size (that is saying, ...
1
vote
2answers
81 views

Is it possible to perform neuroevolution without a fitness function?

My question is about neuroevolution (genetic algorithm + neural network): I want to create artificial life by evolving agents. But instead of relying on a fitness function, I would like to have the ...
0
votes
0answers
14 views

Transforming neural network target values before training

Consider the scenario in which I am measuring certain $f(a,x)$, which i want to be the target value for some related input $g(a,x)$. In other words, I am trying to map $$g(a,x)\Rightarrow f(a,x)$$ I ...
0
votes
1answer
52 views

Predicting probabilities of events using neural networks

I've got a few thousands of sequences like 1.23, 2.15. 3.19, 4.30, 5.24, 6.22 where the numbers denote times on which an event happened (there's just a single ...
2
votes
0answers
34 views

How efficient is SCAWI weight initialization method?

I'm currently in the middle of a project (for my thesis) constructing a deep neural network. Since I'm still in the research part, I'm trying to find various ways and techniques to initialize weights. ...
1
vote
2answers
53 views

How is the error calculated with multiple output neurons in the neural network?

Machine Learning books generally explains that the error calculated for a given sample $i$ is: $e_i = y_i - \hat{y_i}$ Where $\hat{y}$ is the target output and $y$ is the actual output given by the ...
3
votes
1answer
41 views

Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously?

Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously? Is there an agreed up protocol on how to extract this information? What ...
0
votes
0answers
21 views

Different predictions across DL Frameworks [closed]

Can anyone give me a reason as to why I can train a neural network in say Tensorflow Flow, build equivalent models in pytorch and keras and any other DL framework, load the weights from the tensorflow ...
0
votes
0answers
16 views

What is the best neural network model to classify an x(t) signal according two classes?

I am a beginner in AI methods. I have a collection of x(t) data, where x are some signal amplitudes and t is a time. My testing data are divided into two classes, say those from good and bad ...
3
votes
1answer
41 views

Improving DQN with fluctuations

Hello :) I'm pretty new to this community, so let me know if I posted anything incorrectly and I'll try to change it. I'm working on the project which aim is to create self-driving agent in CARLA. I ...
0
votes
0answers
11 views

XOR problem with bipolar representation

I am taking a course in Machine Learning and the Professor introduced us to the XOR problem. I understand the XOR problem is not linearly separable and we need to employ Neural Network for this ...
1
vote
1answer
13 views

How do LSTMs work if the following two matrices are not able to be multiplied?

In the above diagram, the shape of some of the matrices can be seen in the yellow highlight. For instance: The hidden state at timestep t-1 ($h_{t-1}$) has shape $(na, m)$ The input data at timestep t ...
2
votes
0answers
19 views

how to handle highly imbalanced multilabel classification?

I am working on a multilabel classification in which I am having 206 labels. When I saw the percentage of the number of 1's in each label they are way less than 0.1% for each label. The maximum ...
6
votes
1answer
97 views

Smallest possible network to approximate the $sin$ function

The main goal is: Find the smallest possible neural network to approximate the $sin$ function. Moreover, I want to find a qualitative reason why this network is the smallest possible network. I have ...
5
votes
1answer
73 views

How does the network know which objects to track in the paper “Label-Free Supervision of Neural Networks with Physics and Domain Knowledge”?

I was reading the paper Label-Free Supervision of Neural Networks with Physics and Domain Knowledge, published at AAAI 2017, which won the best paper award. I understand the math and it makes sense. ...
1
vote
0answers
28 views

Dynamically adapting activation function

I am training a network through reinforcement learning. The policy network learns rotations, but depending on the actual input (state), the output of the network should be restricted to be in certain ...
2
votes
1answer
190 views

Can the hidden layer prior to the ouput layer have less hidden units than the output layer?

I attended an introductory class about neural network and I had a question regarding how to choose the number of hidden units per hidden layer. I remember that the Professor saying that there is no ...
26
votes
3answers
10k views

Is it possible to train a neural network incrementally?

I would like to train a neural network where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every time ...
0
votes
0answers
24 views

How did they use their dataset with VAEs?

Old Photo Restoration via Deep Latent Space Translation (https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space) In the article, it says : "We propose to restore old photos ...
1
vote
1answer
27 views

How to define Agar.io state and action space?

I am trying to implement an AI bot for my Agar.io clone using deep neural network. However, I am struggling with the state and action space of the AI bot. Because the bot can take real number for ...
1
vote
1answer
40 views

What is the status of the capsule networks?

What is the status of the capsule networks? I got an impression that capsule networks turned out not to be so useful in applications more complicated than the MNIST (at least according to this reddit ...
1
vote
1answer
86 views

Why does this multiplication of $Q$ and $K$ have a variance of $d_k$, in scaled dot product attention?

In scaled dot product attention, we scale our outputs by dividing the dot product by the square root of the dimensionality of the matrix: The reason why is stated that this constrains the ...
1
vote
1answer
31 views

Computation of initial adjoint for NODE

I'm reading the paper Neural Ordinary Differential Equations and I have a simple question about adjoint method. When we train NODE, it uses a blackbox ODESolver to compute gradients through model ...
0
votes
0answers
17 views

Wasserstein GAN with gradient penality - Loss values

I have trained a WAN with gradient penalty and the loss values ​​seem to me much higher than the examples I have seen on the net. The generator receives 2 images as input and must generate a ...
2
votes
1answer
62 views

What is the difference between neural networks and other ways of curve fitting?

For simplicity, let's assume we want to solve a regression problem, where we have one independent variable and one dependent variable, which we want to predict. Let's also assume that there is a ...
0
votes
0answers
19 views

Random Initializations with ReLU gives puzzling results

this may sound naive, but I’m getting a really puzzling result. I was experimenting with MNIST on vanilla MLP (784, 256, 128, 10) with ...
0
votes
0answers
30 views

Why do we have a sigmoid function in the input layer in LSTMs? [closed]

I'm particularly confused about the sigmoid function in the forget and input layer. If we use a sigmoid in the forget layer to look at $h_{t-1}$ and $x_{t}$, and output a number between 0 and 1 for ...
3
votes
1answer
87 views

What sort of Neural Network is best suited to predicting a future purchase?

I have previously implemented a Neural Network with Back-Propagation that was able to learn Tic-tac-toe and could go pretty well at Connect-4. Now I'm trying to do a NN that can make a prediction. ...
0
votes
1answer
47 views

Is it possible to know the distance objects are from camera based on only knowing one object's height?

I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
1
vote
1answer
38 views

Should the range and initial values of weights and biases be adjusted to fit input and output data?

As a routine (in typical everyday tasks) of a data scientist, should they usually decide about weights and biases range and initial values as a function of which data they are planning to insert as ...
2
votes
1answer
32 views

Evaluate model multiple times in loss function? Is this reinforcement learning?

I am interested in models that exhibit behavior. My goal is a model that survives indefinitely on a two dimensional resource landscape. One dimension represents the location (0 to 1) and the second ...
1
vote
0answers
29 views

Linear output layer back propagation

So I'm stack to something that it's probably very easy but I can't get my head around it. I'm building a Neural Network that will consist of many layers with non-linear activation functions (probably ...
0
votes
0answers
21 views

Computer vision - Can you put more weight on a specific part of the object?

Let's say I'm looking for any item that has a certain shape (outline) in a photo. but I can further classify it only according to particular features, that most of them are expected to be shown only ...
0
votes
1answer
36 views

Zero shot learning available labels in testing set

As we all know, zero shot learning involves a model predicting classes that it has not seen. But we are given all the attributes each class might have. Is it fair to assume that we are "aware&...
0
votes
1answer
59 views

Can entire neural networks be composed of only activation functions?

Inverse Reinforcement Learning based on GAIL and GAN-Guided Cost Learning(GAN-GCL), uses a discriminator to classify between expert demos and policy generated samples. Adversarial iRL, build upon GAN-...
1
vote
2answers
762 views

Minimax combined with machine learning to determine if a path should be explored

I have an idea for a new type of AI for two-player games with alternating turns, like chess, checkers, connect four, and so on. A little background: Traditionally engines for such games have used the ...
1
vote
1answer
39 views

What are some suitable positive functions as activations of neural networks?

I am working on a deep Q-learning project. My project is different than normal deep Q-learning. The rewards of my neural network must be positive because I need their values to importance sample ...
3
votes
2answers
64 views

What happens when an opponent a neural network is playing with does not obey the rules of the game (i.e. cheats)?

For example, if AlphaZero plays with an opponent who has a right to move chess figures any way she wants, or make more than 1 move in a turn? Will a neural network adapt to that, as it adapted to an ...
3
votes
2answers
67 views

What is the advantage of using cross entropy loss & softmax?

I am trying to do the standard MNIST dataset image recognition test with a standard feed forward NN, but my network failed pretty badly. Now I have debugged it quite a lot and found & fixed some ...
2
votes
1answer
48 views

What is the Preferred Mathematical Representation for a Forward Pass in a Neural Network?

I know this may be a question of semantics but I always see different articles explain forward pass slightly different. e.g. Sometimes they represent a forward pass to a hidden layer in a standard ...
8
votes
1answer
1k views

What is the time complexity of the forward pass algorithm of a feedforward neural network?

How do I determine the time complexity of the forward pass algorithm of a feedforward neural network? How many multiplications are done to generate the output?
1
vote
1answer
80 views

When are multiple hidden layers necessary?

I know that my question probably seems like being asked many times, but Ill try to be more speciffic: Limitations to my question: I am NOT asking about convolutional neural networks, so please, try ...
2
votes
2answers
502 views

Why isn't my model learning satisfactorily?

The problem to solve is non-linear regression of a non-linear function. My actual problem is to model the function "find the max over many quadratic forms": ...
4
votes
1answer
110 views

Is running more epochs really a direct cause of overfitting?

I've seen some comments in online articles/tutorials or Stack Overflow questions which suggest that increasing number of epochs can result in overfitting. But my intuition tells me that there should ...
3
votes
1answer
40 views

What should the dimension of the input be for text summarization?

I am trying to build a model for extractive text summarization using keras sequential layers. I am having a hard time trying to understand how to input my x data. Should it be an array of documents ...
1
vote
0answers
32 views

When do the ensemble methods beat Neural Networks?

In many applications and domains : Computer Vision, Natural Language Processsing, Image Segmentation, and many other tasks - neural networks of a certain architecture are considered to be by far the ...
2
votes
1answer
65 views

Semantic Segmentation For Multiple Objects When Trained On Single Object

More of a conceptual question here: I'm working on semantic segmentation tasks in the medical space using the U-Net. Let's say that I train a U-Net model on medical images with the goal of segmenting ...
0
votes
1answer
26 views

How to train the NN of simple agents given a reward system?

I'm not an expert in AI or NN, I gathered most of the information I have from the internet, and I'm looking for advice and guidance. I'm trying to design a NN that is going to be used by all the ...
1
vote
1answer
49 views

Chess Neural Network - Most Optimal Input vector/matrix?

I'm wanting to build a NN that can create a policy for each possible state. I want to combine this with MCTS to eliminate randomness so when expansion occurs, I can get the probability of the move to ...

1
2 3 4 5
37