Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0
votes
1answer
40 views

How do CNNs handle inputs of different sizes and shapes?

I am new to deep learning so feel free to correct me where I am wrong. Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over ...
-1
votes
1answer
110 views

Is training a neural network all about "balancing of weights and biases"?

I was trying to build an OCR system and heard about ANNs. I am weak at mathematics and statistics and couldn't stick up to reading those massive mathematical documents (research papers or ANN-related ...
1
vote
2answers
42 views

What is the purpose of "alignment" in the self-attention mechanism of transformers?

I've been reading about transformers & have been having some difficulty understanding the concept of alignment. Based on this article Alignment means matching segments of original text with their ...
1
vote
1answer
391 views

Is this LSTM model underfitting?

I think this model is underfitting. Is this correct? ...
2
votes
1answer
66 views

What do echo state networks give us over a generic RNN resevoir?

Slightly generalizing the definition in Jaeger 2001, let's define a reservoir system to be any system of the form $$h_{t}=f(h_{t-1}, x_t)$$ $$y_t=g(Wh_t)$$ where $f$ and $g$ are fixed and $W$ is a ...
0
votes
0answers
32 views

What's the best way to feed stories to a neural network?

I'm trying to train a model that would generate stories. I have a dataset of 2000 stories prepared. They are tokenized and one-hot encoded. I can't load them all at once as a one big dataset, because ...
2
votes
0answers
43 views

How does back propagation adjust the hidden layers' weights and biases?

I'm new to neural networks and trying to figure out its fundamentals but I cannot fully understand the back propagation algorithm. In back propagation, I understand we want to go backwards from the ...
1
vote
1answer
40 views

Can the optimal learning rate differ for different architectures?

In several courses and tutorials about neural networks, people often say that the learning rate (LR) should be the first hyper-parameter to be tuned before we tweak the others. For example, in this ...
0
votes
1answer
54 views

Residual Blocks - why do they work?

I've learnt that idea that the residual block was invented to solve the vanishing gradient problem due to the deep layer to layer multiplication. I understand that for example if I have 10 layers, and ...
1
vote
1answer
93 views

Unable to achieve expected outputs using NEAT for the snake game

I am trying to implement NEAT for the snake game. My game logic is ready, which is working properly and NEAT configured. But even after 100 generations with 200 genomes per generation, the snakes ...
1
vote
0answers
32 views

Is stability an attribute of model or training algorithm used or combination of both?

From this answer, stability is attributed to learning algorithm A stable learning algorithm is one for which the prediction does not change much when the training data is modified slightly. At some ...
0
votes
1answer
50 views

Using a Neural Network (LSTM) to approve/reject word-type sequences

I would like to train an LSTM neural network to either "approve" or "reject" a string based on the word-type sequence. For instance: "Mike's Airplane" would output "...
0
votes
0answers
6 views

What is an intuitive explanation for the weighted sum of inputs plus bias that cause a neuron to be activated when it sees some samples but not others

Im stuck on some of the intuition thats cause specific neurons to fire versus others. Take a feedforward MLP that is able to classify MNIST (and has been optimised). A silly example might be that in ...
0
votes
0answers
6 views

Are there any connectionist parametric models with non-neuron building blocks?

Parametric models allows learning by converging to the desired parameters, which are randomly initialized initially. Among the parametric models, especially in connectionist AI, neural networks are ...
2
votes
1answer
42 views

What is the state of the art in melody generation?

Generative Adversarial Networks can generate realistic photos of people, such as thispersondoesnotexist.com. I wonder whether one can train an artificial intelligence on a batch of plain solo melodies ...
4
votes
1answer
47 views

Is a neural network able to optimize itself for speed?

I am experimenting with OpenAI Gym and reinforcement learning. As far as I understood, the environment is waiting for the agent to make a decision, so it's a sequential operation like this: ...
0
votes
1answer
40 views

Is precision of weights unimportant in neural networks?

While reading about Module in PyTorch, I came across a new data type called half datatype. half() method when calls on a Module ...
4
votes
1answer
326 views

Get the position of an object, out of an image

I have some images with a fixed background and a single object on them which is placed, in each image, at a different position on that background. I want to find a way to extract, in an unsupervised ...
12
votes
1answer
432 views

How can Viv generate new code based on some user's query?

I have been looking into Viv, an artificial intelligent agent in development. Here is a demonstration of Viv (by Dag Kittlaus). Based on what I understand, this AI can generate new code and execute it ...
3
votes
1answer
41 views

How can I interpret the way the neural network is producing an output for a given input?

I'm using a small neural network (2 hidden layers, 60 neurons apiece) for a rather complex binary classification problem. The network works well, but I'd like to know how it is using the inputs to ...
4
votes
1answer
354 views

What are the differences between Bytenet and Wavenet?

I recently read Bytenet and Wavenet and I was curious why the first model is not as popular as the second. From my understanding, Bytenet can be seen as a seq2seq model where the encoder and the ...
3
votes
2answers
2k views

How to perform gradient checking in a neural network with batch normalization?

I have implemented a neural network (NN) using python and numpy only for learning purposes. I have already coded learning rate, momentum, and L1/L2 regularization and checked the implementation with ...
2
votes
1answer
97 views

Is there any way to train a neural network without using gradients?

The only algorithm I know for updation of weights of a neural network is based on gradients. The update equation can be roughly written as $$w \leftarrow w - \nabla_{w}L$$ where $\nabla_{w}L$ is the ...
0
votes
0answers
13 views

Time series forecasting with some challenges

I'm attempting to devise a strategy to make time series forecasts based on costs accumulated over time. My dataset contains about 7500 time-series sequences (call it an instance for now), each having ...
2
votes
1answer
165 views

Can neural networks have continuous inputs and outputs, or do they have to be discrete?

In general, can ANNs have continuous inputs and outputs, or do they have to be discrete? So, basically, I would like to have a mapping of continuous inputs to continuous outputs. Is this possible? ...
1
vote
0answers
33 views

Is input standardisation better than input normalisation?

Consider a network which takes samples of single values. And consider the training set of 5 samples: $$ inp = [5, 6, 7, 8, 9] $$ Input normalisation: $$ min = 5, max = 9, span = 9-5 = 4 \\ Input1 = [(...
3
votes
2answers
246 views

Should you reload the optimizer for transfer learning?

For example, you train on dataset 1 with an adaptive optimizer like Adam. Should you reload the learning schedule, etc., from the end of training on dataset 1 when attempting transfer to dataset 2? ...
1
vote
1answer
1k views

How do you explain Hebbian Learning in an intuitive way?

In these lecture slides, it's written The neuropsychologist Donald Hebb postulated in 1949 how biological neurons learn: "When an axon of cell A is near enough to excite a cell B and repeatedly ...
1
vote
1answer
38 views

In the NEAT algorithm, what is the purpose of treating disjoint and excess genes differently?

In the NEAT algorithm, what is the purpose of treating disjoint and excess genes differently? They are treated so (or may be treated potentially) at least when calculating the distance between 2 ...
0
votes
1answer
40 views

What does all the formula and pictures mean?

https://www.nature.com/articles/s41467-020-17419-7 I am a medical school graduate and I really want to learn AI/ML for computer-aided diagnosis. I was building a symptom checker and I found the ...
1
vote
1answer
62 views

How does a single neuron in hidden layer affect training accuracy

I'm currently a student learning about AI Networks. I've came across a statement in one of my Professor's books that a FFBP (Feed-Forward Back-Propagation) Neural Network with a single hidden layer ...
2
votes
1answer
45 views

What does "differentiable architecture" mean?

I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term "Differentiable Architecture", ...
0
votes
2answers
175 views

Why isn't the loss of my neural network reduced after 2500 iterations?

I have developed a basic feedforward neural network from scratch to classify whether image is of cat or not cat. It works fine, but after 2500 iterations, my cost function is not reducing properly. ...
1
vote
1answer
410 views

Could the normalisation of the inputs make the neural network insensitive to changes in the inputs?

When using neural networks (NNs), we often normalized the inputs. I think this is done to equally capture the changes in any input feature, that is, if any feature takes huge values and other features ...
3
votes
2answers
136 views

When should I create a custom loss function?

I'm using a neural network to solve a multi regression problem because I'm trying to predict continuous values. To be more specific, I'm making a tracking algorithm to track the position of an object, ...
1
vote
0answers
33 views

Can NeuralHash be used as a loss for an Autoencoder?

I've recently read about NeuralHash, and immediately thought that it might be used as a loss for an autoencoder. However, it only seems to preserve "structure" from what I've read, not ...
0
votes
2answers
59 views

How to design a neural network with arbitrary input and output length?

I am trying to build a neural network that has an input of $n$ pairs of integer values (where $n$ is random) and a corresponding output of a binary array with length $n$. The input will be a set of ...
2
votes
2answers
63 views

Does it make sense for a logistic regression model to perform better than a neural network on the Iris data set?

Per a review post, a simple Logistic Regression model on the Iris data set gets about 97% test accuracy on iris dataset whereas a neural network gets just 94%. The neural network model used in Keras ...
2
votes
2answers
48 views

Maximize loss on non-target variable

I have a neural network that should be able to classify documents to target label A. The problem is that the network is actually classifying label B, which is an easier task. To make the problem more ...
0
votes
0answers
13 views

Graph neural network - what level (node or link or graph) prediction should be used for my problem?

I posted this on cross-validated but did not get a response. Trying my luck here. Sorry if this is not recommended. I have an undirected graph with nodes separated within a specified distance, say d, ...
0
votes
0answers
13 views

Training a sequential model that can only evaluate after several hundred cycles

I'm attempting to build a neural network to play the card game, Lost Cities. A brief overview of the game: The game involves two players taking turns to play cards on expeditions. Expeditions incur a ...
1
vote
0answers
22 views

How to pass variable length data as feature to a neural network?

I am working on building a model to classify the type of touch the user makes(Long Press, Left Swipe, Right swipe and so on). I have data with features that characterise the user's touch, like ...
1
vote
1answer
321 views

In a single neuron output layer should the output be a scalar?

Given a neural network with 3 inputs, 4 hidden layers, and 1 output, should the output neuron be a vector or a scalar? I thought that at the end of the summation only one number between 0 and 1 would ...
1
vote
0answers
49 views

What causes high differences in neural network accuracy each run?

I trained a CNN using Keras in R to multi-dimensional image data for image classification of five classes. I realized that each run (I retrained the network on the same data for ten times), although I ...
1
vote
1answer
224 views

Parallelised my training or the dataset

I have some plans in working with Reinforcement Learning in order to predict the stock price movement. For a stock like TSLA, some training features might be the pivot price values and the set of the ...
2
votes
0answers
33 views

How do neural networks deal with inputs of different sizes that are padded in order to have them of the same size?

I am trying to create an environment for RL where the size of my input (observation space) is not fixed. As a way around it, I thought about padding the size to a maximum value and then assigning &...
1
vote
1answer
148 views

Dropout causes too much noise for network to train

I am using dropout of different values to train my network. The problem is, dropout is contributing almost nothing to training, either causing so much noise the error never changes, or seemingly ...
0
votes
0answers
12 views

Train separate AutoEncoder's on each class or one AE for all classes to learn features?

I'm working on a project where the dataset contains time series of three classes, depending on the shape of the series. I want to learn the representations of these series as vectors, so naturally I ...
2
votes
1answer
97 views

Validation Loss Fluctuates then Decrease alongside Validation Accuracy Increases

I was working on CNN. I modified the training procedure on runtime. As we can see from the validation loss and validation accuracy, the yellow curve does not fluctuate much. The green curve and red ...
3
votes
1answer
51 views

How can I create an embedding layer to convert words to a vector space from scratch?

For an upcoming project, I am trying to build a neural network for classifying text from scratch, without the use of libraries. This requires an embedding layer, or a way to convert words to some ...

1
2
3 4 5
44