Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
1
vote
0answers
11 views

What are some good loss functions used to minimize extreme errors in regression and time series forecasting?

I'm working on a time series forecasting task, and, in some specific cases, I don't need perfect accuracy, but the network cannot by any means miss by a lot. So, in detriment of a smaller mean error, ...
2
votes
0answers
23 views

Is better to spend parameters on weights or bias?

If a neural network has a limited number of neuron parameters to find, -let's say only 1000 parameters-, it is generally better to spend the parameters on weights or neuron bias? For example, if each ...
0
votes
0answers
46 views

Why is my neural network not able to approximate this function?

I'm trying to approximate the following function with a neural network (in Python). ...
1
vote
1answer
22 views

What are the real applications of hierarchical temporal memory?

What are the real applications of hierarchical temporal memory (HTM) in machine learning (ML) these days?
1
vote
1answer
22 views

What is the main contribution of the paper Disentangling by Factorising?

Considering the paper Disentangling by Factorising, in addition to introducing a new model for Disentangled Representation Learning, FactorVAE (see figure), what is the main theoretical contribution ...
3
votes
2answers
133 views

What is the effect of using pooling layers in CNNs?

I know how pooling works, and what effect it has on the input dimensions - but I'm not sure why it's done in the first place. It'd be great if someone could provide some intuition behind it - while ...
1
vote
1answer
37 views

What is the intuition behind the Xavier initialization for deep neural networks?

The aim of weight initialization is to prevent layer activation outputs from exploding or vanishing during the course of a forward pass through a deep neural network I am really having trouble ...
0
votes
0answers
23 views

What is an auto-associator?

What is an auto-associator, and how does it work? How can we design an auto-associator for a given pattern? I couldn't find a clear explanation for this anywhere on the internet. Here's an example of ...
2
votes
1answer
36 views

Is there a way of deriving a loss function given the neural network and training data?

There is some sort of art to using the right loss function. However, I was wondering if there is a way to derive the loss function if I gave you a neural network model (the weights) as well as the ...
0
votes
0answers
8 views

Flatten in simple feedforward networks [migrated]

I am working on the CIFAR10 dataset and came across this example in Keras, using data augmentation: https://keras.io/examples/cifar10_cnn/ The example uses CNN. I want to implement just a simple ...
1
vote
1answer
20 views

How could a NN be trained to output a cyclic (e.g. hue) number?

I was thinking about training a neural network to colourize images. The input would be the luminosity/value for each pixel, and the output would be a hue and/or saturation. Training data would be ...
1
vote
0answers
25 views

Why do we use a delay when feeding our input data to the echo state network?

I'm new to working with neural networks and have recently began implementing neural networks for time series forecasting in some of my work. I've been particularly using Echo State Networks and have ...
0
votes
0answers
8 views

Will GPU optimized model run on TPU? [migrated]

There is a project which contains models in DLC format (Snapdragon Neural Processing Engine - SNPE) which I guess are optimized for the Qualcomm Snapdragon 820 chipset (see second link). The project ...
0
votes
0answers
14 views

Low accuracy during training for text summarization

I am trying to implement an extractive text summarization model. I am using keras and tensorflow. I have used bert sentence embeddings and the embeddings are fed into an LSTM layer and then to a Dense ...
0
votes
0answers
12 views

Is the number of layers in the simple RNN fixed or is it random? [closed]

When I search for RNN, I find LSTM most of the times. Before I go on o for reading more about LSTM, I want to explore the vanilla RNN. I want to know an explanation of working with mathematical ...
0
votes
0answers
19 views

Regularization to enforce feature learning

Is there any research into ways to enforce feature selection across classes by network structure? Given the number of parameters in NN, even convnets are prone to over fitting. I'm curious if there ...
1
vote
0answers
11 views

How are the conditional probabilities for the visible and hidden units computed, in a RBM with multinomial units?

When using multinomial units for visible units of an RBM as described in section 13.1 of this training guide, how are the conditional probabilities for the visible and hidden units computed? Is it the ...
1
vote
1answer
82 views

Why not replacing the simple linear functions that neurons compute with more complex functions?

In a neural network, a neuron typically computes a linear function $f(x) = w*x$, where $w$ is the weight and $x$ is the input. Why not replacing the linear function with more complex functions, such ...
3
votes
0answers
29 views

How does backpropagation work in LSTMs?

After reading a lot of articles (for instance, this one Understanding LSTM Networks), I know that the long short-term memory (LSTM) is an artificial recurrent neural network (RNN) architecture used in ...
1
vote
2answers
43 views

How to train a neural network with a data set that in which the target is a mix of 0-1 label and numeric real value label?

I am running into an issue in which the the target (label collums) of my dataset contain a mixture of binary label (yes/no) and some numeric value label. The value of these numeric value (resource 1 ...
1
vote
0answers
31 views

Policy Gradient on Tic-Tac-Toe not working

I wanted to implement the Policy Gradient on Tic-Tac-Toe. I tried to use the code that worked for any environment like CartPole-v0 to my Tic-Tac-To game. But it is not learning. There are no errors. ...
2
votes
1answer
92 views

Why do we need convolutional neural networks instead of feed-forward neural networks?

Why do we need convolutional neural networks instead of feed-forward neural networks? What is the significance of a CNN? Even a feed-forward neural network will able to solve the image classification ...
3
votes
1answer
56 views

What is the intuition behind the attention mechanism?

Attention idea is one of the most influential ideas in deep learning. The main idea behind attention technique is that it allows the decoder to "look back” at the complete input and extracts ...
0
votes
0answers
11 views

Is the number of bidirectional LSTMs in seq2seq model equal to the maximum length of input text/characters?

I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/. It ...
1
vote
0answers
18 views

How can I find the similar non-zero connections between different levels of sparsity of the same network?

I am pruning a neural network (CNN and Dense) and for different sparsity levels, I have different sub-networks. Say for sparsity levels of 20%, 40%, 60% and 80%, I have 4 different sub-networks. Now, ...
2
votes
1answer
28 views

What are the main differences between sparse autoencoders and convolution autoencoders?

What are the main differences and similarities between sparse autoencoders and convolution autoencoders? When should one be preferred over the other? What are their applications? (References are ...
1
vote
1answer
22 views

What are the most common feedforward neural networks?

What are the most common feedforward neural networks? What kind of inputs do they receive? For example, do they receive binary numbers, real numbers, vectors, or matrics? Is there such a taxonomy?
0
votes
1answer
23 views

How to add a pretrained model to my layers to get embeddings?

I want to use a pretrained model found in [BERT Embeddings] https://github.com/UKPLab/sentence-transformers and I want to add a layer to get the sentence embeddings from the model and pass on to the ...
1
vote
0answers
22 views

If the output of a model is a ridge function, what should the activation functions at all the nodes be?

I have the following assignment. I can't understand the b part of this question in my assignment. I have completed the 1st part and understand the maths behind it, but the 2nd part has me stumped. I ...
1
vote
0answers
28 views

Does the reduction of the dimensions over multiple layers allow more details to be stored within the final representation?

From : https://debuggercafe.com/implementing-deep-autoencoder-in-pytorch/ the following autoencoder is defined ...
1
vote
0answers
23 views

How estimate the minimum size of an autoencoder to overfit the training data?

Given e.g. $1$M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$, how to estimate the minimum network size required ...
1
vote
1answer
31 views

Is it possible to have the latent vector of an auto-encoder with size 1?

Given e.g. 1M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$: Is it possible to have the bottleneck of the AE network ...
1
vote
1answer
26 views

What are mono-variable and multi-variable neural networks?

In this document, the terms "Redes Neuronales estáticas monovariables" and "Redes Neuronales estáticas multivariables" are mentioned. What are mono-variable and multi-variable neural networks? Is it ...
1
vote
0answers
19 views

How do we choose the filters for the convolutional layer of a convolution neural network?

Since the hidden layers of a CNN work as a trainable feature extractor, more detailed content based on a larger number of pixels shall require bigger filter sizes. But for cases where localized ...
2
votes
0answers
17 views

Can you use transformer models to do autocomplete tasks?

I've researched online and seen many papers on the use of RNNs (like LSTMs or GRUs) to autocomplete for, say, a search engine, character by character. Which makes sense since it inherently predicts ...
4
votes
1answer
45 views

How does the repetition of features across states at different time steps affect learning?

Let's say you are training a neural network in an RL setting, where the state (i.e. features/input data) can be the same for multiple successive steps (~typically around 8 steps) of an episode. For ...
0
votes
0answers
11 views

Methods in training models to minimize distance between statistical summaries of data and samples from model, to get a better approximation function

Introduction: A big problem with deep learning methods involving neural networks is that they tend to do really poorly outside the boundaries of the approximation it has learned from the data it is ...
1
vote
1answer
39 views

Why does this multiplication of $Q$ and $K$ have a variance of $d_k$, in scaled dot product attention?

In scaled dot product attention, we scale our outputs by dividing the dot product by the square root of the dimensionality of the matrix: The reason why is stated that this constrains the ...
1
vote
0answers
26 views

Confidence Interval around prediction with bootstrapping

I want to generate a confidence interval around my prediction (vector) $\hat{y}$. I have implemented the following procedure. However, I am not sure whether this makes sense in a statistical way: I ...
1
vote
0answers
32 views

What is a Hidden Markov Model - Artificial Neural Network (HMM-ANN)?

As far as I know, neural networks have hidden computational units and HMM has hidden states. Hidden Markov Models can be used to generate a language, that is, list elements from a family of strings. ...
2
votes
1answer
45 views

How to evaluate a Deep Q-Network

Good day, it's a pleasure having joined this Stack. In my master thesis I have to expand a Deep Reinforcement Learning Network, to be precise a Deep Q-Network, which is used to control machines in an ...
0
votes
1answer
72 views

Is my understanding of how AI works correct?

In my discussion over my question on Math SE, I explained to a user, how I think AI works, I wrote that with the sigmoid(logistic) function, features of a data set are identified, many such iterations ...
0
votes
0answers
11 views

Fitting models to 2d data [migrated]

I am reading the documentation on tf.lattice (https://www.tensorflow.org/lattice/overview) I am wondering how the training data was created/trained. Is it represented by (x,y) intpositions and some ...
2
votes
0answers
10 views

Why is ANFIS important in general?

I am actually working with the iris dataset from sklearn and try to understand the ANFIS-Package for python. But that does not really matter! I have a more general question. During thinking about ...
2
votes
0answers
9 views

What is the difference between fuzzy neural networks and adaptive neuro fuzzy inference systems?

I have, like you see, just a general question about the combination of fuzziness and neural networks. I understood it as follows Fuzzy neural networks as a hybrid system: the neural network helps me ...
1
vote
1answer
32 views

Are my steps correct for a proper classification of a sick brain?

I have a dataset with MRI of patients with a specific disease that affects the brain and another dataset with MRI of healthy patients. I want to create a classifier (using neural networks) to ...
0
votes
0answers
11 views

What are swarm optimization techniques used for: training the ANN by weight optimization or for feature selection?

Swarm intelligence (SI) is the collective behavior of decentralized, self-organized systems, natural or artificial. The concept is employed in work on artificial intelligence. SI-based algorithms, ...
3
votes
1answer
23 views

What should the dimension of the input be for text summarization?

I am trying to build a model for extractive text summarization using keras sequential layers. I am having a hard time trying to understand how to input my x data. Should it be an array of documents ...
2
votes
0answers
34 views

How to calculate the data noise variance for a prediction interval?

I have a neural network that connects $N$ input variables to $M$ output variables (qoi). By default, neural networks just give out point estimations. Now, I want to plot some of the quantity of ...
1
vote
0answers
21 views

What pre-processing of the image is needed before feeding it into the convolutional neural network?

I can't figure out what preprocessing of the image is needed before feeding it into the convolutional neural network. For example, I want to recognize circles on a 1000 by 1000 px photo. The learning ...

1
2 3 4 5
33