Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

CNN positional based feature extraction

Not sure where to ask this as it's a fairly advanced question but: A major problem with deep learning according to hinton is that operations like max-pooling remove the position information of ...
1
vote
0answers
8 views

How to build sort objects model using Deep Learning

How to build sort objects model using Deep Learning if i have some objects in an image enter image description here if i give him an similar image he sort the detected order according to Learning ...
2
votes
2answers
20 views

Why does the bias need to be a vector in a neural network?

I am learning to use tensorflow.js. I am also using the tfvis library to print information about the neural net to the web browser. When I create a create a dense neural net with a layer with 5 ...
1
vote
0answers
28 views

How is clustering used in the unsupervised training of a neural network?

How is clustering used in the unsupervised training of a neural network? Can you provide an example?
2
votes
0answers
10 views

Can Bert be used to extract embedding for large categorical features?

I've lot of training data points (i.e in millions) and I've around few features but the issue with that is all the features are categorical data with 1 million+ categories in each. So, I couldn't use ...
3
votes
1answer
64 views

Can we get the inverse of the function that a neural network represents?

I was wondering if it's possible to get the inverse of a neural network. If we view a NN as a function, can we obtain its inverse? I tried to build a simple MNIST architecture, with the input of (784,...
1
vote
1answer
42 views

How is back-propagation useful in neural networks?

I am reading about backpropagation and I wonder why I have to backpropagate. For example, I would update the network by randomly choosing a weight to change, $w$. I would have $X$ and $y$. Then, I ...
2
votes
1answer
62 views

Is it possible to train a neural network with 3 inputs and 12 outputs?

The selection of experimental data includes a set of vectors of different dimensions. The input is a 3-dimensional vector, and the output is a 12-dimensional vector. The sample size is 120 pairs of ...
1
vote
0answers
8 views

Regional specialization in neural networks (especially for language processing)?

What is the status of the research on regional specialization of the artificial neural networks? Biology knows that such specialization exists in the brain and it is very important for the functioning ...
2
votes
1answer
22 views

Is greedy layer-wise pretraining obsolete?

I was looking into the use of a greedy layer-wise pretraining to initialize the weights of my network. Just for the sake of clarity: I'm referring to the use of gradually deeper and deeper ...
1
vote
0answers
22 views

What activation functions are better for what problems?

I’ve been reading about neural network architectures. In certain cases, people say that the sigmoid "more accurately reflects real-life" and, in other cases, functions like hard limits reflect "the ...
2
votes
0answers
31 views

How can I compare EEG data with accelerometer data in 1 algorithm?

I have frequency EEG data from fall and non-fall events and I am trying to incorporate it with accelerometer data that was collected at the same time. One approach is, of course, to use two separate ...
1
vote
0answers
18 views

Is there any paper that uses truncated neural networks?

Recently, I've found good success in truncated neural networks ie functions of the form $$ g=f1_{[-M,M]^d}, $$ where $f:\mathbb{R}^d\to\mathbb{R}^n$ is a feed-forward neural network and $1_{[-M,M]^d}$ ...
1
vote
1answer
31 views

Why does PyTorch use a different formula for the cross-entropy?

In my understanding, the formula to calculate the cross-entropy is $$ H(p,q) = - \sum p_i \log(q_i) $$ But in PyTorch nn.CrossEntropyLoss is calculated using this ...
3
votes
2answers
101 views

What's the function that SGD takes to calculate the gradient?

I'm struggling to fully understand the stochastic gradient descent algorithm. I know that gradient descent allows you to find the local minimum of a function. What I don't know is what exactly that ...
3
votes
0answers
28 views

What are some ways to quickly evaluate the potential of a given NN architecture?

Main question Is there some way we can leverage general knowledge of how certain hyperparameters affect performance, to very rapidly get some sort of estimate for how good a given architecture could ...
1
vote
0answers
19 views

Generating 5 numbers with 1 input before loss function

I am trying make an ANN model that takes a constant m (will be changed later but now it is just a constant, let's say 0) as an input and generate 5 non-integer numbers (a1,a2..a5) after some layers ...
2
votes
1answer
18 views

Semantic Segmentation For Multiple Objects When Trained On Single Object

More of a conceptual question here: I'm working on semantic segmentation tasks in the medical space using the U-Net. Let's say that I train a U-Net model on medical images with the goal of segmenting ...
0
votes
0answers
29 views

How to Layer based Feature extraction?

I have read that in deep networks you can engineer each layer for a particular purpose with regards to feature learning. I'm wondering how that is actually done and how it is trained? In addition ...
1
vote
2answers
28 views

How to deal with random weights initialization in hyperparameters tuning?

In the step of tuning my neural networks I often encounter a problem that every time I train the exact same network, it gives me different final error due to random initialization of the weights. ...
1
vote
0answers
16 views

What's the difference between semi-supervised VAEs and conditional VAEs?

Can someone explain the difference? I'm assuming the difference is just that the neural nets representing the encoder and decoder are trained in a semi-supervised manner in semi-supervised VAE, which ...
5
votes
1answer
201 views

What is the mathematical definition of an activation function?

What is the mathematical definition of an activation function to be used in a neural network? So far I did not find a precise one, summarizing which criterions (e.g. monotonicity, differentiability, ...
1
vote
0answers
21 views
1
vote
0answers
33 views

Brain.js Artificial Neural Network predicting missing variables value [closed]

I am using this code for the training and then how would I alter the process so when I run the test data inside the neural network so that it shall give the value of the missing variable (z)? ...
5
votes
0answers
41 views

Learning from NNs

When a neural network learns something from a data set, we are left with a bunch of weights which represent some approximation of knowledge about the world. Although different data sets or even ...
4
votes
0answers
55 views

Is there a mathematical formula that describes the learning curve in neural networks?

In training a neural network, you often see the curve showing how fast the neural network is getting better. It usually grows very fast then slows down to almost horizontal. Is there a mathematical ...
0
votes
1answer
24 views

Extract product information from email receipt HTML

I am trying to extract product information from email receipts HTML. Most services I have found focus on OCR from paper receipts or PDFs. I would imagine that extraction of product information would ...
2
votes
0answers
32 views

How can I extract the reason of the legal compensation from a court report?

I'm working on a project (court-related). At a certain point, I have to extract the reason of the legal compensation. For instance, let's take these sentences (from a court report) Order mister X ...
4
votes
2answers
56 views

Does summing up word vectors destroy their meaning?

For example, I have a paragraph which I want to classify in a binary manner. But because the inputs have to have a fixed length, I need to ensure that every paragraph is represented by a uniform ...
1
vote
1answer
97 views

My idea for general AI [closed]

I haven't built it, I haven't tested it. It's just a concept which can hopefully inspire someone. It aims at replicating how the human brain works. It has plasticity and a hierarchical structure. The ...
1
vote
1answer
44 views

Why does my model overfit on pseudo-random numbers training data?

I am trying to predict pseudo-random numbers using the past numbers with a multiplayer perceptron. The error while training is very low. However, as soon as I test it with a test set, the model ...
2
votes
1answer
25 views

How can a system recognize if two strings have the same or similar meaning?

How can a system recognize if two strings have the same or similar meaning? For example, consider the following two strings Wikipedia provides good information. Wikipedia is a good source of ...
2
votes
0answers
42 views

Is there a recurrent neural network where the output becomes a partial input?

I am aware of the way RNN works (finite and infinite impulse) and I have seen a lot of use (e.g. in speech recognition). I have understood it is used to "store" value and/or re-use them. But I am ...
3
votes
0answers
24 views

Rarely predict minority class imbalanced datasets

I have a dataset in which class A has 99.8%, class B 0.1% and class C 0.1%. If I train my model on this dataset, it predicts always class A. If I do oversampling, it predicts the classes evenly. I ...
3
votes
1answer
47 views

Can dropout layers not influence LSTM training?

I am working on a project that requires time-series prediction (regression) and I use LSTM network with first 1D conv layer in Keras/TF-gpu as follows: ...
2
votes
0answers
23 views

Determine Frequency from Noisy Signal With Neural Networks (With Adeline Model)

I'm trying to determine the frequency from a signal with NN. I'm using the Adeline model for my project and I'm taking a few samples in each 0.1-volt step in a true signal and a noisy one. First ...
2
votes
1answer
65 views

Why is a softmax used rather than dividing each activation by the sum?

Just wondering why a softmax is typically used in practice on outputs of most neural nets rather than just summing the activations and dividing each activation by the sum. I know it's roughly the same ...
8
votes
3answers
158 views

Why can neural networks generalize at all?

Neural networks are incredibly good at learning functions. We know by the universal approximation theorem that, theoretically, they can take the form of almost any function - and in practice, they ...
2
votes
1answer
99 views

Do all neurons in a layer have the same activation function?

I'm new to machine learning (so excuse my nomenclature), and not being a python developer, I decided to jump in at the deep (no pun intended) end writing my own framework in C++. In my current design ...
2
votes
0answers
32 views

How to understand my CNN's training results?

I created a multi-label classification CNN to classify chest X-ray images into zero or more possible lung diseases. I've been doing some configuration tests on it and analyzing its results and I'm ...
0
votes
0answers
19 views

Is it possible to combine multiple SVMs that were trained on sublayers of a CNN into one combined SVM?

I have created a CNN for use on the MNIST dataset for now (so I have 10 classes). I have trained SVMs on the sublayers of this trained CNN and wish to combine them into a combined SVM as to give a ...
3
votes
0answers
56 views

Is running more epochs really a direct cause of overfitting?

I've seen some comments in online articles/tutorials or Stack Overflow questions which suggest that increasing number of epochs can result in overfitting. But my intuition tells me that there should ...
1
vote
0answers
18 views

Hinton's Capsule network 16 dimensions

As you may know, Hinton's Capsule Network has been around for about 2 years now. https://arxiv.org/abs/1710.09829 Much ado has been made about how the Capsules output a vector (magnitude = ...
3
votes
1answer
46 views

What is a “batch” in batch normalization?

I'm working on an example of CNN with the MNIST hand-written numbers dataset. Currently I've got convolution -> pool -> dense -> dense, and for the optimiser I'm using Mini-Batch Gradient Descent with ...
2
votes
1answer
38 views

How to make DNN learn multiplication/division?

A single neuron with 2 weights and identity activation can learn addition/subtraction as the 2 weights will converge to 1 and 1 (addition), or 1 and -1 (subtraction). However, for multiplication and ...
3
votes
0answers
37 views

How does the memory mechanism (reading and writing) work in a neural Turing machine?

In neural Turing machine (NTM), reading memory is represented as \begin{align} r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2} \end{align} and writing to memory is represented as ...
3
votes
1answer
36 views

How to classify human actions?

I'm quite new to machine learning (I followed the Coursera course of Andrew Ng and now starting deeplearning.ai courses). I want to classify human actions real-time like: Left-arm bended Arm above ...
4
votes
1answer
72 views

What are some new deep learning models for learning latent representation of data?

I know that autoencoders are one type of deep neural networks that can learn the latent representation of data. I guess there should be several other models like autoencoders. What are some new deep ...
2
votes
1answer
34 views

How does a batch normalization layer work?

I understood that we normalize to input features in order to bring them on the same scale so that weights won't be learned in arbitrary fashion and training would be faster. Then I studied about ...
3
votes
1answer
35 views

What is a working configuration of a neuronal network (number of layers, lerning rate and so on) for a specific dataset?

I try to solve some easy functions with a neuronal network (aforge-lib): This is how I generate the dataset: ...