Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

651 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
0
votes
0answers
20 views

Why is MNIST training time linear with (1/batchSize)?

I ran a simple experiment of training MNIST by varying the batch size, so each trial at training used a constant batch size. The left plot is intuitive to me, since I expect training time to decrease ...
0
votes
0answers
48 views

How to detect forgery on scanned document images?

I am trying to detect forgeries done after a document is scanned by a scanner. I already tried to access the metadata, and, if it is edited with any software after scanning, then it is easily detected ...
0
votes
0answers
62 views

Why is my neural network not able to approximate this function?

I'm trying to approximate the following function with a neural network (in Python). ...
0
votes
0answers
29 views

Regularization to enforce feature learning

Is there any research into ways to enforce feature selection across classes by network structure? Given the number of parameters in NN, even convnets are prone to over fitting. I'm curious if there ...
0
votes
0answers
19 views

Methods in training models to minimize distance between statistical summaries of data and samples from model, to get a better approximation function

Introduction: A big problem with deep learning methods involving neural networks is that they tend to do really poorly outside the boundaries of the approximation it has learned from the data it is ...
0
votes
0answers
23 views

What are swarm optimization techniques used for: training the ANN by weight optimization or for feature selection?

Swarm intelligence (SI) is the collective behavior of decentralized, self-organized systems, natural or artificial. The concept is employed in work on artificial intelligence. SI-based algorithms, ...
0
votes
0answers
27 views

Do we scale our target feature in regression problems?

I know this is a basic problem, but still could not find answer, I feel like most books/tutorials avoid talking about scaling the output feature instead they just mention scaling input features. So ...
0
votes
0answers
30 views

What are some papers on face comparison in videos?

I have the following problem. Given two random frames in a video of some people, assume I can detect all faces in both frames. I would like to create a classifier that takes in pairs of faces out of ...
0
votes
0answers
40 views

If artificial neural networks are a special case of computation graphs, so maybe let's optimize computational graphs rather than neural networks?

Existing ANNs are so good for solving complicated tasks on a different domain of data. But creating a neural network is always a hassle and its success mostly relies on an engineer's intuition and ...
0
votes
0answers
101 views

Backward pass of CNN like Resnet: how to manually compute flops during backprop?

I've been trying to figure out how to compute the number of Flops in backward pass of ResNet. For forward pass, it seems straightforward: apply the conv filters to the input for each layer. But how ...
0
votes
0answers
54 views

Reinforcement learning random agent always performing the same few actions

I have a DQN model which has 3 features as inputs (properly normalized) and should output q-values for each of the 100 possible actions. However, prior to any training, I would like to examine the ...
0
votes
0answers
18 views

Why do code implementations average the loss over a batch instead of finding the expected sample of that batch (using sampling probabilities)

Usually, our training objective over a batch is written in terms of the expected value of a sample in that batch such as $objective = E_{x \sim data} * log(P(x))$ But in the code implementations, ...
0
votes
0answers
23 views

How to stabilize the training of a Conv-Siamese Neural Network if the results after different trainings vary relatively strongly?

I am training a neural network using MSE and ADAM optimizer. More precisely, a siamese architecture with a convolutional encoder and euclidean distance on top. I am using MSE because I have different ...
0
votes
0answers
26 views

How can I find the correlation between the input and output of a neural network?

I'm trying to get a value for a correlation between a function input and its output. One brute force way to get this is to sample the entire space and find the standard deviation of the resulting ...
0
votes
0answers
21 views

Larger neural network with almost similar training and validation error as smaller network

I am building a neural network that takes as input 202 units and outputs a 200 dimension continuous variable. While trying to find the best model, one of the parameters i tune is the the number of ...
0
votes
0answers
40 views

Are there any papers on this alternate neural net training approach?

I developed a custom callback for Keras. Initially, it monitors training accuracy. If on a given epoch the accuracy is below that of the previous epoch it lowers the learning rate by a factor. If for ...
0
votes
0answers
30 views

Can neural networks always be assembled like Lego blocks?

BACKGROUND Consider a supervised problem which is based on two scalar features (1) and (2) as well as a third, "time-dependent", feature consisting of a sequence of five values (3)-(7). For ...
0
votes
0answers
57 views

Can neural networks handle redundant inputs?

I have a fully connected neural network with the following number of neurons in each layer [4, 20, 20, 20, ..., 1]. I am using TensorFlow and the 4 real-valued ...
0
votes
0answers
13 views

One model to output the whole vector vs different model to output each vector element

Let's say I'm using a neural net to solve some problem where the output of the net is a vector of some size. What are the advantages (and disadvantages if there are any) of training a single net to ...
0
votes
0answers
33 views

How to convert something to vectors

I wanted to create an encoder, which is the first part of an autoencoder. I do not want to build the whole autoencoder but rather wanted to test whether my mobile device can support running an encoder ...
0
votes
0answers
20 views

Keras word ordering task

I'm trying to solve the word ordering task: given a syntactically unordered sentence, recover the right order of the words. The adopted approach is to transform each sentence in a dependency tree and ...
0
votes
0answers
48 views

Do Multi-resolution CNN exist?

I am currently working on a problem for which the topographic data is in very different resolution. Let say I have data of 20x20 with 1km2 tiles and also high resolution data of 50m2 tiles. I would ...
0
votes
0answers
24 views

Understanding an extract on the motivation behind residual networks

I was reading about ResNets from this page, and I couldn't understand the following extract, about the motivation behind ResNets: "Since neural networks are good function approximators, they ...
0
votes
0answers
34 views

What is the degree of linearity in the error propagated by Gradient Descent?

Neural Network is trained to learn a non-linear function, the more layers it has, the more is the quality of the prediction and the ability to match the real-world function correctly (lets leave aside ...
0
votes
0answers
19 views

Interpreting I/O Transformation Matrix in Convolution

I've been reading this article on convolutional neural networks (I'm a beginner) - and I'm stuck at a point. What I understand: We have a 4x4 input, and want to transform it to a 2x2 grid. I'm ...
0
votes
0answers
36 views

Non-linear regression with a neural network

I have to perform a regression on three curves as shown in the following plot. Here, accA (y-axis) is the dependent variable, and ...
0
votes
0answers
48 views

My CTC loss model's loss stagnates and then outputs only blank characters

I am trying to implement BaiDu's DeepSpeech1 in keras using CTC loss, my code is below: ...
0
votes
0answers
34 views

Semi-supervised: Can I predict the label of purposely unlabelled observations?

Let's say I have a data set with of length N. A small proportion N2 is labeled. Can I remove some labels and then 'reverse' this action with a trained neural network? I could then use the same process ...
0
votes
0answers
12 views

Finding effect of inputs on output (shapely values)

I've developed a neural network which takes in n inputs returning m outputs. I want to see which inputs contribute most with each output. One idea I had is for all inputs/output combinations, lock ...
0
votes
0answers
15 views

What will be the sequence of steps in a human activity recognition model using LSTM?

In the context of these steps detection, tracking, action classification and activity recognition. Which step will be first and further sequence?
0
votes
0answers
29 views

Web stream requests prediction architecture

What's in your opinion the best possible architecture for the following problem ? If you have any code that can be used it would be great . Dataset : 400.000 samples given in hex format in an .xlsx ...
0
votes
0answers
21 views

How to pass observation from CartPole-v0 to neural network using tensorflow

I am trying to learn about RL by implementing DQN with tensorflow. However, I am really stuck with tensorflow.. I just don't understand it. I think I have found the core of what I understand - I dont ...
0
votes
1answer
52 views

Predicting probabilities of events using neural networks

I've got a few thousands of sequences like 1.23, 2.15. 3.19, 4.30, 5.24, 6.22 where the numbers denote times on which an event happened (there's just a single ...
0
votes
1answer
142 views

What is the best neural network architecture for this problem?

I built a three-layer neural network (first is 1D convolutional and the remaining two are linear). It takes an input of 5 angles in radians, and outputs two numbers from 0 to 1, which are respectively ...
0
votes
0answers
95 views

How to calculate covariance matrix of the mini-batch in the k-th layer using Python?

I am a beginner in Python. I want to calculate the covariance matrix of a mini-batch in a given hidden layer.
0
votes
0answers
66 views

What to do when an image classifier does good for a class but bad for another?

So I wrote a convolutional neural network for a binary image classification. I have around 5300 images for each class which I thought would be enough to at least give me a good accuracy on the ...
0
votes
0answers
46 views

Pooling vs Subsampling: Multiple Definitions?

I have seen people using pooling and subsampling synonymously. I have also seen people use them as different processes. I am not sure though if I have correctly inferred what they mean, when they use ...
0
votes
0answers
42 views

What neural network can be used to detect patterns and anomalies of a network map?

What neural network can be used to detect patterns and anomalies of a network map. The network data comes from tools like nmap and masscan.
0
votes
0answers
60 views

Encoding real valued inputs

UPDATE: After reading more about the topic, I've tried implementing the DDPG algorithm instead of using a variation of Q-Learning and still have the same issue. I have the following issue: I want ...
0
votes
0answers
50 views

Confused about NeuralODE

I am a bit confused about NeuralODE and I want to make sure that what I understood so far is correct. Assume we have (for simplicity) 2 data points $z_0$ measured at $t_0$ and $z_1$ measured at $t_1$...
0
votes
0answers
20 views

Is there any known visual neural network, capable of image clasterisation?

The difference in clasterisation task is, that the classes of objects are not given , but must be determined from the dataset. Usual usecase for NN is like having images of cats and dogs, and the ...
0
votes
0answers
93 views

How would you feed a neural network a variable sized array as an input?

From what I've seen, neural networks take a set of atomic inputs. I want an input to be a variable array, i.e. a group of people (with unique IDs). If I didn't care about their ID, I could simply ...
0
votes
0answers
405 views

Neural Network Regression predicting negative values

I am training a model using the DNN Regressor estimator from the Tensorflow API to predict prices based on 1035 features. My dataset contains a little over 500 millions inputs and none of the target ...
0
votes
0answers
39 views

Is there a RNN that can predict the next substitute in a floorball match?

Floorball is a type of floor hockey. During the game, substitutions can be made. The team is also allowed to change players any time in the game; usually, they change the whole team. Individual ...
0
votes
0answers
195 views

Building AI as an Operating System

This is in continuation of my last question here. I am just starting to build an OS which will be based on AI or rather Synthetic Super Intelligence. I also wrote a paper on it for which I am working ...
0
votes
0answers
86 views

Using AI to guess a mathematical pattern of certain polynomials in four variables: practical challenge

I'd like to use machine learning to guess a mathematical pattern: the input are certain polynomials in four variables $q_1,q_2,q_3,q_4$, the output can be zero or one. Allowed polynomials are such ...
0
votes
0answers
38 views

Being able to see how tensorflow “weighs” features in classifier

Say you follow a tutorial on the tensorflow website for a wide and deep model (https://www.tensorflow.org/tutorials/wide_and_deep) I create a model based on the US census data to predict whether or ...
0
votes
1answer
85 views

Back propagation on matrix of weights

I am trying to implement a Neural Network for binary classification using python and numpy only. My network structure is as follows: input features: 2 [1X2] matrix Hidden layer1: 5 neurons [2X5] ...
-1
votes
0answers
22 views

computational complexity of attention in dynamic-convolution

I'm reading this paper , Dynamic convolution-Attention over Convolution kernels. I couldn't understand the complexity of attention i.e., How to calculate O(π(...
-1
votes
1answer
62 views

How to extract the main text from a formated text file?

My idea is to model and train a neural network that receives a text version of a PDF file as the input and gives the content text as output. Take the scenario: One prints a PDF file to a text file (...