Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
1
vote
0answers
26 views

Initial LSTM hidden state and cell

If we use LSTMCell from torch: The initial hidden and cell layers should be CONSTANT (from the first time you run the program) and saved right? Like random seeds? ...
3
votes
1answer
61 views

Is normalizing the data a way to improve generalization?

There are many known ways to overcome overfitting or make a model generalize better to unseen data. Here I would like to ask if normalizing/standardizing/similiraizing the train and test data is a ...
4
votes
0answers
35 views

Is it possible to control asymptotic behaviour of neural network models?

Is it possible to specify what the asymptotic behaviour of a Neural Networks (NN) model should be? I am thinking on NN which try to learn a mapping $\vec y=f(\vec x)$ with $\vec x$ a vector of ...
1
vote
0answers
41 views

CNN - Visualizing images near decision boundary - Pixels inexplicably tend to edges

We are exploring the images classified by a CNN at its decision boundary, using Genetic Algorithms to generate them. We have created a fine-tuned binary grayscale image classifier for cats. As the ...
0
votes
0answers
19 views

Suggestions for Deep Learning for regression on huge 3D volumes

I have a dataset of 3D images (volumes) with dimensions 400x250x400. For each input image I have an output of the same dimensions. I would like to train a machine learning (or deep learning) model on ...
1
vote
0answers
67 views

How do I create a chatbot using tensorflow or pytorch using like the one defined in dialogflow?

How do I create a chatbot using TensorFlow or PyTorch using like the one defined in DialogFlow? What are the best datasets that I can use so to create my own personal assistant like google assistant? ...
1
vote
1answer
151 views

Online Learning for Neural Networks

There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform online learning with neural ...
0
votes
1answer
81 views

How to develop neural networks for face recognition?

I have developed face recognition algorithms by using pre-built libraries in Python and open CV. However, suppose if i want to make my own neural network algorithm for face recognition, what are the ...
2
votes
0answers
22 views

What is the difference between Squeeze-and-excite and bottleneck modules from Mobilenet v2?

Squezee-and-excite networks introduced SE blocks, while MobileNet v2 introduced linear bottlenecks. What is the effective difference between these two concepts? Is it only implementation (depth-wise ...
2
votes
1answer
56 views

Aesthetics analysis with deep learning

I'm trying to score video scenes in terms of aesthetics and cinematography features. Basically, how "interesting" a scene or video frame can be for a viewer. Simpler, how attractive a scene is. My ...
6
votes
3answers
1k views

Why is the derivative of the activation functions in neural networks important?

I'm new to NN. I am trying to understand some of its foundations. One question that I have is: why the derivative of an activation function is important (not the function itself), and why it's the ...
1
vote
0answers
15 views

How do I recover the 3D structure of a layer after a fully-connected layer?

I want to implement a CNN, but I want to explore what happens when my first layer is a fully-connected one. I still want to use convolutions, of course, but I want to apply them after the first layer. ...
0
votes
1answer
37 views

Library for rendering neural network NEAT

I just finished my implementation of NEAT and I want to see the phenotype of each genome. Is there a library for displaying a neural network like this? Example of my genome syntax: ...
2
votes
3answers
80 views

One vs multiple output neurons

Consider an MLP that outputs an integer 'rating' of 0 to 4. Would it be correct to say this could be modeled in either of the following ways: map each rating in the dataset to a 'normalized set' ...
4
votes
2answers
319 views

Iteratively and adaptively increasing the network size during training

For an experiment that I'm working on, I want to train a deep network in a special way. I want to initialize and train a small network first, then, in a specific way, I want to increase network depth ...
0
votes
1answer
37 views

Preparation of input data

Tell me why my val_acc is always the same and how to solve this problem? I saw several topics on the Internet specifically on this problem but they did not help me (for example, use SGD with different ...
1
vote
1answer
34 views

Alpha Zero queen promotion

"The final 9 planes encode possible underpromotions for pawn moves or captures in two possible diagonals, to knight, bishop or rook respectively. Other pawn moves or captures from the ...
4
votes
1answer
65 views

Is tabular Q-learning considered interpretable?

I am working on a research project in a domain where other related works have always resorted to deep Q-learning. The motivation of my research stems from the fact that the domain has an inherent ...
0
votes
0answers
12 views

Camera pose to environment Mapping

I would like to teach a model the environment of a room. I'm doing so by mapping a camera pose (x, y, z, q0, q1, q2, q3) to its corresponding image; where x, y, z represent location in Cartesian ...
0
votes
2answers
42 views

How should I interpret the weights file of the Leela Zero neural network?

I am trying to understand the NN architecture given at https://github.com/leela-zero/leela-zero/blob/next/training/caffe/zero.prototxt. So, I downloaded the NN weights from http://zero.sjeng.org/. ...
1
vote
0answers
76 views

Polynomial Regressor vs Neural Network Regressor

So as far as my knowledge (might be a bit vague and not mathematical) goes a Neural Network can and should only be able to approximate a bounded function, which is not the case of a Polynomial ...
0
votes
1answer
56 views

What is the correct way to read and analyse images in machine learning?

I am trying to understand the best practice to read and analyze images. If your image has 10,000 pixels, your input layers will have 10,000 inputs? It sounds that my neural network will have too many ...
5
votes
3answers
96 views

Preventing bias by not providing irrelevant data

This seems like such a simple idea, but I've never heard anyone that has addressed it, and a quick Google revealed nothing, so here it goes. The way I learned about machine learning is that it ...
1
vote
2answers
78 views

Can neural networks be used to find features importance?

I am wondering if I can use neural networks to find features importances in similar manner as it can be done for random forests or decision trees and if so, how to do it? I would like to use it on ...
0
votes
0answers
13 views

Can Artificial Neural Networks learn traditional Latent Class models?

A colleague is a statistician who is an expert in building multi-dimensional Latent Class models. I would like to know if it is possible to use artificial neural networks with several hidden layers to ...
0
votes
0answers
14 views

What is interval of the initial weights at Matlab neural network tool?

What is interval of the initial weights at Matlab neural network tool?Yes I know it is randomly but are there limitation?
0
votes
0answers
16 views

Neuro fuzzy systems and it's application

I was wondering if it's possible to use Neuro fuzzy systems where ANNs are used, e.g having a tabular data for regression task or using it for classification tasks. What kind of advantage can give me ...
-3
votes
3answers
224 views

Evolutionary neural architecture?

I'm working on an idea for an AI architecture, and would like to know if there are any apparent flaws, or if there is prior work in this vein. Set I/O so that the neural network can read and write ...
1
vote
1answer
25 views

Clarifying when a loss/cost function calculation and backprop take place

I read different articles and keep getting confused on this point. Not sure if the literature is giving mixed information or I'm interpreting it incorrectly. So from reading articles my ...
2
votes
2answers
48 views

Standard deviation in the # of neurons per layer

In a neural network, by how much does the # of neurons typically vary from layer to layer? PLEASE NOTE: I am NOT asking how to find the optimal # of neurons per lyr. As a hardware design engineer ...
3
votes
2answers
48 views

Are fully connected layers necessary in a CNN?

I have implemented a CNN for image classification. I have not used fully connected layers, but only a softmax. Still, I am getting results. Must I use fully-connected layers in a CNN?
2
votes
1answer
51 views

What are the most common methods to enable neural networks to adapt to changing environments?

For real applications, concept drifts often exist, i.e., the relationship between the input and output changes overtime. Thus, we need our AI or machine learning system to quickly adapt to the ...
1
vote
1answer
75 views

Deep Q Learning Algorithm for Simple Python Game makes player stuck

I made a simple Python game. A screenshot is below: Basically, a paddle moves left and right catching particles. Some make you lose points while others make you gains points. This is my first Deep Q ...
3
votes
1answer
72 views

Can Neural Networks be considered as “Strong AI”?

I've been reading on the differences between "Strong" and "Weak "AI. I was wondering, where do Neural Networks (especially deep ones) fall in this spectrum? Can they be considered "Strong AI"? If ...
3
votes
1answer
58 views

How is REINFORCE used instead of Backpropagation?

In neural networks with stochastic layers I've seen the use of the REINFORCE estimator for estimating the gradient (because it can't be computed directly). Some such examples are Show, Attend and ...
0
votes
0answers
33 views

Not able to properly tune Neural Network via Back Propagation properly

I have a custom code Neural Network(not using keras or any package...Trying to learn the essence of Neural Network from scratch)... Code can be found here I have the per iteration training output(<...
2
votes
0answers
41 views

Does Retina-net's focal loss accomplish its goal?

Taking out the weighting factor we can define focal loss as $$FL(p) = -(1-p)^\gamma log(p) $$ Where $p$ is the target probability. The idea being that single stage object detectors have a huge ...
1
vote
1answer
30 views

What loss function is appropriate for finding “points of interest” in a array of x,y inputs

I am looking into whether a neural network is appropriate to detect "points of interest" (POI) in a set of tuples (say length, and some sensor value). A POI is essentially a quick change in the value ...
1
vote
1answer
36 views

Effect of rescaling of inputs on loss for a simple neural network

I've been trying out a simple neural network on the fashion_mnist dataset using keras. Regarding normalization, I've watched this video explaining why it's necessary to normalize input features, but ...
3
votes
1answer
47 views

Is there a Continuous Conditional Variational Autoencoder?

The Conditional Variational Autoencoder (CVAE), introduced in the paper Learning Structured Output Representation using Deep Conditional Generative Models (2015), is an extension of Variational ...
0
votes
1answer
56 views

How to detect patterns in a data set of given IP addresses using a neural network?

How to detect patterns in a data set of given IP addresses using a neural network? The data set is actually a list of all the vulnerable devices on a network. I want to use a neural network that ...
4
votes
2answers
198 views

When should I use 3D convolution?

I am new to convolutional neural networks, and I am learning 3D convolution. What I could understand is that 2D convolution gives us relationships between low-level features in the X-Y dimension, ...
1
vote
1answer
43 views

Is there a theory behind which model is good for a classification task for the convolutional neural network?

Let say I'm trying to apply CNN for image classification. There are lots of different models to choose and we can try an ensemble, but given a limit amount of resources, it does not allow to try ...
1
vote
0answers
37 views

What are the possible neural network architecture for linear regression or time series regression?

I started modeling a linear regression problem using dense layers (layers.dense), which works fine. I am really excited, and now I am trying to model a time series linear regression problem using CNN, ...
3
votes
1answer
46 views

What weights should I use while back-propagating?

I've started to learn about neural networks recently and I can't find the answer to this question. Let's assume there's a neural network (fig. 1) So if the loss function is: and the derivative is: ...
1
vote
0answers
28 views

AI for Warcraft 3 Dota

I want to create AI for Dota1. Is it possible to create AI for Warcraft 3? How Open AI works in Dota2? I want to know more about algorithms what are in foundation in Open AI.
0
votes
0answers
26 views

Pooling vs Subsampling: Multiple Definitions?

I have seen people using pooling and subsampling synonymously. I have also seen people use them as different processes. I am not sure though if I have correctly inferred what they mean, when they use ...
3
votes
1answer
103 views

Which approach can I use to generate text based on multiple inputs?

I have a little experience in building various models, but I've never created anything like this, so just wondering if I can be pointed in the right direction. I want to create (in python) a model ...
1
vote
1answer
71 views

Would this neural network have short term memory?

I want to design a NN that can remember it's last 7 actions and use them as inputs. So for example it would be able to store words in it's memory. Therefore if it had a choice of 10 different actions, ...
1
vote
1answer
38 views

How do I identify the number and type of objects in the same picture?

I need to identify the number and type of all objects in a picture, so there can be multiple objects of the same type. For example, I have a picture with $10$ animals, and I want my program to tell ...