Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
-1
votes
1answer
49 views

Regularize the weights of the neural networks to binary values?

I have multiple layers of neural network and I am trying to do an autoencoder network. I am interested to make the weights of the encoding layers of the neural networks to take only -1 and +1 and the ...
0
votes
2answers
31 views

Is “Pruning” only applicable to CNNs?

What Is Neural Network Pruning And Why Is It Important Today? The above article only talks about Convolutional Neural Networks: One of the first methods of pruning is pruning entire convolutional ...
0
votes
0answers
7 views

Mapping input vectors of variable length to output vectors of variable lengths with dummy variables

I have a general question about supervised ANNs that map inputs to outputs. It is possible to vary the length of the input and output vectors by inserting some dummy variables that will not be ...
1
vote
1answer
232 views

Why is my validation/test accuracy higher than my training accuracy

Is this due to my dropout layers being disabled during evaluation? I'm classifying the CIFAR-10 dataset with a CNN using the Keras library. There are 50000 samples in the training set; I'm using a ...
0
votes
0answers
8 views

Is it possible to use a Neural Network to interpolate data

I am completely new to Artificial intelligence and Neural Networks. I am currently working on a plasma physics simulation project which requires a very high resolution data set. We currently have the ...
1
vote
1answer
32 views

Can the optimal learning rate differ for different architectures?

In several courses and tutorials about neural networks, people often say that the learning rate (LR) should be the first hyper-parameter to be tuned before we tweak the others. For example, in this ...
2
votes
3answers
43 views

Why not make the training set and validation set one if their roles are similar?

If the validation set is used to tune the hyperparameters and the training set adjusts the weights, why don't they be one thing as they have a similar role, as in improving the model?
0
votes
0answers
39 views

What are the numbers thar are useful (need be stored) other than parameters of a model?

Consider the following method related to buffers in PyTorch ...
2
votes
0answers
18 views

ReLU function converging to local optimum in one case and diverging in the other one

I implemented a simple neural network with 1 hidden layer. I used ReLU as activation function for the hidden layer and the output layer just uses the linear function. To check my implementation I ...
0
votes
0answers
10 views

Deep NN architecture for predicting a matrix from a matrix and list of floats

I am trying to predict a matrix (size RxC) based on an input matrix (size RxC) and a list of floats ...
1
vote
1answer
18 views

Is reconciling shape discrepancies the only purpose of padding?

Padding is a technique used in some of the domains of artificial intelligence. Data is generally available in different shapes. But in order to pass the data as input to a model in deep learning, the ...
2
votes
1answer
62 views

How to create a neural network from a set of equations?

Say I have these equations: $$x_1 = x_2 + 2y_1 + b$$ $$x_2 = y_2 + c$$ $$y_1 = z + a$$ $$y_2 = y_3 + d$$ $$z = z_1 + e$$ $x_1$ depends on $x_2$ (depends on $y_2$ (depends on $y_3$)) and $y_1$ (depends ...
1
vote
0answers
19 views

Is there any existing mechanism that allows us to pass input from randomly selected layers of neural network per iteration?

Consider the following neural network with $\ell$ layers. $$i_0 \rightarrow h_1 \rightarrow h_2 \rightarrow h_3 \cdots \rightarrow h_{\ell-1} \rightarrow o_{\ell} ,$$ where $i, h, o$ stands for ...
0
votes
0answers
29 views
0
votes
0answers
28 views

Neural networks in models and in reality [closed]

I have recently read a modern book on neural systems in biology and found a lot of misconceptions between current models and real systems. At first, real neurons use both inhibitory (negative, -) and ...
3
votes
1answer
124 views

Is batch normalization not suitable for non-gaussian input?

I generate some non-Gaussian data, and use two kinds of DNN models, one with BN and the other without BN. I find that the model DNN with BN can't predict well. The codes is shown as follow: <...
0
votes
1answer
38 views

Using a Neural Network (LSTM) to approve/reject word-type sequences

I would like to train an LSTM neural network to either "approve" or "reject" a string based on the word-type sequence. For instance: "Mike's Airplane" would output "...
1
vote
0answers
37 views

Why doesn't anyone use reinforcement learning to find the best possible alternative to backpropagation?

To be clear, I'm very uninformed on the topic of alternative learning algorithms to backprop, all my knowledge comes from articles like these: lets-not-stop-at-backprop backprop-alternatives we-need-a-...
2
votes
0answers
19 views

Methodologies for passing the best samples for a neural network to learn

Just an idea I am sure I read in a book some time ago, but I can't remember the name. Given a very large dataset and a neural network (or anything that can learn via something like stochastic gradient ...
2
votes
0answers
34 views

How much research, approximately, is done in ANNs?

Does someone know where can I find information about how much research, nowadays, is done in ANNs? I've checked in this document Redes Neuronales: Conceptos básicos y aplicaciones, Universidad ...
2
votes
1answer
110 views

Price difference predictions curve almost vanished

With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well ...
0
votes
1answer
32 views

Is precision of weights unimportant in neural networks?

While reading about Module in PyTorch, I came across a new data type called half datatype. half() method when calls on a Module ...
0
votes
1answer
33 views

Where does batch normalization layers present in a neural network?

Batch normalization is a procedure widely used to train neural networks. Mean and standard deviation are calculated in this step of training. Since we train neural network by dividing training data ...
0
votes
0answers
31 views

What are the input and output gradients in PyTorch?

Suppose I want to train a neural network with $m-$length inputs of form $x = [x_1, x_2, x_3, \cdots, x_m]$ and $n-$length outputs of form $y = [y_1, y_2, y_3, \cdots, y_n]$. Let the number of ...
1
vote
2answers
100 views

How to extract parameters from a text using AI/NLP

lets say I have three texts: "make a heading that says hello word" "make a heading of hello world" "create heading consist of hello world" How can I fetch those groups ...
1
vote
1answer
83 views

How does the regression layer in the localization network of a spatial transformer work?

I am trying to understand the spatial transformer network mentioned in this paper https://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf. I am clear about the last two stages of the ...
5
votes
1answer
145 views

How do big companies, like Facebook, model individuals and their interaction?

As a layman in AI, I want to get an idea of how big data players, like Facebook, model individuals (of which they have so many data). There are two scenarios I can imagine: Neural networks build ...
2
votes
1answer
184 views

Anomaly Detection in distributed system using generated log file

I am developing an AI tool for anomaly detection in a distributed system.  The system supports an interface that combines several individual logs into a single log file generating approx. 7000 entries/...
1
vote
1answer
87 views

How to properly optimize shared network between actor and critic?

I'm building an actor-critic reinforcment learning algorithm to solve environments. I want to use a single encoder to find representation of my environment. When I share the encoder with the actor ...
3
votes
2answers
77 views

Is there any difference between affine transformation and linear transformation?

Consider the following statements from A Simple Custom Module of PyTorch's documentation To get started, let’s look at a simpler, custom version of PyTorch’s Linear module. This module applies an ...
0
votes
0answers
7 views

How to approach receipt interpretation with neural networks

I have never worked with neural networks before, but I want to learn and I have a specific use case that I would want to solve (or at least try to). I have developed software for multiple years, in a ...
1
vote
1answer
88 views

Why does my “entropy generation” RNN do so badly?

I'm new to relatively RNNs, and I'm trying to train generative and guessing neural networks to produce sequences of real numbers that look random. My architecture looks like this (each "circle&...
2
votes
1answer
224 views

Is the dropout technique specific only to neural networks?

In one Udemy course was mentioned that "dropout is unique to neural networks". However, I remember an example of decision trees where nodes that are not participating in the overall result ...
0
votes
1answer
65 views

classification of unseen classes of image in open set classification

I have a scanned image, and they need to be classified in one of the pre-defined image classes, so that it can be sorted. However, the problem is the open nature of the classes. At testing time, new ...
1
vote
3answers
1k views

How to create Partially Connected NNs with prespecified connections using Tensorflow?

I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer....
0
votes
0answers
15 views
2
votes
1answer
98 views

Can you explain me this CNN architecture?

I am starting to get my head around convolutional neural networks, and I have been working with the CIFAR-10 dataset and some research papers that used it. In one of these papers, they mention a ...
1
vote
1answer
38 views

How is the convolution operation connected to neural networks?

I've been reading up on the convolution operation and neural networks. I understand that the convolution operation is defined as: $$(f * g)(t)=\int_{-\infty}^{\infty} f(\tau) g(t-\tau) d \tau$$ The ...
0
votes
1answer
10 views

Can we modelize an RNN by an ANN that takes precedent output as a part of input?

Is it possible to consider an RNN as a classical feedforward neural network that just take the precedent output as a part of the input ?
0
votes
0answers
12 views

Comparing results of different image splicing methods on a part of the CASIA 2.0 dataset

So I am working on an image splicing detection algorithm using ResNet-50 model. I am using the CASIA 2.0 dataset which consists of 7491 Authentic images and 5123 Fake images. However out of the fake ...
1
vote
2answers
2k views

Understanding the loss function in deep Q-learning

I am trying to understand how deep Q learning (DQN) works. To my current understanding, each $Q(s, a)$ functions is estimated to be a function of a feature vector of its state $\phi$(s) and the weight ...
0
votes
1answer
34 views

What are the labels of EMNIST?

I'm using pytorch and downloading the EMNIST dataset using ...
1
vote
0answers
31 views

Do the terms multi-task and multi-output refer to the same thing in the context of deep learning?

Do the terms multi-task and multi-output refer to the same thing in the context of deep learning (with neural networks)? For example, do neural networks for multi-task learning use multiple outputs? ...
4
votes
2answers
2k views

What is the general procedure to use and train neural networks for multi-class classification?

I am very new to machine learning. I am following the course offered by Andrew Ng. I am very confused about how we train our neural network for multi-class classification. Let's say we have $K$ ...
0
votes
0answers
21 views

RLLib - What exactly do the avail_action and action_embed_size represent? How do they work with the action_mask to phase out invalid actions?

So, I'm fairly new to reinforcement learning and I needed some help/explanations as to what the action_mask and avail_action fields alongside the action_embed_size actually mean in RLlib (the ...
1
vote
0answers
36 views

Identifying rotating and resizing letters with background noise

I'm trying to complete a captcha, and here is what it looks like: Between captchas the calligraphy of the letters is the same, but the letters may be resized and rotated. And the background noise (...
2
votes
1answer
31 views

What does “differentiable architecture” mean?

I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term "Differentiable Architecture", ...
0
votes
0answers
19 views

Does Keras or Tensorflow calculate backpropagation?

I am using tensorflow 2.1.0. Does Keras or Tensorflow calculate backpropagation? Where can I find the code for backpropagation in Keras and Tensorflow? Thanks, Jianqiao Huang
2
votes
3answers
2k views

Confusing on GAN loss function

I was trying to understand the loss function of GANs, while I found a little mis-match between different papers. This is the screen-shot from the original paper of Goodfellow at https://arxiv.org/...

1
2 3 4 5
43