Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
2
votes
1answer
17 views

What is the goal of weight initialization in neural networks?

This is a simple question. I know the weights in a neural network can be initialized in many different ways like: random uniform distribution, normal distribution, and Xavier initialization. But what ...
0
votes
0answers
14 views

What are some good models to use for spelling corrections?

I used OCR to extract text from an image, but there are some spelling mistakes in it : The text is as follows : ...
1
vote
2answers
22 views

Is it possible to have a negative output using only ReLU activation functions, but not in the final layer?

I know that if you use an ReLU activation function at a node in the neural network, the output of that node will be non-negative. I am wondering if it is possible to have a negative output in the ...
0
votes
1answer
29 views

What is the search depth of AlphaGo and AlphaGo Zero?

I cannot find reliable sources but someone says it is 40 moves and someone else says it is 50+ moves. I read their papers and they use value function (NN) and policy function to trim the tree, so more ...
2
votes
1answer
26 views

How much can an inclusion of the number of iterations have on the training of an MLP?

My doubt is like this : Suppose we have an MLP. In an MLP, as per the backprop algorithm (back-propagation algorithm), the correction applied to each weight is : $$ w_{ij} := -\eta\frac{\partial E}{\...
1
vote
0answers
18 views

Literature on computational modelling involving neuronal ensemblies

Straying from the current trends in deep learning, there is an, arguably, interesting idea of neuronal ensembles possibly providing an alternative to the current "layered feature detectors" ...
4
votes
1answer
92 views

What, exactly, does the REINFORCE update equation mean?

I understand that this is the update for the parameters of a policy in REINFORCE: $$ \Delta \theta_{t}=\alpha \nabla_{\theta} \log \pi_{\theta}\left(a_{t} \mid s_{t}\right) v_{t} $$ Where 𝑣𝑡 is ...
1
vote
0answers
18 views

Duplicating calculations in CNN-LSTM architecture

I want to use frames from video game and analyze them using CNN and LSTM. But when I have the model defined like that ...
0
votes
1answer
21 views

Can a neural network be trained on a dataset containing only values for true output for a classification problem?

I am using a dataset from Google which contains 1,27,000 data points on simulated concentrations of the atmosphere of exoplanets which can sustain life. So, the output label of all these data points ...
0
votes
0answers
37 views

Could the neural network automatically calculate and get different one-to-many quantities relative to their parent quantity?

Let's say I have a primary dataset that its secondary dataset is hundreds to match and group like an one-to-many relationship. I'm new in this world of the AI but my problem is that many child groups ...
1
vote
2answers
37 views

Extract features with CNN and pass as sequence to RNN

I read an article about captioning videos https://blog.coast.ai/five-video-classification-methods-implemented-in-keras-and-tensorflow-99cad29cc0b5 and I want to use solution number 4 (extract features ...
0
votes
1answer
24 views

Zero shot learning available labels in testing set

As we all know, zero shot learning involves a model predicting classes that it has not seen. But we are given all the attributes each class might have. Is it fair to assume that we are "aware&...
1
vote
1answer
30 views

What are some suitable positive functions as activations of neural networks?

I am working on a deep Q-learning project. My project is different than normal deep Q-learning. The rewards of my neural network must be positive because I need their values to importance sample ...
0
votes
2answers
35 views

How could I convolve a 4D image and a 4D filter with stride?

I want to create a CNN in Python, specifically, only with NumPy, if possible. For optimizing the time of convolution (actually correlation) in the network, I wanna try to use FFT-based convolution. ...
0
votes
1answer
37 views

What are the rules behind vector product in gradient?

Let's suppose we have calculated the gradient and it came out to be $f(WX)(1-f(W X))X$, where $f()$ is the sigmoid function, $W$ of order $2\times2$ is the weight matrix, and $X$ is an input vector of ...
-2
votes
0answers
63 views

How do I write the architecture (layers, activation functions, etc.) of a neural network in pseudocode?

I am looking to convert a CNN model written in Python (keras) to pseudocode. I am mostly trying to find out the logic on how to describe the layers, the filters, the activation functions etc in ...
0
votes
0answers
39 views

How to input a given sequence to a transformer (or an RNN) with probability of occurrence?

I'm experimenting with music and transformers, and I have sequences $S$ of shape: $(B,L,N)$ where $B$ is the batch size, $L$ is the sequence length, and $N=12$ are the number of musical notes with ...
1
vote
0answers
37 views

Is there any way where you can train a Neural Network with only one data point in the dataset?

I was working on a project involving the search for biosignatures (signs of life) on exoplanets and the probability of that planet harboring life. In this case, we know that Earth is the only planet ...
-1
votes
0answers
33 views

How to have neural network maximize the number of outputs with error less than x?

I want to have as many outputs with error (output-desired output) less than 0.004, without caring about the error of outputs that are larger. They might as well be a million. I tried changing the cost ...
1
vote
0answers
12 views

Neural Network for locating shifting resonant frequencies

I have multiple FFT's taken from a sample at different pressures, through different analysis I can see that the resonant frequencies are shifting in the spectrum for each FFT at a different pressure. ...
2
votes
0answers
14 views

Is regular/offline gradient descent ANN training equivalent to “rehearsal” in incremental learning?

I am self-learning incremental learning and read that rehearsal learning is retraining with old data. In essence, isn't this the exact same thing as normal batch/stochastic gradient descent? You train ...
2
votes
2answers
134 views

How to identify if 2 faces contain the same person?

I have got numerous frames and I've detected all the faces in all the frames using Retinaface. However I need to track the faces of people over frames. For this purpose, I assumed I could try finding ...
0
votes
1answer
51 views

Can entire neural networks be composed of only activation functions?

Inverse Reinforcement Learning based on GAIL and GAN-Guided Cost Learning(GAN-GCL), uses a discriminator to classify between expert demos and policy generated samples. Adversarial iRL, build upon GAN-...
0
votes
1answer
42 views

Are there neural networks with 3-dimensional topologies?

The topologies (or architectures) of the neural networks that I have seen so far are only 2-dimensional. So, are there neural networks whose topology is 3-dimensional (i.e. they have a width, height, ...
0
votes
1answer
52 views

What are the mathematical prerequisites needed to understand research papers on neural networks? [closed]

I know we have developed some mathematical tools to understand deep neural networks, gradient descent for optimization, and basic calculus. Recently, I encountered arxiv paper that describes higher ...
1
vote
0answers
46 views

What is the efficiency of trained neural networks?

Training neural networks takes a while. My question is, how efficient is a neural network that is completely trained (assuming it's not a model that is constantly learning)? I understand that this is ...
2
votes
0answers
26 views

Handling a Large Discrete Action Space in Deep Q Learning

I am attempting to solve a timetabling problem using deep Q learning. It could be thought of as a resource allocation problem to obtain some certificate of 'optimality'. However, how to define and ...
1
vote
1answer
46 views

What's a good neural network for this problem?

I am very new to the field of AI so please bear with me. Say there is a dice with three sides, -1,0 and 1, and I want to predict which side it lands on (so only one output is needed I guess). The ...
0
votes
2answers
60 views

Is there a way to make my neural network discard inputs with bad results from learning?

What I want to achieve is this: If my desired outputs are [1, 2, 3, 4] I would rather have my network produce this output: [0.99, 2.01, 999, 4.01] than say this: [0.94, 1.88, 3.12, 4.1] So I'd rather ...
0
votes
0answers
37 views

What prerequisite knowledge should I have to learn about neuromorphic chips and computing?

What prerequisite knowledge should I have and what are the best resources to learn about neuromorphic chips/computing on a deep, technical level? I would like to understand everything from the ...
0
votes
0answers
13 views

Why is domain adaptation and generative modelling for knowledge graphs still not applied widely in enterprise data? What are the challenges?

I see that domain adaptation and transfer learning has been widely adopted in image classification and semantic segmentation analysis. But it's still lacking in providing solutions to enterprise data, ...
0
votes
1answer
26 views

What is the dimension of my output of the form (2n + 1, 2n + 1, #filters) after a MaxPooling layer

I'm trying to white board the different mechanisms behind a convolutional neural network. I have on question regarding the dimension of my volume after using a max pooling layer. Let's suppose I have ...
1
vote
1answer
72 views

Is my “Insane Mind” design for a classifier novel or effective?

This question is in relation to a previous doubt of mine : Are there neural networks where nodes are randomly selected from among a set of nodes (in random orders and a random number of times)? I have ...
0
votes
1answer
35 views

Is there any network/paper used to analyse music scores?

As I am curious on music theory I would like to know that If is there any such network that analyse like labeling chords, or doing a roman numeral analysis. Like an example below: Source It does not ...
2
votes
1answer
41 views

Why does using a higher representation space lead to performance increase on the training data but not on the test data?

I read the following from a book: You can intuitively understand the dimensionality of your representation space as “how much freedom you’re allowing the model to have when learning internal ...
0
votes
0answers
32 views

Given the same features, do logistic regression and neural networks produce the same output?

I have a binary classification problem. I have variables (features) var1, var2, var3, ..., var14. Using these variables (aka features) in a logistic regression, I get their weights. If I use the same ...
1
vote
1answer
22 views

Can I use one-hot vectors for text classification?

For an upcoming project I'm trying to write a text classifier for the IMDb sentiment analysis dataset. This needs to vectorize words using an embedding layer and then reduce the dimensions of the ...
0
votes
2answers
82 views

What are examples of problems where neural networks have achieved human-level or higher performance?

What are examples of problems where neural networks have been used and have achieved human-level or higher performance? Each answer can contain one or more examples. Please, provide links to research ...
1
vote
0answers
28 views

How to understand this NN architecture?

I was reading a paper Multi-Agent Reinforcement Learning for Adaptive User Association in Dynamic mmWave Networks and I was stuck understanding the deep neural network architecture that was used. The ...
1
vote
0answers
19 views

How do non-local neural networks relate to attention and self-attention?

I've been reading non-local neural networks as explained in the original paper. My understanding is that they solve the restrained reception of local filters. I see how they are different from ...
2
votes
1answer
43 views

What is the weight matrix in self-attention?

I've been looking into self-attention lately, and in the articles that I've been seeing, they all talk about "weights" in attention. My understanding is that the weights in self-attention ...
6
votes
1answer
85 views

Why are neural networks preferred to other classification functions optimized by gradient decent

Consider a neural network, e.g. as presented by Nielsen here. Abstractly, we just construct some function $f: \mathbb{R}^n \to [0,1]^m$ for some $n,m \in \mathbb{N}$ (i.e. the dimensions of the input ...
3
votes
1answer
71 views

Are there neural networks where nodes are randomly selected from among a set of nodes (in random orders and a random number of times)?

I am trying to make a classifier. I am new to AI (even if I know the definition and all such a bit) , and also I have no idea of how to implement it properly by myself even if I know a bit of Python ...
1
vote
1answer
41 views

Is it feasible using today's technology to use an AI training algorithm to custom teach a robot to do common household cores?

Like making a bed, washing dishes, taking out the garbage, etc., by training it on the video of specific individuals doing those cores in their own unique environments? I have researched what machine ...
0
votes
1answer
35 views

Why does the output shape of a Dense layer contain a batch size?

I understand that the batch size is the number of examples you pass into the neural network (NN). If the batch size is 10, it means you feed the NN 10 examples at once. Assuming I have an NN with a ...
1
vote
1answer
30 views

How classification neural nets are different from simple dimension reduction + clustering?

I know the training of neural nets involves some sort of dimension manipulation to separate classes of different features. If there is no variation of features, no matter for neural nets or simple ...
0
votes
0answers
19 views

Logistic Regression or General Machine Learning Model using Federated Learning

Past few days I am doing some research on Federated Learning. I got many solutions with MNIST dataset using Nural Net but I am thinking to solve some common Machine Learning problem like Churn ...
0
votes
0answers
17 views

Class activation maps for 3D Convolutional neural network?

I have implemented a 3D convolutional neural network and I was not able to find resources for interpretation for my model. I have found some techniques such as GradCam and GradCam++ but these generate ...
1
vote
0answers
42 views

Why scaling reward drastically affects performance?

I have devised an gridworld-like environment where a RL agent is tasked to cover all the blank squares by passing through them. Possible actions are up, down, left, right. The reward scheme is the ...
0
votes
0answers
24 views

What is the differences among CNN RNN DNN ANN? [duplicate]

Im a beginner, interesting in AI recently. What is the differences among CNN RNN DNN ANN?

1
2 3 4 5
36