Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
5
votes
2answers
873 views

What is the current research in artificial intelligence in the field of data compression?

What is the current research in artificial intelligence and machine learning in the field of data compression? I have done my research on the PAQ series of compressors, some of which use neural ...
6
votes
2answers
411 views

Which machine learning algorithm could I use to break up a poem by lines?

I want to create a network to predict the break up of poetry lines. The program would receive as input an unbroken poem, and would output the poem broken into lines. For example, an unbroken poem ...
4
votes
2answers
198 views

Is a multilayer perceptron a recursive function?

I read somewhere that a multilayer perceptron is a recursive function in its forward propagation phase. I am not sure, what is the recursive part? For me, I would see an MLP as a chained function. So, ...
2
votes
1answer
67 views

Is there anything theoretically revolutionary about Deep Neural Networks?

In recent years, we have seen quite a lot of impressive display of Deep Neural Network (DNN), as demonstrated most famously by AlphaGo and its cousin programs. But if I understand correctly, deep ...
5
votes
1answer
3k views

Why most imperfect information games usually use non machine learning AI?

To provide a bit of context, I'm a software engineer & game enthusiast (card games, especially). The thing is I've always been interested in AI oriented to games. In college, I programmed my own ...
0
votes
1answer
17 views

How to make input variable as trainable parameter in a neural network?

I am working on an optimization problem. First, I have done forward training to work the network as a surrogate model, then I freeze the output and I want to find an optimal value of input for a given ...
7
votes
3answers
363 views

What are the best machine learning models for music composition?

What are the best machine learning models that have been used to compose music? Are there some good research papers (or books) on this topic out there? I would say, if I use a neural network, I would ...
0
votes
0answers
21 views

How to think and build an AI project?

I am starting to study artificial intelligence by my own, since my college stopped the classes by the covid. For learning purposes, I want to build a neural network that can optimize the builds of a ...
3
votes
1answer
160 views

How does the neural-network know how to tweak weights for a specific neuron?

I know backpropagation uses cost and gradient descent to tweak the weights to minimize the cost. But how does it know which weights to give more weight to in the first place? Is there something inside ...
4
votes
1answer
1k views

Do we know what the units of neural networks will do before we train them?

I was learning about back-propagation and, looking at the algorithm, there is no particular 'partiality' given to any unit. What I mean by partiality there is that you have no particular ...
5
votes
1answer
113 views

Why evolutionary training of neural networks is not popular?

Evolutionary algorithms are mentioned in some sources as a method to train a neural network (finding weights, not hyperparameters). However, I have not heard about one practical application of such an ...
0
votes
0answers
10 views

What is the best way to convert an historical log data file to a Markov Decision Process (MDP) to perform Q-learning?

Hypothetically, I have an historical log file whose entries contain the instantaneous throughput for the transfer of a set of files (25,000 files ranging in size from 101KB to 222MB) recorded every ...
3
votes
0answers
38 views

When do two identical neural networks have uncorrelated errors?

In Chapter 9, section 9.1.6, Raul Rojas describes how committees of networks can reduce the prediction error by training N identical neural networks and averaging the results. If $f_i$ are the ...
3
votes
1answer
74 views

How to classify human actions?

I'm quite new to machine learning (I followed the Coursera course of Andrew Ng and now starting deeplearning.ai courses). I want to classify human actions real-time like: Left-arm bended Arm above ...
20
votes
3answers
2k views

How are Artificial Neural Networks and the Biological Neural Networks similar and different?

I've heard multiple times that "Neural Networks are the best approximation we have to model the human brain", and I think it is commonly known that Neural Networks are modelled after our brain. I ...
1
vote
1answer
54 views

Are artificial neural networks based on human neural networks? [duplicate]

Are artificial neural networks based on human neural networks? Additionally, are the current computers capable of handling the power our brain has with the same speed?
31
votes
4answers
1k views

How to find the optimal number of neurons per layer?

When you're writing your algorithm, how do you know how many neurons you need per single layer? Are there any methods for finding the optimal number of them, or is it a rule of thumb?
1
vote
2answers
2k views

How can we compare, in terms of similarity, two pieces of text?

How can we compare, in terms of similarity (and/or meaning), two pieces of text (or documents)? For example, let's say that I want to determine whether a document is a plagiarized version of another ...
8
votes
3answers
796 views

What makes the animal brain so special?

Whenever I read any book about neural networks or machine learning, their introductory chapter says that we haven't been able to replicate the brain's power due to its massive parallelism. Now, in ...
3
votes
0answers
90 views
+50

Is it really possible to create the “Perfect Cylinder” used in Universal Approximation Theorem for 1-hidden layer Neural Network?

There are proofs for the universal approximation theorem with just 1 hidden layer. The proof goes like this: Create a "bump" function using 2 neurons. Create (infinitely) many of these ...
2
votes
1answer
300 views

Should I apply ReLU to non negative output?

Suppose I want to predict the position of a sensor based on its reading. I can first predict the unit vector and predict the distance to be multiplied to this vector. And I know that distance will ...
2
votes
0answers
140 views

If we use a perceptron with a non-monotonic activation function, can it solve the XOR problem?

I found several papers about how to build a perceptron able to solve the XOR problem. The papers describe a solution where the heaviside step function is replaced by a non-monotonic activation ...
5
votes
5answers
574 views

Why can't the XOR linear inseparability problem be solved with one perceptron like this?

Consider a perceptron where $w_0=1$ and $w_1=1$: Now, suppose that we use the following activation function \begin{align} f(x)= \begin{cases} 1, \text{ if }x =1\\ 0, \text{ otherwise} \end{cases} \...
3
votes
1answer
183 views

Using tensor networks as machine learning models

Tensor networks (check this paper for a review) are a numerical method originally introduced in condensed matter physics to model complex quantum systems. Roughly speaking, such systems are described ...
1
vote
1answer
2k views

What’s the difference between LSTM and RNN?

What's the difference between LSTM and RNN? I know that RNN is a layer used in neural networks, but what exactly is an LSTM? Is it also a layer with the same characteristics?
3
votes
1answer
60 views

How are training hyperparameters determined for large models?

When training a relatively small DL model, which takes several hours to train, I typically start with some starting points from literature and then use a trial-and-error or grid-search approach to ...
2
votes
2answers
182 views

Why can't we use Google Translate for every translation task?

Once a book is published in a language, why can't the publishers use Google Translate AI or some similar software to immediately render the book in other languages? Likewise for Wikipedia: I'm not ...
10
votes
2answers
659 views

Are the shortcomings of neural networks diminishing?

Having worked with neural networks for about half a year, I have experienced first-hand what are often claimed as their main disadvantages, i.e. overfitting and getting stuck in local minima. However, ...
6
votes
3answers
243 views

What kinds of problems can AI solve without using a deep neural network?

A lot of questions on this site seem to be asking "can I use X to solve Y?", where X is usually a deep neural network, and Y is often something already addressed by other areas of AI that ...
8
votes
3answers
953 views

Are ReLUs incapable of solving certain problems?

Background I've been interested in and reading about neural networks for several years, but I haven't gotten around to testing them out until recently. Both for fun and to increase my understanding, I ...
5
votes
1answer
1k views

What is an appropriate fitness function for a simulated self-driving car?

I have been working for ages on a neuro-evolution AI program, where cars learn how to race around a track. Presently, I have a rudimentary fitness function that awards points for every degree ...
6
votes
2answers
898 views

Can neural networks efficiently solve the traveling salesmen problem?

Can neural networks efficiently solve the traveling salesmen problem? Are there any research papers that show that neural networks can solve the TSP efficiently? The TSP is an NP-hard problem, so I ...
32
votes
3answers
22k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, with ...
4
votes
1answer
75 views

How can I use Generative Adversarial Networks to solve the imbalanced class problem?

Problem setting We have to do a binary classification of data given a training dataset $D$, where most items belong to class $A$ and some items belong to class $B$, so the classes are heavily ...
3
votes
1answer
97 views

Which books or papers clearly explain the relation between Ising models and deep neural networks?

I am looking for a book or paper which clearly explains the relationship between Ising models and deep neural networks. Can anyone provide any references?
5
votes
2answers
234 views

What are some intermediate or advanced books on neural networks?

Is anyone able to recommend some resources (preferably books) on the topic of neural networks that goes beyond that of introductory reading? I'm still relatively new to the subject, however, I have ...
2
votes
1answer
162 views

Why am I getting a difference between training accuracy and accuracy calculated with Keras' predict_classes on a subset of the training data?

I'm trying to solve a binary classification problem with AlexNet. I split the original dataset into training and validation datasets using a 70/30 ratio. I have trained my neural network with a ...
1
vote
2answers
427 views

How can I use a 2-dimensional feature matrix as the input to a neural network?

How can I use a 2-dimensional feature matrix, rather than a feature vector, as the input to a neural network? For a WWII naval wargame, I have sorted out the features of interest to approximate the ...
4
votes
2answers
139 views

Will a neural network always predict the correct label if it sees the exact same input during training and testing?

If I'm performing a text classification task using a model built in Keras, and, for example, I am attempting to predict the appropriate tag for a given Stack Overflow question: How do I subtract 1 ...
2
votes
1answer
58 views

Can we use ML to do anything else other than predicting (in the case of mathematical problems)?

(The math problem here just serves as an example, my question is on this type of problems in general). Given two Schur polynomials, $s_\mu$, $s_\nu$, we know that we can decompose their product into a ...
3
votes
1answer
183 views

Why does every neuron in hidden layers of a multi-layer perceptron typically have the same activation function?

Why does every neuron in a hidden layer of a multi-layer perceptron (MLP) typically have the same activation function as every other neuron in the same or other hidden layers (so I exclude the output ...
0
votes
1answer
22 views

Why won't my model train with CTC loss?

I am trying to train an LSTM using CTC loss, but the loss does not decrease when I train it. I have created a minimal example of my issue by creating training data where the network simply has to copy ...
2
votes
1answer
281 views

How does a batch normalization layer work?

I understood that we normalize to input features in order to bring them on the same scale so that weights won't be learned in arbitrary fashion and training would be faster. Then I studied about ...
7
votes
1answer
591 views

How should the neural network deal with unexpected inputs?

I recently wrote an application using a deep learning model designed to classify inputs. There are plenty of examples of this using images of irises, cats, and other objects. If I trained a data ...
5
votes
1answer
172 views

Why do layered neural nets struggle with continous data?

In this article here, the writer claims that a new type of neural net is required to deal with data that is both continuous, and also sparsely sampled. It was my understanding that this was the ...
2
votes
1answer
59 views

Why is neural networks being a deterministic mapping not always considered a good thing?

Why is neural networks being a deterministic mapping not always considered a good thing? So I'm excluding models like VAEs since those aren't entirely deterministic. I keep thinking about this and my ...
0
votes
0answers
15 views

Architecture for Logistic Regression with Arbitrary Number of Options

Suppose I want to design a neural network to choose one of several mutually exclusive options. This may normally be done via logistic regression, where the input of the network would be a [batch x ...
1
vote
0answers
72 views

Given the coordinates of an object in an image, is it possible to predict the coordinates of the same object in a different perspective?

I am trying to figure out how to approach this. Given training data of images and the pixel coordinates of the centre of an object in that image, would it be possible to predict the pixel coordinates ...
2
votes
0answers
22 views

How to train/update neural networks faster without a decrease in performance?

I noticed that there are many studies in recent years on how to train/update neural networks faster/quicker with equal or better performance. I find the following methods(except the chips arms race): ...
1
vote
1answer
139 views

Finding patterns in binary files using deep learning

I am a newbie in deep learning and wanted to know if the problem I have at hand is a suitable fit for deep learning algorithms. I have thousands of fragments each of about 1000 bytes size (i.e. ...

1
2 3 4 5
39