Linked Questions

1 vote
1 answer

How does a single neuron in hidden layer affect training accuracy [duplicate]

I'm currently a student learning about AI Networks. I've came across a statement in one of my Professor's books that a FFBP (Feed-Forward Back-Propagation) Neural Network with a single hidden layer ...
Konrad Ł's user avatar
56 votes
11 answers

What are some well-known problems where neural networks don't do very well?

Background: It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of their universal approximation capabilities. However, in ...
ABIM's user avatar
  • 545
23 votes
5 answers

What is the difference between machine learning and deep learning?

Can someone explain to me the difference between machine learning and deep learning? Is it possible to learn deep learning without knowing machine learning?
Addis's user avatar
  • 333
11 votes
2 answers

Why do we prefer ReLU over linear activation functions?

The ReLU activation function is defined as follows $$y = \operatorname{max}(0,x)$$ And the linear activation function is defined as follows $$y = x$$ The ReLU nonlinearity just clips the values ...
imflash217's user avatar
9 votes
1 answer

What are some resources on computational learning theory?

Pretty soon I will be finishing up Understanding Machine Learning: From Theory to Algorithms by Shai Ben-David and Shai Shalev-Shwartz. I absolutely love the subject and want to learn more, the only ...
PMaynard's user avatar
  • 248
5 votes
4 answers

What is the fundamental difference between an ML model and a function?

A model can be roughly defined as any design that is able to solve an ML task. Examples of models are the neural network, decision tree, Markov network, etc. A function can be defined as a set of ...
hanugm's user avatar
  • 3,792
7 votes
2 answers

How to estimate the capacity of a neural network?

Is it possible to estimate the capacity of a neural network model? If so, what are the techniques involved?
jaeger6's user avatar
  • 308
11 votes
3 answers

Are there any rules of thumb for having some idea of what capacity a neural network needs to have for a given problem?

To give an example. Let's just consider the MNIST dataset of handwritten digits. Here are some things which might have an impact on the optimum model capacity: There are 10 output classes The inputs ...
Alexander Soare's user avatar
4 votes
1 answer

What is the meaning of "stationarity of statistics" and "locality of pixel dependencies"?

I'm reading the ImageNet Classification with Deep Convolutional Neural Networks paper by Krizhevsky et al, and came across these lines in the Intro paragraph: Their (convolutional neural networks') ...
Shirish Kulhari's user avatar
19 votes
1 answer

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
mark mark's user avatar
  • 763
4 votes
2 answers

Mathematical foundations of the ability to learn

I am an undergraduate student in applied mathematics with an interest in artificial intelligence. I am currently exploring topics where I could do research. Coming from a mathematical background I am ...
Matheo's user avatar
  • 143
2 votes
2 answers

If deep learning is a black box, then why are companies still investing in it?

If deep learning is a black box, then why are companies still investing in it?
Case Msee's user avatar
  • 129
0 votes
1 answer

Do authors generally use fully connected layer instead of affine transformation?

We generally encounter the following statement several times The input vector is first fed into a fully connected layer...... Since linear activation functions, such as identity function, can so ...
hanugm's user avatar
  • 3,792
2 votes
1 answer

Are CNN, LSTM, GRU and transformer AGI or computational intelligence tools?

Will CNN, LSTM, GRU and transformer be better classified as Computational Intelligence (CI) tools or Artificial General Intelligence (AGI) tools? The term CI arose back when some codes like neural ...
user_1_1_1's user avatar
0 votes
1 answer

How many layers and neurons in a FFNN do I need to make it equivalent to a CNN?

I started to learn machine learning early, and I studied the convolutional neural network and its ability to understand images and how it helps to reduce the number of parameters that need to be tuned....
Mahmoud Kanbar's user avatar

15 30 50 per page