1 vote
86 views

How does a single neuron in hidden layer affect training accuracy [duplicate]

I'm currently a student learning about AI Networks. I've came across a statement in one of my Professor's books that a FFBP (Feed-Forward Back-Propagation) Neural Network with a single hidden layer ...
• 11
12k views

What are some well-known problems where neural networks don't do very well?

Background: It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of their universal approximation capabilities. However, in ...
• 545
4k views

What is the difference between machine learning and deep learning?

Can someone explain to me the difference between machine learning and deep learning? Is it possible to learn deep learning without knowing machine learning?
• 333
8k views

Why do we prefer ReLU over linear activation functions?

The ReLU activation function is defined as follows $$y = \operatorname{max}(0,x)$$ And the linear activation function is defined as follows $$y = x$$ The ReLU nonlinearity just clips the values ...
• 499
3k views

What are some resources on computational learning theory?

Pretty soon I will be finishing up Understanding Machine Learning: From Theory to Algorithms by Shai Ben-David and Shai Shalev-Shwartz. I absolutely love the subject and want to learn more, the only ...
• 248
2k views

What is the fundamental difference between an ML model and a function?

A model can be roughly defined as any design that is able to solve an ML task. Examples of models are the neural network, decision tree, Markov network, etc. A function can be defined as a set of ...
• 3,792
2k views

How to estimate the capacity of a neural network?

Is it possible to estimate the capacity of a neural network model? If so, what are the techniques involved?
• 308
495 views

Are there any rules of thumb for having some idea of what capacity a neural network needs to have for a given problem?

To give an example. Let's just consider the MNIST dataset of handwritten digits. Here are some things which might have an impact on the optimum model capacity: There are 10 output classes The inputs ...
• 1,329
2k views

What is the meaning of "stationarity of statistics" and "locality of pixel dependencies"?

I'm reading the ImageNet Classification with Deep Convolutional Neural Networks paper by Krizhevsky et al, and came across these lines in the Intro paragraph: Their (convolutional neural networks') ...
982 views

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
• 763
400 views

Mathematical foundations of the ability to learn

I am an undergraduate student in applied mathematics with an interest in artificial intelligence. I am currently exploring topics where I could do research. Coming from a mathematical background I am ...
• 143
225 views

If deep learning is a black box, then why are companies still investing in it?

If deep learning is a black box, then why are companies still investing in it?
• 129
229 views

Do authors generally use fully connected layer instead of affine transformation?

We generally encounter the following statement several times The input vector is first fed into a fully connected layer...... Since linear activation functions, such as identity function, can so ...
• 3,792
293 views

Are CNN, LSTM, GRU and transformer AGI or computational intelligence tools?

Will CNN, LSTM, GRU and transformer be better classified as Computational Intelligence (CI) tools or Artificial General Intelligence (AGI) tools? The term CI arose back when some codes like neural ...
• 123