I'm new to Neural Network and would like understand its essential parts and difference from simple logistic regression.

Let's take an example of Coffee Roasting prediction (example from Andre NG course), where we have temperature and roasting duration as input (features), and output is 0 - coffee is bad, 1 - coffee is good.

With simple logistic regression, I can train the model with training dataset (temperature, duration -> 0(bad)/1(good)).

That's clear for me, I do understand how it works, how model is trained (with loss/cost function, gradient descent, etc). We can come up with non-linear decision boundary with polynomial model.

Let's use neural network with 1 hidden layer and 2 units (neurons) in it.

Q1. As I understand, each unit is logistic regression as described above, for instance, unit1 produces prediction, but what's the mathematical meaning of combinations of predictions (activations) of unit1 and unit2? Is it just a way to make model non-linear, more random? (Like Sigmoid of Sigmoid, when we pass activation from layer to layer) and come up with more complex decision boundary?

Q2. Do I understand it right that benefit of Neural Network is that it comes up with best new features automatically compared to manual feature engineering in logistic regression? (in case I want to design more complex decision boundary with manually iterating with different combination of polynomial model)

Q3. Do I understand correctly that random initialization of weights in the beginning of training process is the essential part of neural network training? This way each layer/unit will learn different aspects of the problem?

  • $\begingroup$ Please, ask only one question per post. So, if you have multiple ones, ask them in separate posts. Thanks $\endgroup$
    – nbro
    Commented Jan 19 at 1:22


You must log in to answer this question.

Browse other questions tagged .