0
$\begingroup$

I am trying to understand the difference between feedforward NN and chained linear regression models, if and why they can model nonlinear functions.

  • both are able to model non-linear dependencies
  • feedforward NN without activation functions cannot model nonlinear dependecies and number of layers does not matter in this context
  • chained regression models can model nonlinear dependencies

I am confused because above statements comes from chatgpt and I am not sure if all are correct I believe they are mutually contradictory, because from what I know feedforward NN without activation function can be cosidered as chained linear regression models.

$\endgroup$

1 Answer 1

2
$\begingroup$

I assume by Chained Linear Regression (CLR) you mean something like $f_1(f_2(x)) = y$, where the functions $f_i$ would be your Linear Regression models. In that case, CLR cannot model non-linear dependencies, as the composition of two linear functions is itself linear. For the same reason, your second point is correct.

And to answer your question: A multilayer perceptron (i.e. feedforward NN with multiple layers) consists of multiple layers and each layer is just a matrix multiplication. In CLR, each of your functions $f_i$ is also just a matrix multiplication (the parameters of these matrices are optimized during training). So in that sense, a multilayer perceptron without non-linearities and chained linear regression are the same thing. However, the data may flow a bit differently through a chained linear regression, so in the exact implementation, the models may still differ a bit, just not in their ability to model non-linear dependencies.

$\endgroup$
2
  • $\begingroup$ "So in that sense, a multilayer perceptron without non-linearities " by non-linearities you meant without activation function, if this is correct can I simplify that CLR + activation function = feedforward NN ? $\endgroup$ Jan 19 at 22:20
  • $\begingroup$ "by [without] non-linearities you meant without activation function", exactly! To your question: "CLR + activation function(s) = feedforward NN + activation function(s)" If you say it like this, then it's correct. Note though that both feedforward NN and CLR are families of models: A feedforward neural network is everyneural network that doesn't contain recurrent connections (neurons feeding back information to an earlier layer) but it can take many forms. It only describes that information flows onyl one way. Multilayer perceptron is a better description of what you mean here I guess. $\endgroup$
    – Chillston
    Jan 20 at 1:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .