# Can multiple linear regression using the least squares(OLS) method, also be used to solve simple linear regression problems? Would both be equivalent?

Simple Linear Regression reference: https://online.stat.psu.edu/stat462/node/93/

Multiple Linear Regression reference: https://online.stat.psu.edu/stat462/node/131/

I see that the way to calculate the coefficients of simple linear regression is different from multiple linear regression. The formula for calculating multiple linear regression coefficients uses an inverse matrix. While the simple linear regression formula does not need the inverse matrix. Simple linear regression, learn the relationships between two continuous quantitative variables. But multiple linear regression is used when we have more variables.

This leaves me with a question: If I know the multiple linear regression formula to calculate the coefficients, if I apply the multiple linear regression formula using only 2 quantitative variables, should the results be equivalent to the simple linear regression formula?

Can multiple linear regression using the least squares(OLS) method, also be used to solve simple linear regression problems? Would both be equivalent?

Yes, multiple linear regression using the ordinary least squares (OLS) method can be used to solve simple linear regression problems, and the results would be equivalent.

Simple linear regression involves one independent variable X and one dependent variable Y, and models the relationship as:

Y = beta0 + beta1 * X + e

where beta0 is the intercept, beta1 is the slope coefficient, and e is the error term.

In contrast, multiple linear regression involves more than one independent variable X1, X2, ..., Xn and one dependent variable Y, and models the relationship as:

Y = beta0 + beta1 * X1 + beta2 * X2 + ... + betan * Xn + e

where beta0 is the intercept, beta1, beta2, ..., betan are the slope coefficients, and e is the error term.

For simple linear regression, the coefficients are calculated using straightforward formulas:

beta1 = sum((Xi - mean(X)) * (Yi - mean(Y))) / sum((Xi - mean(X))^2) and beta0 = mean(Y) - beta1 * mean(X)

Multiple linear regression generalizes this to more than one independent variable, but when reduced to a single variable, the matrix algebra used to compute the coefficients simplifies to the same formulas as above.

Here's a Python example that demonstrates that you get equivalent coefficients using simple and multiple linear regression:

from sklearn.linear_model import LinearRegression
from sklearn.datasets import make_regression
import numpy as np

# Generate some data with three features
X, y = make_regression(n_samples=100, n_features=3, noise=10)

# Isolate the first feature for simple linear regression
X_simple = X[:, 0].reshape(-1, 1)  # Only use the first feature

# Replace the last 2 features with random noise for multiple linear regression
# to avoid multicollinearity between features
X[:, 1:] = np.random.normal(size=(100, 2))

# Fit Simple Linear Regression with only the first feature
simple_lr = LinearRegression()
simple_lr.fit(X_simple, y)

# Fit Multiple Linear Regression with all three features
multiple_lr = LinearRegression()
multiple_lr.fit(X, y)

# Coefficients from Simple Linear Regression
print("Simple Linear Regression (using only the first feature):")
print(f"Intercept: {simple_lr.intercept_:.4f}, Slope for first feature: {simple_lr.coef_[0]:.4f}")

# Coefficients from Multiple Linear Regression
print("\nMultiple Linear Regression (using all three features):")
print(f"Intercept: {multiple_lr.intercept_:.4f}, Slopes for all features: {multiple_lr.coef_}")

# Check if the coefficient for the first feature in both models are close
assert np.isclose(simple_lr.coef_[0], multiple_lr.coef_[0], rtol=1e-1), "The coefficient for the first feature from both models should be close."
$$$$
`