I am working on Multiple Linear Regression (Multiple variables). I am been able to predict and get a good r2 score. But I am not sure that I understood the part of plotting the best fit line, I can't understand the visualization of such a model. As I am using 50 startups dataset, which have 3 features and 1 target variable (Profit).

The three features and other dataset info from Kaggle(Advertising dataset) : https://www.kaggle.com/datasets/yasserh/advertising-sales-dataset

I will show some of my main code snippets to better illustrate my model.


df =  pd.read_csv('ad_sales.csv')

# Perform feature scaling
# Separate features and target variable
features = df[['TV Ad Budget ($)', 'Radio Ad Budget ($)', 'Newspaper Ad Budget ($)']]
target = df['Sales ($)']

# Perform feature scaling
scaler = MinMaxScaler()
scaled_features = scaler.fit_transform(features)

# Create a new DataFrame with scaled features
features = pd.DataFrame(scaled_features, columns=features.columns)

features = np.array(features)
labels = np.array(target)
## all features are Min-Max Scaled

model = MultipleLinearRegression()  

model.train(x_train = features, y_train = labels, epochs=150)

My model : 

    def predict(self,x):
        y_pred = [self.b0 + ( (self.b1 * xi[0]) + (self.b2 * xi[1]) + (self.b3 * xi[2]) ) for xi in x]
        return y_pred
    def train(self, x_train, y_train, epochs):
        for epoch in range(epochs):

            y_pred = self.predict(x_train)
            loss = mse(y_pred, y_train)
            self.optimize(x_train, y_pred, y_train, learning_rate=0.001)

        r2 = r2_score(y_train, y_pred)
        print("R2 Score:", r2)

        # # Scatter plot - Target vs Each Feature
        plt.scatter(x_train[:, 0], y_train, color="blue", label="Actual feature 1")
        plt.scatter(x_train[:, 0], y_pred, color="red", label="Predicted feature 1")
        # Line of best fit
        plt.plot(x_train[:, 0], y_pred, color="green", linewidth=2, label="Best fit line")
        plt.xlabel("Feature 1")

        plt.scatter(x_train[:, 1], y_train, color="blue", label="Actual feature 2")
        plt.scatter(x_train[:, 1], y_pred, color="red", label="Predicted feature 2")
        # Line of best fit
        plt.plot(x_train[:, 1], y_pred, color="green", linewidth=2, label="Best fit line")
        plt.xlabel("Feature 2")

        plt.scatter(x_train[:, 2], y_train, color="blue", label="Actual fetaure 3")
        plt.scatter(x_train[:, 2], y_pred, color="red", label="Predicted feature 3")
        # Line of best fit
        plt.plot(x_train[:, 2], y_pred, color="green", linewidth=2, label="Best fit line")
        plt.xlabel("Feature 3")
  ## As it have 3 features, instead of 3d plot, I am plotting one feature with true labels and y_predicted. the plot screenshot is shared in attachment.

enter image description here enter image description here enter image description here

As you can the plot is not showing a straight line, it have a many straight lines combined to match all data points, how ? Isn't the best fit line supposed to be a single line in between the data points . Or it is because of some visualization techniques I used wrong ?

|| Edited : I want to visualize my Features and if possible want to fit a bit fit line, or curve line if in Poly Model. Like in this Figure : enter image description here https://www.megatrend.com/wp-content/uploads/2022/10/2-1.jpg

  • $\begingroup$ Can you upload your 3 feature input raw files? Apparently not even needing to dig deeper into your own optimize() algo details, your pyplot is plotting based on raw (unsorted) input feature slice zipped with their corresponding target labels (noticing the red-only connecting lines). $\endgroup$
    – cinch
    Jul 14, 2023 at 23:11
  • $\begingroup$ Yeah sure, I have edited and added the features link from Kaggle, all those first three are my features ['TV Ad Budget ($)', 'Radio Ad Budget ($)', 'Newspaper Ad Budget ($)'] . @mohottnad $\endgroup$ Jul 15, 2023 at 2:13
  • $\begingroup$ So indeed you are just plotting the raw unordered zipped $(x_i, y)$ tuples, while you seem to want to draw the resultant regression lines for the 3 pairs. Multivariable (homoscedastic) linear regression is a straightforward model with analytics closed form solution for all the coefficients as $\hat{\beta}=(X^TX)^{-1}X^Ty$, or you can use simple gradient descent and even sciki-learn's LinearRegression model. $\endgroup$
    – cinch
    Jul 16, 2023 at 1:26

2 Answers 2

Isn't the best fit line supposed to be a single line in between the data points

Not here. The linear relationship is between prediction y and each feature $x_i$ respectively, while keeping other features constant. So there is no guarantee that the relationship is linear between predictions.

Recall for simple linear regression: $y = ax+b$, so $x$ and $y$ hold a linear relationship.

But for multivariate linear regression: $y = {a_1}{x_1}+{a_2}{x_2}+{a_3}{x_3} + ... + b$

If you think about the relationship between $y$ and each of the $x_i$, it is clear that in general their relationship is linear only when the other $x_j$ are constant, which is usually not the case.

Under the same argument, the outputted predictions $y$ are in general nonlinear too.

  • $\begingroup$ Can you explain a bit further. But I am trying to fit best fit line through features and y_pred, right ? I understand that the regressed points are good and close, but i want to see best fit line in single plot or multiple like i plotted (x_train[:, 0], (x_train[:, 1] ...... like so on. Can you explain further a bit more. $\endgroup$ Jul 8, 2023 at 0:23
  • $\begingroup$ @NiranjanadasM No, the linear plot you want does not make sense. For 2+ variables it is not a line, but a hyperplane. To visually inspect the prediction result, check e.g. the partial residual plot. $\endgroup$
    – lpounng
    Jul 9, 2023 at 5:57
  • $\begingroup$ Guess I have to study more on hyperplane and how does it effects the plotting. plotting more than 1 feature won't get me a straight best fit line against y_pred. I have to look onto it. Thanks tho for the explanation $\endgroup$ Jul 12, 2023 at 7:43

If you want to plot some fitted line, you have to project the hyperplane (your overall fitted model) onto different slices (2D planes), which will indeed produce lines.

To find the equation of the line, you just need to inject the constants values for each feature into the global model $$ y = {a_1}{x_1}+{a_2}{x_2}+{a_3}{x_3} + ... + b $$

For example, if you want to observe the best fitted line on a plot $y$ vs $x_1$, you can choose the slice $x_2=x_3=...=x_n=0$ . This corresponds to the line equation $y=a_1x_1+b$.

Also if you look at a different slice $x_2=x_3=...=x_n=1$, then the new line equation is $y=a_1x_1+b+\sum_{i=2}^na_i$ .

This also shows that the overall slope will not change (here it's still $a_1$ in both examples) for slices corresponding to the same feature but the intercept will be affected by the slice choice.

However you need to be cautious about how you interpret these plots as they only show you a partial view of the full model !

  • $\begingroup$ Hyperplanes works best till 3 or 4 features maybe. i don't know how does this plot and best fit line works when it comes to Multiple variables against y_pred. So , are you saying that for 3 features f = x[:0], x[:1], x[:2], i need to use those slices in this equation = y = mx + b , where x is each x[:0] etc. right ? I have somehow plotted x[:0], y_pred. Iam unable to draw conclusions out of it. $\endgroup$ Jul 12, 2023 at 7:49
  • $\begingroup$ You just cant visualise the full model if you have more than two features. $\endgroup$
    – Lelouch
    Jul 12, 2023 at 10:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .