Are these steps to get a final linear regression model correct?

Question

I am new to machine learning. I know Logistic Regression (LR) is a supervised learning technique. Therefore, we need training data to train the model.

I tried to understand the basic steps to get the final RL model. According to my understanding, here are the steps.

1. We define the LR model, that is, $y = \text{sigmoid}(W x + B)$. Set $W$ and $B$ to zero or another value.

2. Given the training data (the inputs are $x_1, x_2, \dots, x_m$, and the outputs are $y_1, y_2, \dots, y_m$), we find $W$ and $B$ values by minimizing a cost function using gradient descent.

3. Then we use the found $W$ and $B$ values. We then again apply a known sample from the training data $\hat{x}$ to get the predicting of $\hat{y}$, that is, $\hat{y} = \text{sigmoid}(W \hat{x} + B)$.

4. We test the final model on unknown data.

Are these steps correct?

Please, if you could also give me the basic idea behind the supervised technique, I would appreciate it.

Answer 1

In general these steps are correct, but are few clarifications would be good. For supervised learning, you can:

1. pick model to use
2. split data into training and test set
3. train model, or determine weights by optimization of our error function (inputting in training data)
4. use trained model to predict on test data
5. utilize metrics of interest to evaluate our model (e.g. MSE for continuous outcomes, AUC for binary)

This however is a very elementary pipeline to train a supervised machine learning model. There are many other steps taken to ensure our model is as robust as possible (like cross validation, feature selection, hyperparameter optimization, model selection...). I would strongly recommend learning about these other steps and seeing how they integrate with the above pipeline.

---

For unsupervised learning, the process depends a lot on the task, but in general it involves optimizing some objective function using the data provided. This is usually done using all the data in one go (not split into training and test) unlike supervised learning. This can be seen more clearly with examples, take methods like PCA and k clustering; the steps taken to apply these methods very much depend on the method used.

---

Going further, I would recommend Andrew Ng's coursera course on machine learning. It is very accessible and a great introduction to this field, covering both unsupervised/supervised learning.