Understanding a few terms in Andrew Ng's definition of the cost function for linear regression

Question

I have completed week 1 of Andrew Ng's course. I understand that the cost function for linear regression is defined as $J (\theta_0, \theta_1) = 1/2m*\sum (h(x)-y)^2$ and the $h$ is defined as $h(x) = \theta_0 + \theta_1(x)$. But I don't understand what $\theta_0$ and $\theta_1$ represent in the equation. Is someone able to explain this?

Answer 1

Linear regression is always associated with an activation function, the weights between layers and the structure of the network. The weights between layers are $\theta_0$ and $\theta_1$. These weights and the input features undergo the dot product operation, which is then the input to the activation function of the next layer's nodes.

An apparently different but the same use of $\theta_0$ and $\theta_1$ is as coefficients to one or more number of terms which themselves are a combination of the input vectors.

Broadly, $\theta_i$ denotes a weight, i.e. how much preference you want to give to a feature.

Answer 2

As said above, they are weights to your hypothesis function that are changed during training to minimize your Error function. You can think of them like slope and y intercept in basic algebra. However, a linear regression hypothesis function can be parameterized by many more weight terms than just theta_0, and theta_1.

I detail this process more in this post: How does an activation function's derivative measure error rate in a neural network?

Answer 3

The prediction made by linear regression can simply be thought of as a vector dot product.

$$\overrightarrow{x}^T \cdot \overrightarrow{y}$$

One of those two vectors is the "data" for one case (like a row in your data matrix), the other is a vector of the model's parameters, which is usually called $\overrightarrow{\theta}$ or $\overrightarrow{\beta}$.

So in the case shown by yourself, we have:

$$h(x) = \theta_0 + \theta_1 \cdot x$$

Often we add a row of ones to the beginning of the data matrix, that way we are consistent in the sense that the $\theta_0 = 1 \cdot \theta_0$

This way we arrive at:

$$h(x) = \overrightarrow{\theta}^T \cdot \overrightarrow{x}$$