0
$\begingroup$

A learning algorithm is a tuple $(\mathcal{H}, \mathcal{O}, \mathcal{L})$ where $\mathcal{H}$, $\mathcal{O}$ and $\mathcal{L}$ are the hypothesis class, optimizer and loss function respectively.

We can also think of the learning algorithm as the whole pipeline that take us from data to a model.

Suppose we have the following machine learning approach:

$$\text{input, output data} \xrightarrow{\phi} \text{features, output data} \to \text{training a classifier} \to \text{model} $$

where $\phi$ is fixed, i.e. not learnable from the data. One such example, is the SIFT method if we assume that our input is an image. In this case, can we define the learning algorithm as the whole pipeline that take us from input, output data to the model including the feature extraction step as the learning algorithm? In other words, is the feature extraction step consider a part of a learning algorithm?

My view, possibly wrong

I will argue yes, since we consider CNNs as a learning algorithm. The essential difference is that in the CNN case, $\phi$ is not anymore fixed but learnable from the data.

Moreover, what is the hypothesis class of the learning algorithm when fixed feature extraction methods are used?

If $z=\phi(x)$, does the hypothesis class consists of functions:

$$f \colon Z \to Y$$

or:

$$f \colon X \to Y $$

?

$\endgroup$

0

You must log in to answer this question.