3
$\begingroup$

If I do supervised learning the model learns from the labeled input data. This seems to be quite often a small set of human annotated data.

Is it true to say this is the only 'learning' the model does?

It seems like the small data set has a huge influence on the model. Can it be made better using future unlabeled data?

Improve this question
$\endgroup$
4
  • $\begingroup$ Could you please explain what you mean saying that the labeled data is quite often a small set of human annotated data? Do you mean that the inputs are manually labeled? $\endgroup$
    – gvgramazio
    Commented Jul 10, 2018 at 11:41
  • $\begingroup$ Also, without labels, how could you expect to train a NN? You cannot provide a cost function for these data. $\endgroup$
    – gvgramazio
    Commented Jul 10, 2018 at 11:42
  • $\begingroup$ @gvgramazio yes manual labeling of a sample. $\endgroup$
    – schoon
    Commented Jul 11, 2018 at 10:44
  • $\begingroup$ Then if you use supervised learning you cannot use those labeled data. Instead, if you use unsupervised learning you could use unlabeled data. $\endgroup$
    – gvgramazio
    Commented Jul 11, 2018 at 12:49

1 Answer 1

Reset to default
1
$\begingroup$

This may not seem trivial but yes, the models we train can potentially learn a variety of things they weren't intended to learn. There are already some examples in computer vision. A typical convolutional network learns things like edge detection, various potentially useful masks etc. in the early layers while learns more high-level features like eyes, nose etc. in higher layers.

It is reasonable too. Given the dataset size is moderately high and the model is trained for long enough, a sufficiently deep network learns various kinds of hidden representations, which may not even be specific to the task at hand. This is the reason transfer learning works very well even on a host of different datasets.

This is limited since not all the learnable things can be described using mathematics. So, the answer is a surprising no. The model does learn some extra things other than the task at hand.

P.S.: There was also a case when a group of researchers trained a model to make a robot walk. It turned out the robot had learned to recognize faces too and reacted in different ways on seeing different faces. I saw the video on YouTube a while ago and couldn't find the exact video to post the link here, anyways.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Good answer, thanks. But for simple regression, say, there is nothing new unless we label new data, is that right? $\endgroup$
    – schoon
    Commented Jul 10, 2018 at 10:51
  • 1
    $\begingroup$ Yeah @schoon, I can't think of any way a simple regression model can learn something interesting. But, since the structure of data in classification and regression problems is more or less the same, it may still learn something which might not be perceivable by us. This gave me an idea though, we can try to do transfer learning in case of regression tasks and see if it yields any useful results. If it does, we can say that the model has indeed learned something else other than the task at hand! $\endgroup$
    – Suyog Jadhav
    Commented Jul 10, 2018 at 11:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .