0
$\begingroup$

I wonder if following characteristic of data has some standard "professional" or scientific term associated with it.

Let's assume that I have a set of dog/cat images labeled 0 for a cat and 1 for a dog. My purpose is to extract (classify) as many true dog images as possible, with the smallest amount of non-dog images. But the problem with data is that:

  • some dogs images are labeled as 0
  • some cats images are labeled as 1
  • some images are of other animals or non-animal objects and can have both labels 1 and 0

Is there any specific term describing such cases, or they are just "noisy" data?

$\endgroup$

1 Answer 1

0
$\begingroup$

This is referred to as label noise (or noisy labels), and it is indeed different from just "noisy data"; quoting from the 2014 paper Classification in the Presence of Label Noise: a Survey:

In this survey, it is assumed that each training sample is associated with an observed label. This label often corresponds to the true class of the sample, but it may be subjected to a noise process before being presented to the learning algorithm. It is therefore important to distinguish the true class of an instance from its observed label. The process which pollutes labels is called label noise and must be separated from feature (or attribute) noise which affects the value of features.

You may also find the (preprint) paper Label Noise Types and Their Effects on Deep Learning useful; there, the authors propose a taxonomy of different types of label noise:

Label noise types can be subdivided into three main groups as follows.

Uniform noise: Flipping probability of label from its true class to any other class is equally distributed. Many works in literature use synthetic uniform label noise by just flipping labels randomly for a given percentage of data instances [21]–[24].

Class-dependent noise: Flipping probability of label depends on the true class of the data instance. This is mostly represented by a confusion matrix and can be designed in different ways. The easiest way is to attain inter-class transition probabilities just random [25], so that there is still class dependence since transition probabilities are given according to classes but without any correlation to class similarities. In a more structured way, noise transition matrix can be designed in a way that similar classes have a bigger probability to be flipped to each other [26]–[30]. Some works use pairwise noise, in which transition from one class can only be defined to one another class [31]–[35]. Work of [36] checks the popularity of classes and constructs transition matrix so that mislabeling happens from popular class to unpopular class or vice versa.

Feature-dependent noise: The probability of mislabeling depends on features of instances. In order to generate feature-dependent noise, features of each instance should be extracted, and their similarities to other instances from different classes should be evaluated. Unlike uniform and class-dependent noise, there are much fewer implemen- tations of synthetic feature-dependent label noise. One particular work in this field is [37], where data is clustered with the kNN algorithm, and labels are flipped randomly for clusters of data. This method provides concentrated noise in the feature space. But, this type of synthetic noise doesn’t evaluate the instance similarities and therefore different from our proposed approach. Alternatively, in case there is a surrounding text for each image in the dataset, some works create noisy labels from the interpretations of these texts [38]–[41], assuming surrounding texts are related to features of data. But this approach is restricted to datasets with surrounding user-defined texts, which is not the case for most of the time.

Other possibly useful resources on the subject:

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .