2
$\begingroup$

Consider an image classification problem. Conceptually, we then have some high dimensional space where all the images can be represented as points, and having large enough labeled data set we can build a classifier. But how do we know that our data in this space has some structure? Like this one in two-dimensional case:

enter image description here

If we have a data set with images of, say, cats and dogs, why these two classes are not just uniformly mixed with each other but have some distribution or shape in appropriate space? Why it cannot be like this:

enter image description here

Improve this question
$\endgroup$
3
  • $\begingroup$ How do you know that data point are shaped together ? $\endgroup$
    – Jérémy Blain
    Dec 19, 2018 at 14:48
  • $\begingroup$ I don't know at all. I wonder why is it so in some cases. $\endgroup$
    – Denisof
    Dec 19, 2018 at 14:57
  • $\begingroup$ What you are trying to do is create a mapping from the raw image space, where the images are just mixed up randomly, to another space, where the dimensions reflect attributes of the image that allow the images to be clustered in separate groups. Each layer of the network represents a mapping from one space to another; you hope to get a final layer with a useful dimensions, so the whole network together provides the mapping you want. $\endgroup$
    – antlersoft
    Dec 20, 2018 at 19:04

1 Answer 1

Reset to default
1
$\begingroup$

This is the classic question of what structure is or can be. It relates directly to the concepts of generalization, pattern recognition, over-fitting in surface fitting strategies, and learning tabula rasa, Latin for blank slate. The underlying questions are these:

  1. How can it be determined whether the organization of data in a set, which appears to correlate well with model $\mathbb{A}$, isn't just a random data set that merely appears to have $\mathbb{A}$ organization.

  2. How can it be determined whether the organization of data in a set, which appears to correlate well with model $\mathbb{A}$, isn't generated by a phenomenon that generally exhibits organization that correlates well with $\mathbb{B}$?

  3. How can we determine the interrelationship between $\mathbb{A}$ and $\mathbb{B}$, especially since inclusion and overlap relates to the above question?

In the second image in the question, if $\mathbb{A}$ is defined as clustering of a type the smallest boundaries of which are not breached by instances of another type with at least five members per cluster, then several false positives would occur if the locations of the points were generated with quantum or thermal noise and the model was intended to illuminate clustering. In such a case there may be the appearance of clusters, but proximity would not be indicative of organization.

In the first image in the question, if $\mathbb{A}$ is the compound angular projection of the orbital paths of the planets in this solar system, then the image would appear as noise.

If a third image existed of the moon's sky, the same model would likely produce a false negative if the tolerance of inaccuracy in input was high.

It is difficult to reliability detect organization in the measured properties of a complex system. The goal of feature extraction, using RBMs or some other approach is to detect patterns. Whether those patterns are organized or merely apparent is a hypothesis that may be strengthened or weakened as the amount of data analyzed increases.

If an AI system is trained to detect rats but knows nothing of cats and dogs, one cannot assume it will differentiate cats from dogs on the basis of the closer relationship between rodent and cat features than between rodent and dog features unless the AI system was meta-trained or programmed statically with a specific conception of what a feature is.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .