K-means tries to find centroid and then clusters around the centroids. But what if we want to cluster based on the complement?
For example, suppose we have a group of animals and we want to cluster Dogs, Cats, (Not Dogs and Not Cats). The 3rd category will not arise from mean clustering.
Note: K-means does not assume an interpretation/label of the clusterings - in fact it is an unsupervised algorithm. The interpretations are a result of human analysis after running K-means.
For example, in the case of cats and dogs one would most definitely chose k = 2 - which provides an easy interpretation. However, what would it mean if we set k = 1000. We no longer have a "clean" interpretation of the centroids.
Note: how I keep saying "interpretation." The algorithm simply assigns a data point to a cluster and calls it a day. Humans then look at the results and try to understand them with an interpretation.
Continuing with the example where k = 2. One could easily interpret "is cat" as "not dog" and "is dog" as "not cat." The idea here is that the data is unlabeled beforehand and humans try to fathom the results retrospectively by assigning the resulting clusters with an understandable label.
I hope this clarifies the issue.
The purpose of clustering is to look for the same characteristics in the data and group them into clusters. The number of clusters we take is based on how our clustering algorithm evaluated.
For example, we use the Elbow Method for evaluates. We take optimal number of clusters that distortion start decreasing in a linear fashion. In the picture below, the optimal number of clusters for the data is 3.
