How do you evaluate a k-medoids cluster model?

Question

So I'm planning on clustering a bunch of observation data using k-medoids. There are seven attributes for each instance and the data is numerical and discrete. I'm a little uncertain of how to evaluate the model to find the correct number of clusters. I was thinking I could run the cluster technique for an increasing number of clusters (say start at 1 and increase by one each time), measure the silhouette coefficient for each cluster model and then select the number of clusters with the highest value?

Would anybody be able to tell me if this is a good idea for evaluating the model and if not what else I could do?

Answer 1

In Finding Groups in Data, Kaufman and Rousseeuw describe ways to evaluate the quality of clusterings. If I remember correctly (it's been some time that I worked with this), for k-means algorithms you add up the average distances of each element to each cluster medoid. This gives you a score for k clusters. Then you repeat the clustering with fewer or more clusters, and do the same calculation. You can then compare the scores and choose the best configuration.

If you have few clusters, the differences will generally be large, but there won't be many of them. If you have too many clusters, there will me more distances to add up, so the score will also be larger. The optimum number of clusters will minimise that score.

PS: I can only highly recommend that book.