5 votes
Accepted

How to refine K-means clustering on a data set?

The usual parameters to adjust in a k-means: Number of clusters (recall many clusters can have same label). Distance definition (euclidean is the most basic, Gauss is an improvement) Selection of ...
user avatar
5 votes

What is graph clustering?

In graph clustering, we want to cluster the nodes of a given graph, such that nodes in the same cluster are highly connected (by edges) and nodes in different clusters are poorly or not connected at ...
user avatar
  • 33.8k
3 votes
Accepted

How to compute the number of centroids for K-means clustering algorithm given minimal distance?

Yes, the silhouette method (which is implemented in sklearn as silhouette_score) is commonly used to assess the quality of ...
user avatar
  • 33.8k
3 votes
Accepted

Is unsupervised learning a branch of AI?

There is a problem with confining Artificial Intelligence to a single definition, because it has become an umbrella term encompassing many fields of science. It has come a long way from the "thinking ...
user avatar
  • 3,093
3 votes

How to define machine learning to cover clustering, classification, and regression?

I report three definitions of machine learning (ML) and I also explain that ML can be divided into multiple sub-tasks or sub-categories in this answer. However, it may not always be clear why ...
user avatar
  • 33.8k
2 votes

How can I cluster this data frame with several features and observations?

A typical clustering algorithm is k-means (and not k-NN, i.e. k-nearest neighbours, which is primarily used for classification). There are other clustering algorithms, such as hierarchical clustering ...
user avatar
  • 33.8k
2 votes

How to detect patterns in salary distribution if we are suspecting malicious distribution based on employee's region?

A simple initial approach would be to separate it by position and check for each: Use linear regression: $\hat{salary} = \sum_i \alpha_i * \hat{region}_i + \sum_k \beta_k * \mathbf{1}[\hat{gender}=k]$...
user avatar
  • 2,249
2 votes

Is this dataset with only two features suitable for clustering with k-means?

One problem with clustering algorithms is that they will typically find you a solution, ie they will split your data set into clusters, but it will find you a structure even if there isn't one. Your ...
user avatar
  • 5,062
2 votes

Could clustering be used to parse pdf documents to get headings and titles?

Yes, you could use clustering: Encode your features as a feature vector and feed it into a clustering algorithm (see Finding Groups in Data for a comprehensive description of these). You could use ...
user avatar
  • 5,062
2 votes

Which metric should I use to assess the quality of the clusters?

You can compute Silhouette Coefficient for your aim. Its values mean: 1: Means clusters are well apart from each other and clearly distinguished. 0: Means clusters are indifferent, or we can say that ...
user avatar
  • 1,663
2 votes
Accepted

What happens if all the features are correlated with each other before clustering?

Essentially, yes. One feature predicts to a reasonably high degree what the other features look like, so the additional features have limited discriminatory power. Obviously there is some effect, as ...
user avatar
  • 5,062
2 votes
Accepted

How to tackle the Human Error made in labeling datasets for Classifcation Tasks like Facial Expression Recognition?

In general the only way to deal with this is by quantifying these labeling mistakes in the output of the model, since the model will learn for them. And in many cases these are not really mistakes, ...
user avatar
1 vote
Accepted

What clustering algorithms work best for datasets with only binary categorical features?

Any clustering algorithm should work -- the main issue is the similarity or distance metric that determines how similar (or different) two elements are. This is often something like Euclidean distance,...
user avatar
  • 5,062
1 vote

How to handle list features in clustering?

You essentially want to have a numerical value to represent the similarity of the lists of two distinct objects. There are a number of metrics to deal with that, eg the Jaccard Index or Dice's ...
user avatar
  • 5,062
1 vote

Is there a clustering algorithm that can make n clusters and the n+1 "others" cluster?

So, I've prepared some data that resembles your sketch: ...
user avatar
  • 1,793
1 vote

Which metric should I use to assess the quality of the clusters?

One more popular metric for this is the Davies Bouldin Score. You can also take a look at the clustering metrics in scikit documentation.
user avatar
1 vote
Accepted

What exactly is the eigenspace of a graph (in spectral clustering)?

In spectral clustering we not find the eigenvectors of a graph (a graph is not a matrix) but the eigenvalues/eigenvectors of the Laplacian matrix related to the adjacency matrix of the graph: graph =&...
user avatar
1 vote

How can I cluster this data frame with several features and observations?

Yes you can use KNN algorithm to cluster (well actually its a classification not a clustering if you use KNN) the data. But, first you need to set one feature as a label because KNN is a supervised ...
user avatar
1 vote

What is the role of the 'fuzzifier' w in Fuzzy Clustering?

Its not required, you can have $m=1$, actually it can be any number $\geq 1$. Now the better question is why to have it? The answer is that it adds a smoothing effect. Lets look at it in each of ...
user avatar
  • 2,249
1 vote

How to compute the number of centroids for K-means clustering algorithm given minimal distance?

If you look at Kaufman & Rousseeuw (1990), Finding Groups in Data, they describe an algorithm to evaluate the quality of clusters in agglomerative clustering. You run the clustering algorithm with ...
user avatar
  • 5,062
1 vote

What techniques to explore for dynamic clustering of documents (emails)?

It sounds like you are trying to do some kind of semi-supervised learning. In semi-supervised learning, some data points are labelled (you know which class they belong to), and others are not. There ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible