5
votes
Accepted
How to refine K-means clustering on a data set?
The usual parameters to adjust in a k-means:
Number of clusters (recall many clusters can have same label).
Distance definition (euclidean is the most basic, Gauss is an
improvement)
Selection of ...
- 1,251
5
votes
What is graph clustering?
In graph clustering, we want to cluster the nodes of a given graph, such that nodes in the same cluster are highly connected (by edges) and nodes in different clusters are poorly or not connected at ...
- 33.8k
3
votes
Accepted
How to compute the number of centroids for K-means clustering algorithm given minimal distance?
Yes, the silhouette method (which is implemented in sklearn as silhouette_score) is commonly used to assess the quality of ...
- 33.8k
3
votes
Accepted
Is unsupervised learning a branch of AI?
There is a problem with confining Artificial Intelligence to a single definition, because it has become an umbrella term encompassing many fields of science. It has come a long way from the "thinking ...
- 3,093
3
votes
How to define machine learning to cover clustering, classification, and regression?
I report three definitions of machine learning (ML) and I also explain that ML can be divided into multiple sub-tasks or sub-categories in this answer. However, it may not always be clear why ...
- 33.8k
2
votes
How can I cluster this data frame with several features and observations?
A typical clustering algorithm is k-means (and not k-NN, i.e. k-nearest neighbours, which is primarily used for classification). There are other clustering algorithms, such as hierarchical clustering ...
- 33.8k
2
votes
How to detect patterns in salary distribution if we are suspecting malicious distribution based on employee's region?
A simple initial approach would be to separate it by position and check for each:
Use linear regression: $\hat{salary} = \sum_i \alpha_i * \hat{region}_i + \sum_k \beta_k * \mathbf{1}[\hat{gender}=k]$...
- 2,249
2
votes
Is this dataset with only two features suitable for clustering with k-means?
One problem with clustering algorithms is that they will typically find you a solution, ie they will split your data set into clusters, but it will find you a structure even if there isn't one. Your ...
- 5,062
2
votes
Could clustering be used to parse pdf documents to get headings and titles?
Yes, you could use clustering: Encode your features as a feature vector and feed it into a clustering algorithm (see Finding Groups in Data for a comprehensive description of these). You could use ...
- 5,062
2
votes
Which metric should I use to assess the quality of the clusters?
You can compute Silhouette Coefficient for your aim. Its values mean:
1: Means clusters are well apart from each other and clearly distinguished.
0: Means clusters are indifferent, or we can say that ...
- 1,663
2
votes
Accepted
What happens if all the features are correlated with each other before clustering?
Essentially, yes. One feature predicts to a reasonably high degree what the other features look like, so the additional features have limited discriminatory power. Obviously there is some effect, as ...
- 5,062
2
votes
Accepted
How to tackle the Human Error made in labeling datasets for Classifcation Tasks like Facial Expression Recognition?
In general the only way to deal with this is by quantifying these labeling mistakes in the output of the model, since the model will learn for them. And in many cases these are not really mistakes, ...
- 566
1
vote
Accepted
What clustering algorithms work best for datasets with only binary categorical features?
Any clustering algorithm should work -- the main issue is the similarity or distance metric that determines how similar (or different) two elements are. This is often something like Euclidean distance,...
- 5,062
1
vote
How to handle list features in clustering?
You essentially want to have a numerical value to represent the similarity of the lists of two distinct objects. There are a number of metrics to deal with that, eg the Jaccard Index or Dice's ...
- 5,062
1
vote
Is there a clustering algorithm that can make n clusters and the n+1 "others" cluster?
So, I've prepared some data that resembles your sketch:
...
- 1,793
1
vote
Which metric should I use to assess the quality of the clusters?
One more popular metric for this is the Davies Bouldin Score.
You can also take a look at the clustering metrics in scikit documentation.
- 763
1
vote
Accepted
What exactly is the eigenspace of a graph (in spectral clustering)?
In spectral clustering we not find the eigenvectors of a graph (a graph is not a matrix) but the eigenvalues/eigenvectors of the Laplacian matrix related to the adjacency matrix of the graph:
graph =&...
- 1,251
1
vote
How can I cluster this data frame with several features and observations?
Yes you can use KNN algorithm to cluster (well actually its a classification not a clustering if you use KNN) the data. But, first you need to set one feature as a label because KNN is a supervised ...
1
vote
What is the role of the 'fuzzifier' w in Fuzzy Clustering?
Its not required, you can have $m=1$, actually it can be any number $\geq 1$.
Now the better question is why to have it? The answer is that it adds a smoothing effect. Lets look at it in each of ...
- 2,249
1
vote
How to compute the number of centroids for K-means clustering algorithm given minimal distance?
If you look at Kaufman & Rousseeuw (1990), Finding Groups in Data, they describe an algorithm to evaluate the quality of clusters in agglomerative clustering. You run the clustering algorithm with ...
- 5,062
1
vote
What techniques to explore for dynamic clustering of documents (emails)?
It sounds like you are trying to do some kind of semi-supervised learning. In semi-supervised learning, some data points are labelled (you know which class they belong to), and others are not. There ...
- 8,867
Only top scored, non community-wiki answers of a minimum length are eligible