Basically what I want to do is to create a single vector representation of a list of skills belonging to employees at a company (one list per employee). The embedding will be a representation of an employee's "profile". The motivation behind this is (among other reasons) that I want to be able to identify clusters among the employees.
Assume I already have a trained FastText model (or Word2vec) that can generate good representations of the individual words in the list.
My current solution is simply to add all the word embeddings in an employee's list together (without any form of normalization). But I'm very unsure about whether this is the best approach to generating a good representation of an employee's profile.
The dimensions of the vectors are 300 and there are usually around 10 to 30 skills in a single list.
Any help would be greatly appreciated!
Let's say we have an it-consulting firm where each employee has their own set of skills. Some consultants are more experienced or versatile, thus having more skills listed in their profiles. eg we have:
alex_skills = ['microsoft azure', 'machine learning', 'data science', 'python', 'sklearn', 'xgboost', 'nginx', 'flask', 'SHAP', 'git', 'word2vec', 'statistics', 'deep learning', 'linux','docker compose', 'pandas'] carla_skills = ['devops', 'machine learning', 'deep learning', 'continuous integration', 'kubernetes', 'python','git', 'speech recognition', 'github', 'bitbucket', 'scikit-learn', 'natural language processing', 'pandas'] adam_skills = ['automation', 'robotic process automation', 'banking and finance', 'process mapping', 'IAM', 'väsentlighetsanalys', 'business intelligence', 'auditor', 'requirements handling', 'risk management', 'coordinator', 'project manager', 'data visualization']
As you can see Alex and Carla are more similar and should possibly be in the same cluster, while Adam might not be.
So I wan't to make a vector representation of the entire list of skills. And then I will use these vector representations in some clustering algorithm (eg HDBscan) and by some distance metric (eg. cosine distance), capture the relation between Alex and Carla.
I suspect the fact that the lists have different lengths might cause problems, therefore maybe divide by the length of the list after adding?