0
$\begingroup$

I've recently embarked on a task to classify an occupation code, given a job title & description. I have come across clustering, a method of grouping data into clusters of which were not previously assigned. As well as, classification to group new data into pre-defined categories.

The only problem is that most of the classification methods I've seen used are for a limited number of categories (usually 2, as high as 30), and the number of occupation codes reaches into the several hundred.

Is there any resources someone can point me to, or an idea of a pre-trained model to use as a foundation? Any insights into this problem (if it's even possible) would be greatly appreciated. I currently have 2M+ human-tagged job postings to work off of. I've read up on the process of freezing a BERT models architectural layers, and feeding the training data into the softmax layer, but I'm hoping someone could let me know if that is just a waste of time.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

My initial (naive) approach would be to

  • embed the textual data to some representation, using a pre-trained text embedding network. While BERT indeed works, I'd even start off with a simpler embedding, working my way into more complicated ones. For example - a starting baseline might even be Bag Of Words
  • play around with $k$, the number of clusters (hyper-parameter) in $k$-means clustering, and see how it works.

P.S. If you have tagged postings, you can also try a simple version of supervised learning, but I might not understand what you're referring to by tagged.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I ended up using a sequential model to start, using a GloVe word embedding matrix, and RELU + Softmax layers. I achieved 68% accuracy on my first run of training, and I have a few optimizations to make to get that number higher. I only started on 20k posts (16k train, 4k test) for this accuracy so far. From what I understand, isn't clustering something that would be done before training? or is there a way to utilize that afterwards? I'm relatively new to NLP and thought I've done well so far. I appreciate the advice and will research what you recommended for my optimizations. Thank you ! $\endgroup$
    – Ryley Keegan
    Jan 13 at 22:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .