# Pre-Trained Model for Occupational Coding

I've recently embarked on a task to classify an occupation code, given a job title & description. I have come across clustering, a method of grouping data into clusters of which were not previously assigned. As well as, classification to group new data into pre-defined categories.

The only problem is that most of the classification methods I've seen used are for a limited number of categories (usually 2, as high as 30), and the number of occupation codes reaches into the several hundred.

Is there any resources someone can point me to, or an idea of a pre-trained model to use as a foundation? Any insights into this problem (if it's even possible) would be greatly appreciated. I currently have 2M+ human-tagged job postings to work off of. I've read up on the process of freezing a BERT models architectural layers, and feeding the training data into the softmax layer, but I'm hoping someone could let me know if that is just a waste of time.

• play around with $$k$$, the number of clusters (hyper-parameter) in $$k$$-means clustering, and see how it works.