I'm dealing with a "ticket similarity task".
Every time new tickets arrive at the help desk (customer service), I need to compare them and find out about similar ones.
In this way, once the operator responds to a ticket, at the same time he can solve the others similar to the one solved.
I expect an input ticket and all the other tickets with their similarity in output.
I thought about using DOC2VEC, but it requires training every time a new ticket enters.
What do you recommend?
Not sure if you are bent on using DOC2VEC, but why not use OpenAI embeddings or any Hugging Face model, and store them as a sparse-dense vector in a vector database (NOTE - the concept of sparse-dense was designed by Pinecone) they have publicly made it transparent how to do it yourself).
Using Sparse-dense makes sure your similarities are a blend of semantics as well as lexical which will work perfectly for your case.
You need to create an active learning loop over the process of the learning. Try to start from a history of tickets and using doc2vec to get the similarity. When you find a bad result in the result of your classifier, then report it and then try to retrain the classifier. Also, you can wait to retrain the model, up to finding the predefined batch0size of the new data which are not in the training set.
Also, to get a better result in the active learning loop, you can testify incoming data by the measuring of the classifier uncertainty over it. If the entropy of the classifier over the data is not in a good situation, you can label the data by the operator (as an oracle) and then up reach to the predefined batch-size, retrain the classifier.
Morevoer, to know better about the active learning process and query strategies follow this link (and other articles in that link like this article).
