What algorithm should I use to classify documents?

Question

I'd like to build a program that would learn to automatically classify documents. The principle would be that, for each new document I add to the system, it would automatically infer in which category to classify the document. If it doesn't know, I would have to manually enter the category. For each hint I give to the system, the system would learn to refine its knowledge of document kinds. Something similar to face recognition in Picasa, but for documents.

More specifically, the documents would be invoices, and I want to classify them by vendors. Documents could be extracted as text, as image, or both.

Is there some know algorithms for this kind of job?

Up to now, I could think at two possible ways I could do it:

• For images, I could add all the images of a given kind together, and record the pixels that are the most common to all images, to create a mask. For a new image, I would compare this mask with the image to determine how similar it is.
• For text, I could record the list of words or sentences that are similar to all documents of a given kind.
• Finally, I could do a combination of both techniques, for example by converting a PDF document to an image, or an image to text by OCR techniques.

I'm just wondering if I'm approaching the problem the right way. Especially about storing just enough information in the database.

Answer 1

Text approach:

Use LDA (Latent Dirichlet Allocation). LDA is unsupervised. Feed it in corpuses of text from the various documents (i.e. OCR them and feed LDA the results of OCR). It will then cluster them based on the contents of the text (with or without stop words - at your discretion). If possible, you could do a supervised approach of using a bag-of-words and any classifier such as an SVM or Random Forest.

Image Approach:

Use a CNN (convolutional neural network) and train it on images of the various vendors. If you don't have this class discrimination, and can't get it, then use an unsupervised approach such as an autoencoder and then cluster the points in the lower-dimensional autoencoder feature space.