I want to train a model based on millions of fields, including text and number, that are stored in a SQL database and recommend a perfect match based on some inputs. Now, which algorithm is the best for this problem?

For instance, consider this database pattern:

Title Content Volume Count
First row1 5.36 34
Second row2 36.1 239
... ... ... ...

The first step

You need to decide if you want to hold each string column or not. Then you must encode your text fields into numbers which you need to use some embedding algorithms like word2Vec. Check here.

Second step

Probably, you will have a lot of columns. Now, you need to reduce the dimension space. PCA, manifold transforms, partial least squares regression, etc., may help you in this way.

Third step

Here, you will have nice and tidy tabular data which you can feed in any Recommender System you want to.

  1. I presumed that you mean millions of columns when you say "millions of fields".

  2. If you mean millions of rows, then you probably need to use methods that can deal with Big Data.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.