I have a use-case where we need a classifier to take decisions in real time, meaning that as data arrives, we need to decide to which category that data belongs and it has to be done fast. The better the classifier's accuracy, the slower it seems to be. Is there a library or type of model you recommend for this sort of problem? I.e. a time constrained classification.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
If you have tabular data (vectors of features) you can opt for a logistic regression which is quite cheap to compute, or a shallow decision tree. Moreover, decision trees can be further pruned to reduce their complexity.
Otherwise, if your data is more complex (e.g., images, audio, etc) you probably need a neural network. If so you can optimize, i.e., compress or accelerate the model via weight or activation quantization (e.g., to fp16 or int8 precision) - there are libraries to do this, like tensorflow-lite, DeepSpeed, PyTorch built's in methods, and OpenVINO to name a few.
There are also more sophisticated optimizations that are hardware related, i.e., you can optimize your neural net for CPU with Intel's specific stuff (now I don't remember how it is called), or use CUDA for Nvidia GPU acceleration. Things can be pushed even further by deploying your ML model on a FPGA accelerator: one solution is through HSL4ML library.
-
$\begingroup$ Hello @Luca Anzalone. Thank you for your comment. Yes, we tried shallow decision trees, but it turns out that the performance degrades significantly when you make them shallow, we see an acceptable performance with a depth of 10. We did not try a logistic regression, but we have never used that. You mean some sort of hyperplane to split our categories? That might not perform well though, because yes our data is probably complex. I see, we might try tensorflow with the neural network. Bear in mind though, that our datasets are only about 10K-20K samples and we have around 10-15 features. $\endgroup$– acampoveCommented May 26 at 13:18
-
$\begingroup$ @acampove If so, you can try a SVM classifier maybe some light implementations. Also, you can look at CUDA-enabled xgboost and/or LightGBM that run on GPU. $\endgroup$ Commented May 27 at 12:38
-
$\begingroup$ so the GPU part helps not just in the training but also in the prediction? $\endgroup$– acampoveCommented May 28 at 15:47
-
$\begingroup$ @acampove Well, the LightGBM docs says that GPU speeds up the feature histogram computation during training, whereas, the XGBoost ones state that "Most of the algorithms in XGBoost including training, prediction and evaluation can be accelerated with CUDA-capable GPUs." I think you should benchmark the GPU vs CPU for inference (i.e., prediction) to be 100% sure $\endgroup$ Commented May 29 at 20:26