4
$\begingroup$

I am working on an anti-fraud project. In the project, we are trying to predict the fraud user in the out time data set. But the fraud user has a very low ratio, only 3%. We expect a model with a precision more than 15%.

I tried Logistic Regression, GBDT+LR, xgboost. All models are not good enough. Step wise Logistic Regression performs best, which has a precision of 9% with recall rate 6%.

Is there any other models that I can use for this problem or any other advise ?

Improve this question
$\endgroup$
2
  • 4
    $\begingroup$ Welcome to ai.se..This is a special class of problem called anomaly detection problem..Just do a quick Google search Andrew ng Coursera anomaly detection. $\endgroup$
    – user9947
    Apr 19, 2018 at 6:24
  • $\begingroup$ @DuttaA This should probably be upgraded to an answer, since the OP's problem is exactly what Anomaly Detection is for. $\endgroup$
    – DrMcCleod
    Apr 22, 2018 at 8:32

2 Answers 2

Reset to default
1
$\begingroup$

You can balance your data-set.

Many models work with batches of samples. If you have a very unbalanced dataset, you can simply split it and ensure your batches are balanced (for example, for a Neural Network, using minibatches of 32 samples, you could draw 16 from your fraud users, and 16 from non-fraud users).

During the learning phase, this ensures the model doesn't just output the most common class, but instead tries to learn to distinguish both.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Are you sure about that? Because the less the number of samples poorer the generalization $\endgroup$
    – user9947
    Apr 19, 2018 at 19:31
  • $\begingroup$ Ive used it before for a skewed movie ratings dataset without issues. Keep in mind ypu still use the same amount of samples. You just change their relative frequency, such that both classes are equally likely to appear $\endgroup$
    – BlueMoon93
    Apr 19, 2018 at 23:21
0
$\begingroup$

Heavily imbalanced classification tasks do not need a certain type of model, you can get different ones to work.

You have two options: either use class weights (for example setting them to 'balanced' in the ScikitLearn SVM) in order to indicate that samples from a class are more important (the underrepresented one) or rebalance your dataset. For rebalancing purposes, and assuming you are using Python, I recommend Imbalanced Learn. There you have algorithms for over-sampling, under-sampling, over-sampling followed by under-sampling or ensemble sampling. If you use them, please check the plausibility of the synthetic samples you created by reducing dimensionality first and then plotting them for example in two dimensions. Are the synthetic samples similar to the true class?

I would also recommend you to think about relevant metrics for (heavily) imbalanced problems and consider the no-information rate. That is another question though.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I tried under-sampling. It performs quite unstable between each run, some times better and most of the time it performs worse. $\endgroup$
    – BerSerK
    Apr 24, 2018 at 9:45
  • $\begingroup$ What I would do is not try one of the techniques randomly, but reduce your data to 2D and plot it before and after the transformation. Then check for plausibility. Try also for example the Smote-Tomek technique, which combines Over- and Undersampling. If both methods do not work to your satisfaction simply use class weights. $\endgroup$
    – AlexGuevara
    Apr 24, 2018 at 11:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .