I am very beginner to this world. I still learning the basics of Machine learning and AI but i have a problem at hand and i am not sure which technique or Algorithm can be applied on it.

I am working on Click-Fraud detection in advertising. I need to predict fraud and learn new frauds with ML.

The dataset I have is the view and click logs from adserver(Service Provider). This data have some fields few of them are listed below:

"auction_log_bid_id": null, 
"banner": 9407521, 
"browser": 0, 
"campaign": 2981976, 
"city": 94965, 
"clickword": null, 
"content_unit": 4335438, 
"country": 1, 
"external_profiledata": {}, 
"external_user_id": null, 
"flash_version": null, 
"id": 6665230893362053181, 
"ip_address": "", 
"is_ssl": true, 
"keyword": "string"
"mobile_device": -1, 
"mobile_device_class": -1, 
"network": 268, 
"new_user_id": 6665230893362118717, 
"operating_system": 14, 
"profile_data": {}, 
"referrer": null, 
"screen_resolution": null, 
"server_id": 61, 
"state": 7, 
"target_url": "string"
"timestamp": 1551870000, 
"type": "CLICK_COMMAND", 
"user_agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16D57", 
"user_id": null, 
"view_log_id": null

There are other fields.

I need to analyse these logs to find patterns for possible frauds but i am not sure where to start and which technique to use. e.g. Supervised, Unsupervised Semi-Supervised or Reinforcement Learning.

| improve this question | | | | |
  • $\begingroup$ what is "Click-Fraud detection"? could you give more explanation about your features and your case? $\endgroup$ – malioboro Mar 28 '19 at 22:57
  • $\begingroup$ @malioboro click fraud is a deliberate clicks either by human or bot to drain advertiser's advertising budget! $\endgroup$ – MGM Mar 29 '19 at 8:05
  • $\begingroup$ Does your dataset definitely contain fraudulent examples? $\endgroup$ – DrMcCleod Mar 29 '19 at 16:43
  • $\begingroup$ @DrMcCleod actually we dont know, but i am sure there is some perecentage of data is fraudulent. P.S. Our data is live dataset not older that 2 days! (4TB for 2 days in the form of log files) $\endgroup$ – MGM Apr 1 '19 at 8:10
  • $\begingroup$ @Mirza Do you know of any indicators of fraud? What fields are reliably provided? If every click has an accurate timestamp, then perhaps you could group your clicks by their timestamp? For example, if there's a relatively huge number of clicks within a relatively short time frame, then it might indicate fake clicks. $\endgroup$ – sfmiller940 Apr 3 '19 at 20:47

There are a couple of different ways you can go about this depending on what kind of data you have.

If you have labels or can separate the normal data from the fraudulent, you can perform either binary classification, or likely more usefully, anomaly detection.

In Anomaly Detection(which is typically now done via autoencoder) you train your model on the normal data, so it learns a compressed representation of that 'signal', from there it will be able to detect any sample that does not fit the learned representation(in theory).

Here is a link to a tutorial in keras: link

| improve this answer | | | | |
  • $\begingroup$ Thanks. I get what you mean but in my case data is not labelled and the variables are not only numeric . I have integer, bool, strings etc variables. Since my data isnt labelled i am more looking into clustering more specifically K-Means. let me know if you know something in this scenario! thanks! $\endgroup$ – MGM Mar 29 '19 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.