5
$\begingroup$

I am very beginner to this world. I still learning the basics of Machine learning and AI but i have a problem at hand and i am not sure which technique or Algorithm can be applied on it.

I am working on Click-Fraud detection in advertising. I need to predict fraud and learn new frauds with ML.

The dataset I have is the view and click logs from adserver(Service Provider). This data have some fields few of them are listed below:

"auction_log_bid_id": null, 
"banner": 9407521, 
"browser": 0, 
"campaign": 2981976, 
"city": 94965, 
"clickword": null, 
"content_unit": 4335438, 
"country": 1, 
"external_profiledata": {}, 
"external_user_id": null, 
"flash_version": null, 
"id": 6665230893362053181, 
"ip_address": "80.187.103.98", 
"is_ssl": true, 
"keyword": "string"
"mobile_device": -1, 
"mobile_device_class": -1, 
"network": 268, 
"new_user_id": 6665230893362118717, 
"operating_system": 14, 
"profile_data": {}, 
"referrer": null, 
"screen_resolution": null, 
"server_id": 61, 
"state": 7, 
"target_url": "string"
"timestamp": 1551870000, 
"type": "CLICK_COMMAND", 
"user_agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16D57", 
"user_id": null, 
"view_log_id": null

There are other fields.

I need to analyse these logs to find patterns for possible frauds but i am not sure where to start and which technique to use. e.g. Supervised, Unsupervised Semi-Supervised or Reinforcement Learning.

Improve this question
$\endgroup$
5
  • $\begingroup$ what is "Click-Fraud detection"? could you give more explanation about your features and your case? $\endgroup$
    – malioboro
    Commented Mar 28, 2019 at 22:57
  • $\begingroup$ @malioboro click fraud is a deliberate clicks either by human or bot to drain advertiser's advertising budget! $\endgroup$
    – Mirza
    Commented Mar 29, 2019 at 8:05
  • $\begingroup$ Does your dataset definitely contain fraudulent examples? $\endgroup$
    – DrMcCleod
    Commented Mar 29, 2019 at 16:43
  • $\begingroup$ @DrMcCleod actually we dont know, but i am sure there is some perecentage of data is fraudulent. P.S. Our data is live dataset not older that 2 days! (4TB for 2 days in the form of log files) $\endgroup$
    – Mirza
    Commented Apr 1, 2019 at 8:10
  • $\begingroup$ @Mirza Do you know of any indicators of fraud? What fields are reliably provided? If every click has an accurate timestamp, then perhaps you could group your clicks by their timestamp? For example, if there's a relatively huge number of clicks within a relatively short time frame, then it might indicate fake clicks. $\endgroup$
    – sfmiller940
    Commented Apr 3, 2019 at 20:47

2 Answers 2

Reset to default
1
$\begingroup$

I could give you my $0.02 on fraud detection.

  1. Read everything you can on the Equifax breach and seek to secure your data
  2. Benfords Law would be a good place to start
  3. If you can isolate log activity that is inhumanly consistant, if you "ip_address", "id" and/or "timestamp" all show a constant 3 second gap between activity or its always a random choice between 3 and 6 seconds between them.

If you plan on investing the time and resources that ML or Ai require, you will need to isolate "good data" as training data and train your model on that. Perhaps you could get the IP address of your known top 10 customers and include that.

Then begin training with that as your sample data, with keeping your test data seperate.

I'm sure there is alot more but I would need to know context of the info provided in terms of what kind of fraud you are looking for, what you have tried or why they think there is fraud to be found.

Improve this answer
$\endgroup$
1
$\begingroup$

There are a couple of different ways you can go about this depending on what kind of data you have.

If you have labels or can separate the normal data from the fraudulent, you can perform either binary classification, or likely more usefully, anomaly detection.

In Anomaly Detection(which is typically now done via autoencoder) you train your model on the normal data, so it learns a compressed representation of that 'signal', from there it will be able to detect any sample that does not fit the learned representation(in theory).

Here is a link to a tutorial in keras: link

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks. I get what you mean but in my case data is not labelled and the variables are not only numeric . I have integer, bool, strings etc variables. Since my data isnt labelled i am more looking into clustering more specifically K-Means. let me know if you know something in this scenario! thanks! $\endgroup$
    – Mirza
    Commented Mar 29, 2019 at 13:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .