Feature Engineering on transactional dataset clustering

Question

I have a data set with transactions details from different business (roughly 1 thousand business entities). Each row is a transaction. The structure of the dataset is as follows:

client_id	Sex	Age	transaction_ammount	business_entity
123	M	88	4829	storeA
123	M	88	1049	storeB
255	F	25	1122	storeH

My goal is to cluster the clients depending on their consuming habits, age and sex.

I am having a hard time on deciding on the best features to feed this dataset into a clustering algorithm (probably K-means as a starter).

Some of the things I am planning to do are:

• One hot encoding on: sex
• Make each store be a column and each row value be the amount of transactions a certain user did pay to that store (for example, if user1 made two transactions to storeB, there will be a 2 in the user1 row on the storeB column).

One of the main things I am struggling with right now is how to sum the transactions' data per user. I would need to run an aggregated operation on them, but don't know which one would be better. Some of the ones I have in my mind:

• Average transaction amount per user
• Min transaction amount per user
• Max transaction amount per user
• The above 3 but for each store per user (which would mean that, if I have 1000 stores, I would have to add 3000 thousand columns). This makes sense since each store have a wide range of product prices and running an operation among all the transactions of a user will be misleading.

What feature engineering technique would you recommend me? Is there any additional data wrangling I should do?

Answer 1

The average transaction is a central measure, while the minimum and maximum transactions together give an idea of dispersion. However, these can be very sensitive to individual purchases that might not be representative of the personal behaviour of a customer.

For instance, the year a consumer changes their fridge or washing machine an outlier purchase happens. You may want to filter that out of your analysis -or not!

Robust alternatives to explore include the median for location and the interquartile range for spread.

A log transformation of the transaction_ammount may be useful as well. This would transform your arithmetic mean into a geometric mean, thus reducing the impact of large purchases.

The table does not display a date column. That can also help modeling. You may have customers buying small quantities on random days and weekend customers going to your stores mainly on Saturday, or even once a month.