Causal Inference: Understanding the impact of an intervention

Question

I'm trying to create a pipeline for a very common business scenerio. I want to see whats the impact of an intervention on an outcome. For example I want to know if I send a marketing email (intervention) to users, how would it impact their propensity to buy (outcome). To be specific there are 3 things I want to know:

1. What is the average impact of the intervention(marketing email) on the outcome(purchase)- This will tell me if sending emails is generally a good idea or not?
2. Who should we email, and who we shouldn't email (uplift modelling)- This will help me customize the treatment at individual level.
3. How can I be confident in the above mentioned model's findings, without having to run the A/B tests? - This will help me understand if the numbers I've calculated are trustworthy or not.

This is what I've been thinking:

1. What is the average impact of the intervention (marketing email) on the outcome (purchase)
   • Raw conditional probabilities
   • Simple logistic regression, to understand what's the coefficient for the treatment variable
   • Average Treatment Effect (ATE) using CausalML
2. Who should we email, and who we shouldn't email (uplift modelling)
   • Uplift Modeling via Class Variable Transformation
   • Meta Learners (S learners/t learners etc.)
   • Tree based uplift modelling
3. How can I be confident in the above mentioned model's findings, without having to run the A/B tests?
   • AUUC (Area under the uplift curve)
   • Sensitivity Analysis

I'll really appreciate any input to improve this pipeline

Answer 1

I think your assumptions are flawed for step 1. From how I understand it, you have a binary predictor (email no/yes) onto a metric outcome (propensity to buy). However, logistic regression assumes that you have a binary outcome variable, so it is not applicable to your case.

For what you want to do, you need to do a comparison of group means. Since you have one binary predictor, you could do a t-test, ANOVA or linear regression and should always obtain the same results. You could then interpret the coefficient for the treatment variable as you mentioned above, which also gives the benefit that it is directly interpretable as these methods are linear models and not generalized linear models, thus dropping the need for a link function.