These challenges, which threaten the efficacy of a particular estimation method are referred to in causal inference literature as threats to external validity. While propensity score matching is a powerful way to control for confounding variables in order to calculate an unbiased estimate of a causal effect, there are a variety of challenges an analyst must be cautious of, as they may diminish the accuracy of their estimate. Many of the challenges I will discuss in this post are common across a variety of causal inference techniques so analyzing these vulnerabilities for propensity score matching, a relatively simple procedure can help an analyst build an intuition of what to look out for when evaluating causal inference techniques.Īs is common with many causal inference techniques, an analyst must be cautious when estimating a causal effect using propensity score matching. In this post, I will describe a set of challenges an analyst should be wary of when attempting to estimate a particular causal effect with propensity score matching. Specifically, I provided a simple procedure for estimating propensity scores, matching individuals based on this estimation, and calculating a measurement of a causal effect by comparing observed individuals within the same match. In that post, I described a scenario in which a marketer may struggle to identify the causal effect of a particular campaign, and discussed a rigorous causal inference technique built to solve this problem. In this post I present the obstacles we may face when leveraging these models as well as the 'adjustments' we can make to remove them.Īs discussed in my previous blog post, propensity score matching is a powerful technique for reducing a set of confounding variables to a single propensity score, so an analyst can easily eliminate all confounding bias. So far, our discussions of causality have been rather straightforward: we've defined models for describing the world and analyzed their implications.

Propensity score matching using xlstat