Stochastic potential outcomes: A discussion with Ang Yu and Jonas Mikhaeil

Ang Yu read the Russian Roulette paper (by Jonas Mikhaeil and me) and wrote:

I was wondering whether it might be useful to alternatively frame the problem in terms of stochastic versus deterministic interventions as opposed to potential outcomes. I have attached a short note on your paper and would appreciate your thoughts.

Jonas replied:

1. If I [Jonas] understand correctly, the interventional framework is meant to avoid counterfactuals altogether (e.g. Didelez, et al., 2012), right? In this case, one might be able to avoid the problems with “counterfactual decision problems” altogether by rephrasing them as classical decision problems under interventions?
Maybe this is exactly what you have in mind. I think this would be a very interesting conceptual contribution. 2. What I like about the stochastic potential outcomes perspective is that it allows us to model which uncertainty/variation we want to incorporate in the definition of the treatment (bullet vs. pistol). By defining the utility as a functional over the distribution of the stochastic potential outcome, we can integrate out the uncertainty that remains after treatment assignment. With this additional model choice, we can choose which uncertainty is irrelevant (and hence integrated out prior) to our decision problem. 3. I’m not entirely sure I understand how this would be accomplished by switching from deterministic to stochastic interventions. I’m sure this is an oversight on my part, and I would be excited to see this perspective be fleshed out.
My immediate question would be the following: potential outcomes are defined on the level of the individual, so it makes sense to distinguish between averaging over the treatment vs. averaging over an individual’s outcome. Accordingly, we can define the utility by only averaging over an individual’s outcome, fixing a treatment, and then consider the expected utility by averaging over treatment assignment.
In the interventional framework, I’m not sure how one would distinguish between the outcome Y of an individual being deterministic vs. stochastic for a given intervention? Because of this, I’m not clear how one would first integrate out individual-level uncertainty to define the utility and then calculate the expected utility based on treatment assignment in the interventional framework. (i.e. can what you call U(R6) be understood as an average over an individual’s outcome given a treatment?) I’m sure this is my oversight. I would love to hear what you think. 4. Following up on point 3: Does the Russian roulette paradox appear/disappear when switching from a deterministic vs. stochastic interventions perspective? Essentially, I’m wondering if the choice of how to model the intervention (deterministic vs. stochastic) allows to include/integrate out different levels of uncertainty in the decision problem.

Ang responded:

1. I think so. Didelez et al. 2012 was one of the earliest works in the stochastic intervention literature. Terminology and notation evolved over time in the literature, but their “regime” is the same as what I call “intervention”, and their distinction between atomic and random interventions is the same as my distinction between deterministic and stochastic interventions. 2. I agree that being able to integrate out sort of the “inner” uncertainty is important. If I understand your point correctly, my sense is simply that we already have an existing framework that can do exactly that: we just need to rephrase your point. In the stochastic intervention framework, intervention is pistol and treatment is bullet. So we can integrate over treatments and get the utility of an intervention for each individual. 3. I think the content of the stochastic intervention and stochastic potential outcomes frameworks is exactly the same, just different labels. In my mind, Y_{i, R6} for individual i is a random variable because of the randomness in the intervention R6. So we can call E[Y_{i, R6}] an average over the randomness of the potential outcomes or the randomness of the intervention. The difference is in naming, not substance. Does that make sense? 4. I think I’m not inventing a new way to solve to paradox, but just pointing out the connection between your solution and the literature on stochastic interventions.

My quick thought on all of this is that you can have stochasticism in the intervention or in the outcome conditional on the intervention. One way to see this is in the Russian roulette example: conditional on the chamber of the bullet, you can consider the intervention to be stochastic, but unconditional on the chamber, you can consider the outcome to be stochastic.

Or here’s an example with both sources of variation (or, you could say, “stochastic” aspects): Consider a treatment that is some sort of naturally growing herb that is gathered in the wild, so not a controlled compound. The intervention is stochastic because you might happen to get a sample of the herb that has a large amount of the active ingredient or you might get a dud that has very little. But the outcome will also be stochastic because the active ingredient will not always work. Actually, we could add a third level of variation by saying that (a) the active ingredient could work for some people and not others, and (b) even for the people for whom the ingredient works, it won’t always be effective.

For another example, consider physical therapy as a treatment for sciatica:

(i) To start with, there are lots of way that physical therapy is done; different physical therapists do different things. So the intervention of “physical therapy” is stochastic, if you don’t define ahead of time what will be done and who will be doing it.

(ii) For any given physical therapy intervention, there will be some people for whom it has no chance of working, because of the condition they have. This could be considered as stochastic, but I’d prefer to think of it as deterministic, conditional on a latent variable which contains details of the patient’s underlying medical condition.

(iii) If a patient is given a therapy that can work, the actual effectiveness will be stochastic. You could go full Rubin and consider the outcome as deterministic, conditional on a latent variable which contains the potential outcomes, but I think there can be a benefit to using a stochastic model, recognizing that there’s randomness in the course of the treatment.

For an even cleaner example, consider a coaching program to improve people’s basketball skills, where the outcome is a test where they take 100 shots and you count how many go in:

(i) Some coaches are better than others. So it makes sense to think of the intervention as stochastic.

(ii) Some people will be more amenable to coaching than others. For example, some people will be motivated by the coaching to practice; others won’t. I prefer to model this with a latent variable representing differences between people, but even then there can be a stochastic element: for example, maybe I’m the sort of person who would be motivated to practice, but I just happen to be very busy during the month of the coaching. You could code my state of busyness as a latent variable, but there could be all sorts of unmeasured or even unmeasurable factors affecting my receptiveness during that month, and at some point it makes more sense to bite the bullet and call it stochastic, in the same way that we consider a die roll to be stochastic even though in theory we could predict it pretty well given enough information on initial conditions.

(iii) Any particular coaching will work better for some people than others. There are different ways to have problems with your basketball shot, and different people will be able to improve in different ways. This could be considered as an interaction between the flavor of treatment and the player in the experiment, and it could be modeled with sufficient pre-treatment information on the players, but in practice it could make sense to model much or even all of this variation as stochastic.

(iiii) Finally, how many shots go in, conditional on the average probability of success? That’s stochastic. Again, you could follow Rubin (1974) and consider the outcome of the test to be deterministic, under the philosophy that “the great book of life has already been written; we just haven’t read it yet”; but to me it makes much more sense to consider the success probabilities or expected outcomes as fixed but unknown, with the actual outcomes as stochastic.

I’ve written before about the border between conditioning and randomness–it comes up in non-causal settings too. See here: https://sites.stat.columbia.edu/gelman/research/published/augie4.pdf and here: https://statmodeling.stat.columbia.edu/2022/12/23/a-probability-isnt-just-a-number-its-part-of-a-network-of-conditional-statements/

To which Ang responded:

I appreciate you laying out more generally different possible sources of randomness. On a philosophical level, they all make sense. From a practical perspective, though, I wonder if it’s useful to fully parse everything out. Stochastic interventions have proved useful in a lot of epidemiological and sociological applications. And I think the Russian roulette can also be framed as a stochastic intervention. In all those cases, it seems sufficient to explicitly consider only two sources of variation: the random variation in the intervention (e.g., bullet or no bullet), and the variation in potential outcomes across individuals. Please feel free to blog our discussion! On a side note, I recently read VanderWeele and Hernan (2013), which framed a variant of stochastic intervention as a case of treatments with multiple versions. The stochastic intervention considered there also directly comes from a real empirical application.

I actually think we’re in agreement here. I too think it can be useful to allow stochastic potential outcomes. Indeed, I think the idea of the “stable unit treatment value assumption” (SUTVA) can be misleading, in that SUTVA requires that outcomes be modeled deterministically. In my above note, I said that any stochastic potential outcome can be expressed in reduced form as a deterministic potential outcome–but for modeling purposes, the stochastic model often makes more sense. So I guess there was something unclear in my paragraphs just above, in that I wasn’t doing a good job in conveying this perspective.