### Reading Avg Treatment Effects & Confidence Intervals: Depression in OHE: Causal Inference Bootcamp

[MUSIC] So you’ve done your experiment: you’ve

randomly assigned some people to have the option of enrolling in Medicaid and some people

to be in the control group, and you’ve waited a year, you’ve gathered your data, and now

it’s time to look at the outcome and see what the results are. So here we’ve got all our outcome data of

our experiment. Over here, we’ve got a whole bunch of different outcome variables that

we’re interested in: we’ve got blood pressure, cholesterol, hemoglobin, depression measures

and cardiovascular risk. And if you’re a medical doctor, then all of these numbers, these variables,

have lots of meaning to you and why they matter for health related outcomes. So what we want

to know is: what is the effect of having Medicaid on things like positive depression screen

– so this is a measure of whether you are depressed or not. So does having Medicaid

lower depression, make you less depressed? Well, we’re going to find out very soon. So remember that experiments are relatively

easy to analyze. We just want to compute the average treatment effect, which is the difference

between the average outcomes in the treatment group and the average outcomes in the control

group. So over here in Column 1, they’ve given us

the mean, or average value, in the control group. So for example, all of these are means,

and in the control group 30% of people had a positive depression screen, for example.

Now they haven’t given us the average values in the treatment group, which is an imaginary

column right here, but what they have given us is the average treatment effect, which

is taking the average value in the treatment group minus the average value in the control

group and then that gives us this column here, Column 2. So these numbers here are all differences

between the average value in the treatment group and the control group. So for example, the positive depression screen,

we have an average value of 30 in the control group, 30, and then the difference is -2.21,

so the value in the treatment group is about 28. So only about 28% of people in the treatment

group are depressed whereas its 30% in the control group. So from this number, it looks

like being in the treatment group lowered whether you got depressed or not. So, because this is an experiment, we know

that this number here, this average treatment effect -2.21, is a true causal effect. We don’t have to worry about confounders or

selection problems or anything like that because of random assignment. But we only have a finite

number of people. We have to deal with the fact that we just gathered some data, we didn’t

actually do the experiment on everybody in the state of Oregon or everybody in the whole

country. So we have to worry about finite sample problems,

that means statistical inference in addition to causal inference. That’s exactly what these

numbers here in parenthesis do. These are 95% Confidence Intervals. So let’s talk about how we interpret these.

So here we got the 95% Confidence Interval for the average treatment effect of having

Medicaid on positive depression screen. So here is the interval: -4.02 to -0.40. The

first thing you ask with confidence intervals is: is zero in the interval? No, it’s not.

If zero is not in the interval, you conclude that your estimate of the average treatment

effect,-2.21, is a statistically significant effect. That means that it’s not due to the

particular people who happen to be put into your sample in the experiment, it’s really

a true causal effect. So just because we found that this confidence

interval doesn’t have zero, we conclude that it’s a statistically significant effect, but

that doesn’t mean that it’s a clinically significant effect. So is a -2.2% decrease in depression

something that matters as far as the world is concerned? Well, that is something that

you have to go to a medical doctor to think about. But as far as we are concerned, there is no

problem with small sample size. We’ve been able to detect an effect. Being in the treatment

group here lowered depression by 2%. Now in the next module, we’ll look at another of

these variables where this confidence interval actually has zero, then we have to start thinking

about few different things. [MUSIC].

Hey awesome video. One question, how do you calculate CI for diff (ATE) so that you know does CI contains 0, which will deny any treatment effect? For example ATE diff CI for depression screen is (-4.02,-0.4), how do you calculate those?