Economics 430
Treatment Effects and Differences-in- Differences
1
Today’s Class
• Treatment Effects
• Differences-in-Differences
2
Treatment Effects
• Avoid the faulty line of reasoning known as post hoc, ergo propter hoc
– One event’s preceding another does not necessarily make the first the cause of the second
– Another way to say this is embodied in the warning that ‘‘correlation is not the same as causation’’
– Another way to describe the problem we face in this example is to say that data exhibit a selection bias, because some people chose (or self-selected) to go to the hospital and the others did not
• When membership in the treated group is in part determined by choice, then the sample is not a random sample
3
Treatment Effects
• Selection bias is also an issue when asking:
– ‘‘How much does an additional year of education
increase the wages of married women?’’
– ‘‘How much does participation in a job-training program increase wages?’’
– ‘‘How much does a dietary supplement contribute to weight loss?’’
• Selection bias interferes with a straightforward examination of the data, and makes more difficult our efforts to measure a causal effect, or treatment effect
4
Treatment Effects
• We would like to randomly assign items to a treatment group, with others being treated as a control group
– We could then compare the two groups
– The key is a randomized controlled experiment
5
Treatment Effects
• The ability to perform randomized controlled experiments in economics is limited because the subjects are people, and their economic well-being is at stake
6
Treatment Effects
• Define the indicator variable d as:
d = ì1 individual in treatment group
i íî0 individual in control group – The model is then:
y=β+βd+e, i=1,!,N i12ii
– And the regression functions are:
E(y )= ìβ1 +β2 if in treatment group, di
= 1 i íîβ1 if in control group, di = 0
7
The Difference Estimator
• The least squares estimator for β2, the treatment
effect, is:
N
with:
å(d -d)(y -y) ii
b = i=1 = y – y 2N10
å(d -d)
2
y=åN1 y N,y=åN0 y N
i i=1
1
i10
i=1 i=1
i0
– The estimator b2 is called the difference estimator, because it is the difference between the sample means of the treatment and control groups
8
The Difference Estimator
• The difference estimator can be rewritten as: N
å(d -d)(e -e) ii
b =β + i=1 =β +(e -e ) 22N 210
å(d -d)
2
– To be unbiased, we must have:
E(e -e )=E(e)-E(e )=0 1010
i i=1
9
The Difference Estimator
• If we allow individuals to ‘‘self-select’’ into treatment and control groups, then:
E(e )-E(e ) 10
is the selection bias in the estimation of the treatment effect
– We can eliminate the self-selection bias is we randomly assign individuals to treatment and control groups, so that there are no systematic differences between the groups, except for the treatment itself
10
Table 7.6a SEumxmarmy Staptistliecs f:or Regular-Sized Classes Project STAR Kindergarten
Effect of classroom size on student learning
Regular Sized Classroom
11
Table 7.6Eb SxumamarypStalteistics for Small Classes Project STAR Kindergarten
Small Sized Classroom
12
Example
Project STAR Kindergarten
• The model of interest is: TOTALSCORE = β1 + β2 SMALL + e
• Adding TCHEXPER to the base model we obtain:
TOTALSCORE = β + β SMALL + β TCHEXPER + e 123
13
Example
Project STAR Kindergarten
14
Example
Project STAR Kindergarten
• The students in our sample are enrolled in 79 different schools
– One way to account for school effects is to include an indicator variable for each school
– That is, we can introduce 78 new indicators:
SCHOOL _ j = ì1 if student is in school j íî0 otherwise
15
Example
Project STAR Kindergarten
• The model is now:
TOTALSCORE =β +β SMALL +β TCHEXPER +
– The regression function for a student in school j is:
student in regular class student in small class
i12i3i
å j=2
δ SCHOOL_ j +e jii
ï β + δ + β TCHEXPER ì( )
E(TOTALSCOREi)=í 1 j 3 i
î β +δ +β +β TCHEXPER ï( )
1j23 i
79
16
Example
Project STAR Kindergarten
• Another way to check for random assignment is to regress SMALL on these characteristics and check for any significant coefficients, or an overall significant relationship
– If there is random assignment, we should not find any significant relationships
– Because SMALL is an indicator variable, we use the linear probability model
17
The Differences-in-Differences Estimator
• Randomized controlled experiments are rare in economics because they are expensive and involve human subjects
– Natural experiments, also called quasi- experiments, rely on observing real-world conditions that approximate what would happen in a randomized controlled experiment
– Treatment appears as if it were randomly assigned 18
The Differences-in-Differences Estimator
19
The Differences-in-Differences Estimator
• Estimationofthetreatmenteffectisbasedon data averages for the two groups in the two periods:
ˆ(ˆ ˆ)(ˆ ˆ) δ= C-E – B-A
= (yTreatment,After – yControl,After )-(yTreatment,Before – yControl,Before )
–Theestimatorδˆ iscalledadifferences-in-differences (abbreviated as D-in-D, DD, or DID) estimator of the treatment effect.
20
The Differences-in-Differences Estimator
• The sample means are:
yControl,Before = Aˆ = mean for control group before policy yTreatment,Before = Bˆ = mean for treatment group before policy yControl , After = Eˆ = mean for control group after policy yTreatment , After = Cˆ = mean for treatment group after policy
21
The Differences-in-Differences Estimator
• Consider the regression model:
y =β+βTREAT+βAFTER+δ(TREAT ́AFTER)+e
it 1 2 i 3 t i t it
• The regression function is:
ïβ + β E ( y i t ) = ïí 1 2
ìβ1
TREAT = 0, AFTER = 0 [Control before = A] TREAT = 1, AFTER = 0 [Treatment before = B] TREAT = 0, AFTER =1 [Control after = E]
TREAT = 1, AFTER = 1 [Treatment after = C]
β +β ï1 3
β + β î123
+ δ
+ β
22
The Differences-in-Differences Estimator
• Using the points in the figure: δ=(C-E)-(B-A)=é(β +β +β +δ)-(β +β )ù-é(β +β )-β ù
ë123 13ûë121û • Using the least squares estimates, we have:
δ= b+b+b+δ-b+b -éb+b -bù ˆé( ˆ)()ù()
ë123 13ûë121û =(yTreatment,After -yConrol,After)-(yTreatment,Before -yConrol,Before)
23
Example: Minimum wages PA vs. NJ
• On April 1, 1992 minimum wages were increased in NJ from $4.25/hr to $5.05/hr but remained at $4.25/hr in PA.
• Q: What effect did this increase have on full-time employment in fast food restaurants in NJ?
Full-time Equivalent Employees by State and Period
24
The Differences-in-Differences Estimator
• We will test the null and alternative hypotheses:
H0 : δ 3 0 versus H1 : δ < 0
– The differences-in-differences estimate of the change in employment due to the change in the minimum wage is:
δˆ =(FTENJ,After -FTEPA,After )-(FTENJ,Before -FTEPA,Before )
=(21.0274-21.1656)-(20.4394-23.3312) =
2.7536
25
Example: Minimum wages PA vs. NJ
• Rather than compute the differences-in- differences estimate using sample means, it is easier and more general to use the regression format
– The differences-in-differences regression is: FTE =β+βNJ+βD+δ(NJ ́D)+e
it12i3t itit
This is the estimate we need
26
27
Example: Minimum wages PA vs. NJ
• In our differences-in-differences analysis, we did not exploit one very important feature of the data -namely, that the same fast food restaurants were observed on two occasions
– We have ‘‘before’’ and ‘‘after’’ data
– These are called paired data observations, or repeat data observations, or panel data observations
28
Example: Minimum wages PA vs. NJ
• We previously introduced the notion of a panel of data – we observe the same individual-level units over several periods
– Using panel data we can control for unobserved individual-specific characteristics
29
Example: Minimum wages PA vs. NJ
• Let ci denote any unobserved characteristics of individual restaurant i that do not change over time:
FTE =β+βNJ+βD+δ(NJ ́D)+c+e it12i3t itiit
30
Example: Minimum wages PA vs. NJ
• Subtract the observation for t = 1 from that for
t = 2:
-FTE =β +β NJ +β 0+δ(NJ ́0)+c +e
where:
DFTEi =FTEi2 -FTEi1
De=e -e i i2 i1
FTE =β+βNJ+β1+δ(NJ ́1)+c+e i2 1 2 i 3 i i i2
i1 1 2 i 3 i i i1 DFTE =β +δNJ +De
i3ii
31
Example: Minimum wages PA vs. NJ
• Using the differenced data, the regression model of interest becomes:
DFTE =β +δNJ +De i3ii
32
Example: Minimum wages PA vs. NJ
• The estimated model is: DFTE=-2.2833+2.7500NJ R2 =0.0146
(se) (1.036) (1.154)
– The estimate of the treatment effect δˆ = 2.75 using the differenced data, which accounts for any unobserved individual differences, is very close to the differences-in-differences
– We fail to conclude that the minimum wage increase has reduced employment in these New Jersey fast food restaurants
33