Development programs in Bangladesh often claim impact, but how do we know if a program actually worked? Impact evaluation provides rigorous methods to answer this question.
The Fundamental Problem
We want to know: what would have happened without the program? Since we can’t observe the same person both receiving and not receiving treatment, we need strategies to construct valid comparisons.
Randomized Controlled Trials (RCTs)
RCTs randomly assign participants to treatment or control groups. If randomization is done correctly and the sample is large enough, the groups should be similar on average—allowing us to attribute differences to the program.
Examples in Bangladesh:
- BRAC’s Ultra-Poor Graduation Program evaluations
- Innovations for Poverty Action studies on microfinance
- Various education interventions in primary schools
Difference-in-Differences (DiD)
When randomization isn’t possible, DiD compares changes over time between treated and untreated groups. Basic implementation in STATA:
* Load panel data
use "program_evaluation.dta", clear
* Basic DiD
regress outcome treated##post, cluster(district)
* With controls and fixed effects
regress outcome treated##post education age i.district, cluster(district)
* Check parallel trends assumption
gen year_treated = year * treated
twoway (scatter outcome year if treated==0, connect(l)) ///
(scatter outcome year if treated==1, connect(l)), ///
legend(label(1 "Control") label(2 "Treatment"))Propensity Score Matching
When you can’t randomize, matching creates comparable groups based on observable characteristics. Use this when the treated and control groups differ systematically.
Choosing the Right Method
- RCT: When you can randomize and have time to collect baseline data
- DiD: When you have panel data and the parallel trends assumption holds
- PSM: When you have rich baseline data to match on
Each method has assumptions. Be honest about limitations in your analysis.