東京大学政策評価研究教育センター

CREPEDP-121

Number CREPEDP-121
Publication Date May 2022
Title Still Biased? A Remaining Classical Selection Problem of RCTs in Education
Author Hikaru Kawarazaki, Minhaj Mahmud, Yasuyuki Sawada, Mai Seki, and Kazuma Takakura
Abstract In the already very rich and crowded literature on education interventions, the use of test scores to capture students’ cognitive abilities has been the norm when measuring the impact. We show that even in randomized controlled trials (RCTs), estimated treatment effects on the true latent abilities can still be biased towards zero, because test scores are often censored outside of zero and full marks. This paper employs sui generis data from a field experiment in Bangladesh as well as data sets from existing highly-cited studies in developing countries to illustrate theoretically and empirically that this remaining classical sample selection problem exists. We suggest three concrete ways to correct such bias: First, to employ the conventional sample selection correction methods; second, to use tests that are designed with an extensive set of questions from easy to challenging levels which allow students to answer the maximum they could; and third, to incorporate each student’s completion time in estimation.
Other information Paper in English (32 pages)