NLPES Awards Program

Arizona Wins 1998 NLPES Excellence in Research Methodology Award

1998

AWARD FOR EXCELLENCE IN RESEARCH METHODOLOGY

The NLPES announces that the recipient of the 1998 Award for Excellence in Research Methodology is the State of Arizona's Office of the Auditor General.

The Arizona evaluation, "At-risk Preschool Program," excelled because it effectively used a mixed-methods approach to measure the educational effects of an at-risk preschool program on children. This was a difficult challenge from both a methodological and practical standpoint.

Lacking a single foolproof methodology, the study enhanced its validity by using a quasiexperimental design combined with additional qualitative and quantitative methods. The design used an analysis of pre-test and post-test scores on a nationally recognized standardized test to compare educational effects between experimental and matched comparison groups. The study also used teacher evaluations, interviews with school district administrators and classroom teachers, as well as on-site observation. Using a variety of quantitative and qualitative approaches allowed evaluation staff to see whether there was convergence in the study's findings. The convergence that was identified strengthened the study's conclusions.

Carrying out this study required the Arizona staff to overcome a number of challenges. First, they had to address the challenge of attempting to gain meaningful results by testing the performance of young children. Second, they had to plan the evaluation before the program was fully implemented. Third, they needed to overcome the challenge of conducting a longitudinal study involving students who were highly mobile.