2015 Submission Information and Award Criteria
The NLPES Executive Committee will present this award to an office or offices which have demonstrated outstanding use of evaluation research methods in a report released during calendar year 2014. At the judges’ discretion, up to three submissions may be selected for the award in a given year.
Submissions for the Excellence in Research Methods Award should include a copy of the report and a narrative describing the qualifying characteristics of the report and how the methodology achieves the selection criteria (see below). The narrative should be no more than four pages in length. If the report does not contain a detailed description of the evaluation methodology, you may submit any additional documentation necessary to explain the analysis as an appendix. (Note: Submissions involving contracted work must demonstrate that office staff were involved in all aspects of selecting, developing, and applying the evaluation methodology, and participated in making decisions involving the analysis of data and the findings contained in the evaluation report.)
Qualifying reports should demonstrate one or more of the following characteristics. Please indicate under which criteria (or criterion) you are applying:
1. Exceptional breadth, depth, and scope of fieldwork–successful application of a methodology that, while maintaining efficiency, takes the evaluation to a more comprehensive level.
2. Innovative or unusual applications of established methodologies–successful application of established qualitative or quantitative techniques in new areas or ways.
3. Technical difficulty and sophistication–appropriate and successful use of advanced statistical techniques or software to efficiently and effectively address issues covered in an evaluation.
Please also include a one-paragraph summary of the report and the name, phone number, and e-mail address of a contact person. (If your office receives this award, this information may be used in handouts for the NCSL Legislative Summit and the NLPES Professional Development Seminar.) If you wish, you may include the names, titles and e-mail addresses for no more than five individuals you would like us to notify if you receive the award.
Judges will evaluate each application based on the information presented in the report and the four-page written narrative. Each submission must address how the research method meets the following three criteria. A description of the types of information to be included for each criterion, as well as their relative importance, is also provided.
1. Methodology Usefulness/Design (35%) – The submission should demonstrate first, the selected methodology’s applicability to the evaluation’s objectives; second, the design of the methodology; and third, the usefulness of the methodology, given any data or other limitations encountered. Information provided should:
- clearly state the desired objectives, and
- describe the methodologies considered and why the selected methodology was the most appropriate to meet the objectives.
2. Methodology Application (35%) – The submission should demonstrate how the selected methodology was applied during the fieldwork phase to meet the evaluation’s objectives in the most efficient and effective manner possible. This section should:
- clearly explain methodology procedures, and
- explain how the execution of the selected research method contributed to the efficiency and/or effectiveness of the field work performed.
3. Evaluation Results (30%) – The submission should demonstrate that the selected methodology and the resulting conclusions were appropriate for the evaluation and were clearly described for the reader. This section should:
- describe how the results of the methodology contributed significantly to the development of the evaluation’s findings (problems) and recommendations (solutions), and
- demonstrate that the methodology’s resulting conclusions were appropriately and clearly described for the reader.
Frequency of Submission
Any program evaluation office may apply for the Excellence in Research Methods Award. An office must refrain from applying for this award for one year after receipt of the award.
Information about Judges
Each year, the NLPES Executive Committee selects a panel of three judges to review application materials. These judges will be selected to minimize the potential for conflict of interest or bias. An office that receives an Excellence in Research Methods Award may be requested to make a staff person available to be a judge for the following year. Please mail or email your application to each of the judges by the submission deadline May 8, 2015.
The Excellence In Research Methods judges for 2015 are:
- Jon Courtney (Head Judge)
Legislative Finance Committee
State Capitol North
325 Don Gaspar, Suite 101
Santa Fe, NM 87501
- Russell Moore
Offices of Research and Education Accountability
Comptroller of the Treasury
James K. Polk Building, Suite 1700
505 Deaderick Street
Nashville, TN 37243-0268
- John Sylvia
Performance Evaluation and Research Division
State Capitol Complex Building 1, Room W-314
1900 Kanawha Boulevard East
Charleston, WV 25305-0610