 Report on the America’s Legislators Back to School Program Evaluation, Louisiana and Wyoming Phases, September 2008
Summary Report
by Jan Goehring
 Full Report (40-page document) -- To read .pdf files, you must install Adobe Acrobat Reader.
The America’s Legislators Back to School Program was founded in 1999 to educate students about our system of representative democracy. Each year, more than 1,200 state lawmakers visit classrooms all across the country to talk about the legislative process and bring civics to life. The National Conference of State Legislatures (NCSL) provides educational materials about representative democracy for legislators’ use in their presentations to K-12 classes. NCSL also provides training, promotional strategies and organizational support for state coordinators and legislators.
NCSL contracted with Turcotte Public Administration Consulting and Training, LLC (TPACT) to design and implement a controlled study of the program’s impact. NCSL selected the state of Louisiana as the site of the study’s first phase. NCSL subsequently expanded the study to include Wyoming. The purpose of the study was to evaluate the impact of the America’s Legislators Back to School Program on participating students’ understanding and appreciation of representative democracy.
Results show that the Back to School Program strengthened students’ understanding and appreciation of representative democracy.
Method
The evaluators used a randomized pre-test/post-test experimental design in both phases of the study. For each participating school, evaluators randomly assigned participating classes to the intervention or control group. Use of random assignment reduced the likelihood that systematic bias entered into the composition of the intervention and control groups. Legislators visited intervention classes and did not visit control classes during the evaluation.
The objectives listed below in Table A formed the basis of a representative democracy test developed by the evaluators and administered by participating teachers as a pre-test and post-test.
Table A - Back to School Program Objectives
|
Objectives on knowledge of representative democracy--Students will demonstrate their understanding in the following areas:
|
| A. |
Law making process. |
| B. |
New laws begin when citizen groups want changes that will help improve government or meet their needs. |
|
C. |
The procedures a state legislature uses to resolve differences about public policy include debate, negotiation and compromise. |
| D. |
The difference between a state legislature and the U.S. Congress (i.e., that a state legislature makes laws that affect individuals within a state, as opposed to the nation as a whole). |
| E. |
Representative democracy is a system in which individuals vote to elect representatives to an assembly that represents the people in deciding issues. |
| F. |
Representative democracy derives its authority from the people. |
| G. |
Representatives are elected from the citizenry. |
| H. |
Checks and balances in government. |
| I. |
The need for compromise as a part of the legislative process. |
|
Objectives on appreciation of representative democracy--Students will appreciate:
|
| J. |
The right of every citizen to be heard by his or her representatives. |
| K. |
The need for citizens to be actively involved in state issues. |
| L. |
Legislators’ interest in meeting the needs of the people. |
| M. |
The need for checks and balances in government. |
The evaluators also developed an observation form for use by classroom monitors accompanying legislators on classroom visits. The observation form was used to collect information on the context of each visit, the amount of time spent on various activity types, the duration of the visit, the extent to which the legislator implemented certain standard program activities, levels of students’ attention and participation, legislators’ main points, and questions asked by students.
Twelve legislators visited the 15 Louisiana classes whose members fully participated in the study’s intervention group. Eight Wyoming legislators visited 10 intervention classes in that state. An additional 14 Louisiana classes and 10 Wyoming classes participated as members of the control group.
Following the study period in each state, the evaluators assessed the extent to which the America’s Legislators Back to School Program design had been implemented by examining three conditions for each monitored visit: the legislator’s implementation of at least one of the lesson types recommended by NCSL; distribution or presentation of NCSL’s Your Ideas Count material in booklet or video form; and presentation of information on or discussion of Back to School program objectives (Table A). Evidence from reports submitted by monitors supports a conclusion of full implementation by Louisiana and Wyoming legislators participating in the study.
Results
Results of this two-phase study demonstrate that Back to School Program classroom visits by legislators strengthened students’ understanding and appreciation of representative democracy. After legislators had visited Wyoming’s intervention group, students in that group demonstrated a higher level of overall understanding of representative democracy than did the Wyoming students in the control group. Knowledge of the following concepts increased:
- New laws begin when citizen groups want changes that will help improve government or meet their needs.
- The procedures a state legislature uses to resolve differences about public policy include debate, negotiation, and compromise.
- Representative democracy is a system in which individuals vote to elect representatives to an assembly that represents the people in deciding issues.
- Representatives are elected from the citizenry.
- There are checks and balances in government.
- Compromise is a part of the legislative process.
The overall average score of Louisiana’s intervention group on the knowledge section of the representative democracy test did not significantly exceed that of the control group. However, Louisiana students in classes visited by legislators scored significantly higher than Louisiana’s control group on their understanding of the difference between a state legislature and the U.S. Congress.
Students' Understanding of Representative Democracy:
Pre-test and Post-test Knowledge Scores
Students in Louisiana and Wyoming classes visited by legislators also showed higher levels of appreciation of representative democracy following their legislative visits. The overall appreciation of representative democracy by Wyoming students whose classes were visited exceeded the appreciation levels of the control group. The Louisiana intervention group’s posttest scores significantly exceeded those of the control group on an item cluster measuring appreciation of the right of every citizen to be heard by his or her representatives.
Students in Wyoming’s intervention group also scored significantly higher than those in the control group on appreciation of representative democracy. These included appreciation of:
- The right of every citizen to be heard by his or her representatives (the item on which Louisiana’s intervention group post-test scores also exceeded those of the control group)
- Legislators’ interest in meeting the needs of the people
- The need for checks and balances in government.
This evaluation of the America’s Legislators Back to School Program shows that the program, when implemented by legislators and teachers who use the recommended lesson types and materials and address the intended program objectives, results in significantly higher levels of understanding and appreciation of representative democracy among middle school students. The program’s long-term impact on students’ knowledge and perceptions of representative democracy may warrant attention in future studies.
Updated 9/22/08
Date Posted 9/4/08
Visitor counts for this page. |
