Linking Research and Policy to Improve STEM Education: A Seminar for Legislative Education Staff

Boy with Science ProjectSan Diego Marriott - San Diego, CA

December 11-13, 2009
Meeting Summaries, Presentations and Videos






Saturday, December 12, 2009

Benchmarking Student Achievement – Lessons from International Comparisons

According to international tests of student achievement in math and science like the PISA and TIMMS, American students rank near the bottom of other industrialized countries.  How concerned should state legislators be about these rankings?  As states discuss revising state standards and assessments, what should legislators know about benchmarking student achievement to international standards?


  •  Martin Carnoy, School of Education, Stanford University, California

Video 1

Summary of TIMMS and PISA results by country.

US students do poorly compared to most developed countries on international tests of math and science. Massachusetts and Minnesota test a larger sample of their students on the TIMMS. Each state’s students achieve significantly higher on math achievement than most US students. Must be cautious about drawing policy conclusions from these results, but should look at what high achieving countries do and learn from them.

Video 2
Why do US students do poorly on math and science compared to our international competitors?

Hypothesis #1 = Curriculum coverage is more limited in countries that have higher achievement in math. US math curriculum covers too many topics and doesn’t cover any of them in depth.

Hypothesis #2 = Teachers aren’t prepared adequately to teach math.

Middle school teachers are not trained at the secondary level. Salaries for teachers compared to salaries for other math professionals is a disincentive to enter teaching. Taiwan (and other countries) pay their teachers (compared to math oriented professions) much better than the United States.

Hypothesis #3 = It’s possible to increase math scores with better accountability. 
Is it worth raising student test scores on international assessments?

It’s relatively less expensive to increase accountability versus increasing the number of teachers or teachers’ education level or increasing the amount of time students spend on math.

Additional Resources

Resources of the Institute of Education Sciences: Regional Labs, What Works Clearinghouse

The Institute of Education Sciences “What Works Clearinghouse” and the system of Regional Education Labs (RELs) around the country are excellent but little-known resources for legislative staff. This session will provide an overview of each and tips on how to access and use them to help guide you to high quality research on your most pressing policy concerns.

  • Jane Best, Senior Policy Associate, REL Midwest at Learning Points Associates, Illinois

Video 1
REL – Regional Education Laboratories (REL)
What are they? How can the support you? Learn generally about the RELS and the services they offer to support you and your elected officials. 

  • Jill Weber, Director, REL Northeast and Islands, Massachusetts

Video 2
REL: Learn how a REL can best serve you and your elected officials through examples from the Northeast REL and their recent partnership with NCSL and the Rhode Island General Assembly.

  • Jill Constantine, Associate Director of Research, Mathematica Policy Research, Inc., New York

Video 3
REL: Learn about the What Works Clearinghouse, a dissemination effort housed out of the U.S. Department of Education’s Institute of Education Sciences.

Additional Resources

Alternative Certification

Schools continue to struggle to find qualified math and science teachers at all grade levels. Many states are creating alternative certification programs in hopes of eliminating barriers to teaching and to attract people from STEM industries to choose teaching as a second career. But what do we know about the effectiveness of teachers who receive alternative certification? This session will highlight an IES report that compares teachers certified through alternative routes to traditional programs while emphasizing the elements of quality needed in any certification program.

  • Jill Constantine, Associate Director of Research, Mathematica Policy Research, Inc., New York

Learn about the basics of an IES study that compares teachers qualified through alternative routes to traditional preparation programs in elementary schools. Impetus for study: alternative certification programs are supplying an increasing number of teachers to the educator workforce. Approximately 30% of new hires are trained through alternative routes to certification. Guiding research question: do the alternative routes adequately prepare teachers for the classroom?

AC- Alternative Certification

TC – Traditional Certification


Primary research questions: What are the effects on student achievement of teachers trained on alternative routes to certification (school principal and school district question)? What aspects of teacher preparation programs are associated with teacher effectiveness (e.g., timing of program, amount of coursework and content of coursework)? Also learn about the study design. 


Learn about the states selected for the study and other elements of the study. AC teachers are largely in schools with a high percentage of minorities and free and reduced lunch eligible, as these are the areas with shortages.


Study findings. Coursework and timing of coursework varied greatly required among AC programs across states.


Teacher findings. AC candidates are more likely to be minorities and much more likely to have children. Most candidates have bachelor degrees in psychology, children development and most are already working with children in school/day care settings. Policy take aways: AC programs are good at recruiting minorities. AC programs are not good at recruiting math/science professionals.

Additional Resources

National Math Panel – Recommendations for Improving Math Teachers and Strengthening Math Standards

As states consider whether they should adopt the forthcoming “common core” for their state standards, what would relevant and rigorous math standards look like? The 2008 National Math Panel declared the delivery system in mathematics education "broken". This session will review research-based strategies to improve math achievement through teacher preparation, instructional materials, and more effective assessments.

  • Russell Gersten, Executive Director, Instructional Research Group, California
Video 1
Introduction and framing of the National Math Panel report

There is a cultural component The United States has weak interventions for identifying problems in learning math and our math curriculum lacks coherence. Our math curriculum is very procedural, rather than conceptual, to its detriment.

Video 2
Math Standards 

Math curriculum and standards need to cover less and go deeper than they currently do. Proficiency with fractions is critical to mastering more complicated math, and US students are particularly lacking in this skill. The National Math Panel recommends standards that are very precise and exact.

Video 3
Instructional Practice

There is good evidence that students should work on intricate, multi-step problems as a culminating activity. Use of rigorous formative assessments, a low-cost tool, can increase student test scores by about 1/10 of a standard deviation (a very good result). Students need an opportunity to talk about, model and explain the steps they took to solve a problem.

Video 4
The “Math Wars” are Ill Conceived

Teaching concepts versus teaching the basics is an ill conceived argument. Both are necessary.

Video 5
What Makes a Great Math Teacher

Research on “what makes a great math teacher” is unclear. Math knowledge seems to be the strongest indicator, and at the high school level, the math courses that teachers took may be a predictor. But evidence is lacking. There is some (old) research that suggests that the more interactive math instruction is the better students perform. There is little evidence that suggests a particular teacher training program is better than others. Some researchers like the idea of job-embedded training as well as math specialists who teach only math to students for most of the day.

Additional Resources

Sunday, December 13, 2009

Minorities and Women in STEM – How “Stereotype Threat” Affects Student Achievement

(And Other Lessons from High-Stakes Testing)

States have taken action to encourage more girls and minorities to pursue STEM degrees in college. But what are the effective strategies for ensuring girls and minorities remain in the STEM pipeline from elementary school through college? How does the social and emotional well-being of a student, and high-stakes tests, effect his or her achievement? This session will focus on effective interventions in K-12 to attract these underrepresented populations to STEM subjects and ensure their academic success.

  • Joshua Aronson, Associate Professor, Department of Applied Psychology, New York University
Additional Resources