Professional Development Resources

Staff Strategies for Strong Data Analytics

Two NLPES officers share their experiences and strategies for staff to ensure high quality data analytics, including the roles and responsibilities of methodologists/data analytics staff and their involvement with project teams.

• Webinar (60 minutes)

Drugs and Money—The data analysis hurdles and solutions involved in measuring participant outcomes and cost savings generated by expansion drug courts

This webinar will explain the rationale and use of the methodological techniques featured in OPPAGA Report 14-02, which won the 2015 NLPES Excellence in Research Methods Award.  The data revolution provides many new opportunities for program evaluators to empirically measure and analyze the real world outcomes of the programs we study, but it also brings a new set of challenges.  One of the major challenges is how to properly compare the outcomes of groups participating in a program to non-participants.  Groups differ in characteristics that could affect outcomes and these measurement challenges are exacerbated when the outcomes being measured are affected by exposure time.  For example, criminal recidivism rates can differ drastically, when measured over the period of six months versus eighteen months.  This E-Learning webinar provides an example of how to deal with these types of data challenges using a technique called matched pair survival analysis, which can be performed in many commonly used statistical software packages.

• Webinar (45 minutes) 
• Can be used for individual training or in a classroom setting.

Teen Pregnancy Prevention Program: A Case Study in Evidence-based Policymaking

As one of the few government programs that both uses evidence as a criteria for funding decisions and continues to rigorously evaluate its efforts and results, the Teen Pregnancy Prevention Program can offer important lessons for state policymakers.  This interactive webinar provides an opportunity for state legislators and legislative staff interested in evidence-based policymaking to learn more about TPP and hear insights that could help inform state-level efforts. 

• Webinar (60 minutes)

2020's Remarkable Research Methods

In 2020, NLPES recognized the New Mexico Legislative Finance Committee and  the Washington Joint Legislative Audit and Review Committee for their outstanding use of evaluation research methods.  During this webinar, you will learn more about each office's award-winning methodology and the lessons learned. New Mexico staff will describe its evaluation of childcare assistance effectiveness. Washington staff will describe how they evaluated the opportunity scholarship and opportunity expansion programs. 

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Confidential Data Access Issues for Auditors and Evaluators

Access to confidential data is often a key factor in the ability to answer legislators' concerns when performing an audit or evaluation of an agency or program. In this webinar, staff from three offices will share their experiences, issues and solutions when it comes to getting access to sensitive or confidential data necessary to do their work. Specifically, we will focus on what types of training and communications are in place to ensure our own staff know their rights and responsibilities regarding confidential data. We will discuss mechanisms our offices have in place to communicate with the auditee on getting confidential data. We also plan to discuss innovative ways or best practices to help overcome agencies' fears or reluctance to provide confidential data to audit and evaluation staff.

• Webinar (33 minutes)
• Webinar PowerPoint
• Can be used for individual training or in a classroom setting.
• See also NLPES Data Access Issues for Auditors and Evaluators

Innovative Evaluation in New Mexico

The New Mexico Legislative Finance Committee is continually working to improve research methodologies to ensure staff are providing policy makers with the best and most up-to-date research available. The office is working with the Abdul Latif Jameel Poverty Action Lab (J-PAL) at the Massachusetts Institute of Technology on a randomized control trial on the effectiveness of early college high school curricula in New Mexico. Learn about New Mexico's innovative research methodology and J-PAL's State and Local Initiative.

• Webinar (50 minutes)
• Can be used for individual training or in a classroom setting.

Pivot Tables Are Easy

This webinar illustrates that a pivot table report is a surprisingly easy, interactive way to quickly explore large amounts of data in many different ways. A pivot table report can be used to analyze numerical data in detail and to answer unanticipated questions about your data.

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Quantitative Methods

Hard numbers can provide compelling evidence. This PowerPoint provides an overview of three quantitative methods: central tendency, regression and survival analysis.

• Power Point (27 slides)
• Designed as a 50 minute class with instructor notes and discussion slides at the end.

Research Methods That Rock

In 2019, NLPES recognized Kansas, Louisiana and Virginia for their outstanding use of evaluation research methods.  During this webinar, you will learn more about from each office's award-winning methodology and the lessons learned. Kansas staff will describe how they evaluated the state's Medicaid program. Louisiana staff will describe how they measured local government bond issuance costs. Virginia staff will discuss its analyses of spending on inmate healthcare.

• Webinar (60 minutes; to be held Mar. 26, 2020; registration is open)
• Can be used for individual training or in a classroom setting.

Reviewing State Agency Contracts

The increase of government contracting has increased legislative interest in the accountability of contracted projects. This training describes how to conduct consistent, well-documented performance reviews of state contracts.

• Power Point (36 slides)
• Designed as a 50 minute class with instructor notes and group exercises.

Samples

Properly executed samples allow us to collect less information but reach reliable conclusions. This PowerPoint provides an overview of types of samples, explains when each is appropriate, and discusses how to determine sample size.

• Power Point (14 slides)
• Can be used as a 50 minute class if local examples are added for discussion purposes.

Survey Development

Getting good survey data starts with survey development.  This PowerPoint discusses determining your survey goals and objectives, sample selection, methodology, survey layout and question and response sets.

• Power Point (29 slides)
• Designed as a 50 minute class with instructor notes and group exercises. Adding local examples would be helpful.

Survey Refresher

These slides present ways to present surveys and when to conduct them.

• Power Point (36 slides)

Two Outstanding Research Methods

Each year, the National Legislative Program Evaluation Society recognizes up to three offices for reports that demonstrate outstanding use of evaluation research methods.  In 2016, the Louisiana Legislative Auditor and the Colorado Office of the State Auditor won Excellence in Research Methods awards.  During this webinar, Louisiana and Colorado staff will explain their award-winning methodologies.

Louisiana staff will describe its method to measure how lottery ticket sales respond to changes in the prize payout percentage and how this method can be applied to estimate how a change in policy will affect outcomes in a variety of settings.

Colorado staff will describe how GIS-based methodologies were applied to the sales tax system and how this method can be used to identify, analyze and communicate patterns and relationships that have a spatial dimension.

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Utah's Award-Winning Methods to Test Regulation of Payday Lending

The Utah Office of the Legislative Auditor General won the 2017 NLPES Excellence in Research Methods Award for its report testing the regulation of payday lending. Staff from the Utah Office of the Legislative Auditor General will explain the audit issue that they were asked to address, the award-winning methodology that they developed to address the issue, and how the methodology might be applied to other contexts.

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Dealing With Difficult Stakeholders

Legislative staff often find themselves interacting with stakeholders (members, constituents, etc.) who present themselves in challenging ways. Whether it’s an emotional issue, or the stakeholder lacks trust, dealing with these types of situations can take a toll. This workshop will explore techniques to deal effectively with these sorts of situations in a way that preserves the relationship.

• Webinar (60 minutes)

Evaluator 101 Training Series

A basic skills training series for NLPES members. The four-part Evaluator 101 training provides newer evaluators with the foundational elements of sound legislative program evaluation. 

Session 1: Understanding Government Accountability and the Types of Evaluations

Explore the basics of governmental accountability and the purposes and types of evaluations. The speakers for this session were Jennifer Sebren (Mississippi; moderator), Patricia Berger (Pennsylvania) and Eric Thomas (Washington).

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Session 2: Planning for Audits/Evaluations and Developing Findings

Discuss critical areas in planning an audit or evaluation, review the elements of a finding, and explore the differences between good and bad findings. The speakers for this session were Sam Hearn (Mississippi; moderator), Leslie McGuire (Georgia) and Jennifer Sebren (Mississippi).

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Session 3: Gathering and Compiling Evidence

Learn tips for collecting data, analyzing data, combining evidence and documenting results. The speakers for this session were Jennifer Sebren (Mississippi; moderator), Jon Courtney (New Mexico) and Gary VanLandingham (Florida).

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Session 4: Clearly Communicating Findings

Discuss how to develop a written report and presentations to clearly and concisely communicate findings. The speakers for this session were Jennifer Sebren (Mississippi; moderator), Kiernan McGorty (North Carolina) and Kristen Rottinghaus (Kansas).

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Legislative Code of Conduct

Legislative staff must make difficult judgments sometimes to guide their service to their state legislatures. This PowerPoint provides basic concepts to help staff develop a sense of responsibility for their actions.

• Power Point (21 slides)
• Can be used for a 50 minute class but does not include local examples of ethical dilemmas and how they should be addressed.

Ethics Training for Program Evaluators and Performance Auditors

Evaluators and auditors generally are the legislative staff responsible for assessing and judging state agencies and programs.  Because of the important role they play, we want to believe that evaluators and auditors are ethical.  Ethical behavior, however, is not always black and white.

• Webinar (archived version)
• Can be used for individual training or in a classroom setting.

Managing A Remote Staff

High quality staff management is a challenge even in the best of times.  But in a work-from-home world, new obstacles have increased the difficulty of connecting with, and effectively guiding, our teams.  This workshop will explore the lessons and best practices the past year of remote work has taught us about overcoming these hurdles, equipping us with leadership skills that can translate to life beyond WFH.

• Webinar (archived version; please note, you must be logged into your NCSL account to access the archived video.)
• Can be used for individual training or in a classroom setting.

Developing Evaluation Plans

This 36-slide PowerPoint is about developing evaluation work plans.  A key part of that process is laying your ideas out on paper.  This presentation describes how to develop a plan to incorporate your research questions/issues, methods, staff assignments, deadlines and communication strategies into one document that will help your team stay on task and on time. Designed as a 60 minute class with instructor notes and group exercises.

• Developing Evaluation Plans without notes (PDF)
• Developing Evaluation Plans with notes (PDF)

Planning and Scoping Evaluations

The audit and evaluation process is broken up into three phases—planning/scoping, fieldwork and writing.  This 60-minute webinar  addressed planning and scoping evaluations.  It covered an overview of Texas's scoping tool and staff training; Michigan's referent group meeting; and general best practices and pitfalls for planning and scoping.

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Publishing Reports Online

Moving to online reports involves cultural, process and technological changes. Staff from the Washington Joint Legislative Audit and Review Committee will share lessons learned and discuss the processes involved with publishing reports online.

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Report Writing

This webinar by Stan Stenersen explores how to convey complex information to busy readers. It gives writers an understanding of how busy readers learn as well as a set of tools for writing documents that can be understood and acted upon. The webinar includes several opportunities for participation using handouts.

• Webinar (90 minutes)
• Can be used for individual training or in a classroom setting.

Using Data Visualization to Improve Messaging

In order to draw attention to work that incorporates issues and convinces others to be proactive, a message must be clearly and accurately articulated.  The webinar will provide a brief synopsis on how people see and understand information from data displays. It will provide fundamentals on how to present good data visualizations and discuss the importance of using the appropriate colors and charts to engage the audience.

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.

Writing for a Legislative Audience

Legislative staff create a wide variety of written products. From newsletters to emails to constituent correspondence—the list goes on and on. But regardless of the purpose, certain tactics are essential to crafting concise, efficient and easy-to-digest content. In this workshop, you will learn how to effectively organize your thoughts and put them across in a form that maximizes the impact of your information.

• Webinar (60 minutes)
• Can be used for individual training or in a classroom setting.