Research Resources

Multi-State Information

Legislative Evaluators:
A Diverse Group of Professionals

Note: This article is based on survey results presented at the Southwest Intergovernmental Audit Forum on Oct. 23, 1996, in Baton Rouge, La. Please address questions to the author, Rakesh Mohan, by phone at (504) 339-3836 or by e-mail.

Although it is generally known among legislative evaluators that they are a diverse group of professionals, current literature offers little in terms of specific data. To sketch a profile of legislative evaluators, I surveyed all NLPES member agencies in September 1996.

Of the 65 NLPES member agencies, 54 (83.1 percent) responded to the survey. Seven of these agencies reported that they do not conduct performance audits or program evaluations, and therefore were not included in the analysis. The total number of agencies responding to the survey differed from question to question. Also, in some cases, responses given were estimates. In this article the term "evaluation" refers to program evaluation, performance audits and like activities.

A total of 1,147 professional staff conducted evaluations within 46 of these agencies. Two states contain almost a third of these staff. The Michigan Auditor General's Office has 152 staff and the Texas State Auditor's Office has 222. Both offices report that they use the same pool of staff to conduct both evaluations and financial audits. Alternatively, agencies like Iowa Legislative Fiscal Bureau, North Carolina Fiscal Research Division, and the Oklahoma Senate report that conducting evaluations is not their staff's primary responsibility.

Educational Background

Exhibit 1 shows that more than half of the staff
in 45 agencies have graduate degrees.

Exhibit 1 Education: Staff With Highest Degree
Degree	Number of Staff	Percent of Total Staff
Masters	545	47.9%
Bachelors	482	42.4%
J.D.	45	3.9%
Doctorate	40	3.5%
J.D. and Masters	14	1.2%
Others	12	1.1%
Total	1,138	100.0%

Exhibit 2 shows that NLPES evaluation staffs have degrees in a wide range of professional disciplines.

Exhibit 2 Professional Disciplines of Staff
Professional Disciplines	Number of Staff with Degrees+	Percent of Total Staff (n=925)
Accounting	298	32.2%
Public Administration	229	24.8%
Business Administration	170	18.4%
Political Science/Public Policy	134	14.5%
Economics/Finance	84	9.1%
Law	55	5.9%
Sociology/Social Work	33	3.6%
English	31	3.3%
Statistics/Math	28	3.0%
Education	26	2.8%
Psychology	25	2.7%
Computer Science	10	1.1%
Other	102	11.0%
*Because a staff may have degrees in multiple disciplines, the total is not a useful number. +Some agencies provided approximate numbers.

Work Classification

Nearly one-third of 47 agencies report that they conduct both performance audits and program evaluations. Exhibit 3 summarizes evaluation work classification for the remaining agencies. The "other" category includes best practices reviews, performance evaluations, performance report reviews, policy analysis, program reviews, special reviews/studies, staff studies, sunset reviews, etc.

Exhibit 3 Work Classification: Program Evaluation, Performance Audit, or Other
Classification	Number of Agencies	Percent of Total Agencies
Both Program Evaluations & Performance Audits	15	31.9%
Performance Audits	10	21.3%
Program Evaluations	8	17.0%
Program Evaluations & Other	5	10.6%
Performance audits & Other	3	6.4%
Program Evaluations, Performance Audits, & Other	1	2.1%
Other than Program Evaluations or Performance Audits	5	10.6%
Total	47	99.9%

Types of Standards Used

Exhibit 4 shows that nearly two-thirds of 47 agencies use the Generally Accepted Government Auditing Standards (Yellow Book) when conducting evaluations. Only two agencies (MN and LFO, OR) use the American Evaluation Association's Guiding Principles. Two other agencies (OH, and OPPAGA, FL) subscribe to Program Evaluation Standards for Educational Programs. Standards for 16 agencies fall under the "other" category that includes state laws, internal standards or no formal standards.

Exhibit 4 Types of Standards Used When Conducting Evaluations
Type of Standards	Number of Agencies	Percent of Total Agencies (n=47)
Generally Accepted Government Auditing Standards (Yellow Book)	30	63.8%
American Evaluation Associations' Guiding Principles for Evaluators	2	4.3%
Program Evaluation Standards: How to Assess Evaluations of Educational Programs	2	4.3%
Other	16	34.0%
Total	*	*
*Because some agencies use more than one type of standards, the total is not a useful number.

Exhibit 5 lists the following information on NLPES member agencies in each state: total number of professional staff conducting evaluations, classification of evaluation work, types of standards used and whether or not the agency is responsible for conducting financial audits.

Conclusion

Survey results support the accepted observation that legislative evaluators are a diverse group of professionals. Although the survey did not address issues like the debate over program evaluation vs. performance audits, it did reveal the following tendencies about NLPES agencies:

Of the 30 agencies that use the Yellow Book, 19 (63.3 percent) are also responsible for financial audits.
Of the 30 agencies that use the Yellow Book, 23 (76.7 percent) classify at least some of their work as performance audits.
Of the 17 agencies that do not use the Yellow Book, only 2 (11.8 percent) agencies report that they also conduct financial audits.
Of the 17 agencies that do not use the Yellow Book, 12 (70.6 percent) report that they classify some of their work as program evaluation.

Acknowledgments

I extend my deep appreciation to everyone who participated in the survey and to Dr. Kathleen Sullivan (MS) who pretested the questionnaire.
-Rakesh Mohan

Exhibit 5 Profile of NLPES Member Agencies
State	Agency Name	Staff Size	Work Classification			Types of Standards Used		FA
			PE	PA	O	YB	O
Alabama	Dept. of Examiners of Public Accounts	11	0	0	1	0	1	1
Alaska	Division of Legislative Audit	30	1	1	0	1	0	1
Colorado	State Auditor's Office	60	0	1	0	1	0	1
Connecticut	Ofc. of Program Review and Investigations	12	1	1	0	0	1	0
Florida	Office of Program Policy Analysis and Government Accountability	79	0	1	1	0	0	0
Florida	Office of the Auditor General	1	0	1	0	1	0	1
Georgia	Department of Audits	28	1	1	0	1	0	1
Hawaii	Office of the Auditor	27	0	1	0	1	0	1
Idaho	Office of Performance Evaluations	5	1	0	0	0	1	0
Illinois	Office of the Auditor General	20	0	1	0	1	0	1
Indiana	Legislative Services Agency	13	1	1	0	0	1	0
Iowa	Legislative Fiscal Bureau	20	1	0	0	0	1	0
Kansas	Legislative Division of Post Audit	18	0	1	0	1	0	1
Louisiana	Office of the Legislative Auditor	29	0	1	1	1	0	1
Maine	Office of Policy and Legal Analysis	15	1	0	0	0	1	0
Maryland	Office of Legislative Audits	9	0	1	0	1	0	1
Massachusetts	Senate Post Audit and Oversight Bureau	3	1	1	0	1	0	0
Michigan	Office of the Auditor General	152	0	1	0	1	0	1
Minnesota	Office of Legislative Auditor	16	1	0	1	1	0	1
Mississippi	Joint Legislative Committee on Perf. Evaluation and Expenditure Review	21	1	1	0	0	1	0
Missouri	Oversight Division	15	1	1	0	1	0	0
Montana	Legislative Audit Division	12	0	1	1	1	0	1
Nebraska	Legislative Research Division	3	1	0	0	1	0	0
Nevada	Legislative Counsel Bureau, Audit Division	15	0	1	0	1	0	1
New Hampshire	Office of Legislative Budget Assistant, Audit Division	7	0	1	0	1	0	1
New Mexico	Legislative Finance Committee	6	1	1	0	1	0	0
N. Carolina	Fiscal Research Division	28	1	0	1	0	1	0
Ohio	Legislative Office of Education Oversight	9	1	0	1	1	0	0
Oklahoma	Chief of Staff, Oklahoma Senate	17	1	0	1	1	1	0
Oregon	Legislative Fiscal Office	13	1	0	0	0	0	0
Oregon	Secretary State, Audits Division	19	1	1	0	1	0	1
Pennsylvania	Legislative Budget and Finance Committee	21	1	1	0	0	1	0
South Carolina	Legislative Audit Council	16	1	1	0	1	0	0
South Carolina	State Reorganization Commission	13	1	1	0	1	0	0
South Dakota	Office of Local Government	N/A	0	0	1	1	0	1
Tennessee	Office of Local Government	1	1	0	0	0	1	0
Tennessee	Ofc. of Research & Educ. Accountability	15	1	0	1	0	1	0
Tennessee	Office of the Comptroller of the Treasury	20	1	1	0	1	0	1
Texas	State Auditor's Office	222	1	1	0	1	0	1
Texas	Sunset Advisory Commission	19	0	0	1	0	1	0
Utah	Legislative Auditor	21	1	1	0	1	0	0
Vermont	Legislative Council	1	1	0	0	0	1	0
Virginia	Joint Legis. Audit and Review Commission	28	0	0	1	0	1	0
Washington	Joint Legis. Audit and Review Commission	13	1	1	1	1	0	0
West Virginia	Legislative Joint Committee on Government and Finance	14	0	0	1	1	0	1
Wisconsin	Legislative Audit Bureau	26	0	1	0	0	1	1
Wyoming	Legislative Service Office	4	1	0	0	1	0	0
	Total	1147	29	29	14	30	16	21
Legends: PE=Program Evaluation; PA=Performance Audit; O=Other; YB=Yellow Book; FA=Conduct Financial Audit; 1=Yes; 0=No; N/A=Not Available

Research Resources

Multi-State Information

Legislative Evaluators: A Diverse Group of Professionals

Educational Background

Work Classification

Types of Standards Used

Conclusion

Acknowledgments

Legislative Evaluators:
A Diverse Group of Professionals