NLPES Site Map

NLPES Question of the Month

October 2001

When your legislators ask you, as an audit or program evaluation office, for answers to policy questions, how do you respond? For example: Is the child protection agency too quick to remove children from their homes? Is the Indian Gaming regulatory agency doing work that should be done by Tribal regulators? Should the corrections department send more inmates to private prisons? How does your office respond to questions like these?

From: John Turcotte, Florida

Our office does policy analysis routinely. When asked or directed by the Legislature, we respond using appropriate research methodologies that develop a basis for our conclusions and recommendations, which are generally unequivocal. During direct encounters with legislators during hearings, I encourage OPPAGA staff to respond to such questions if we have spoken previously about the issue through one of our reports. However, if we haven't, I suggest that our staff answer that our office has not taken a position on that issue and then suggest that the legislator may want to request us to do an examination so that we would be in a position to answer. That is what professional independence is all about. It will be taken away from you if you don't use it to answer the tough questions.

A good example is our recent report on the Blind Babies program in Florida. The Legislature directed us to determine if it would be "sound public policy to continue funding the program." We found that the program lacked accountability and recommended that it not be funded. See: http://www.oppaga.state.fl.us/reports/educ/r01-42s.html

From: John Sylvia, West Virginia

We can only answer a policy question after a review of the program. If we haven't done a recent audit that directly addresses the issue we cannot speak to it. We normally respond to policy questions by asking if they would like us to examine the issue; however, the request must go through either the chairs of our committee or the Senate President or House Speaker.

From: Ethel Detch, Tennessee

My office responds to such legislative requests whenever we can. If we feel qualified to answer the question based on previous work, we do. Otherwise, we would tell them that we need to gather some additional information and get back with them. We would normally answer the question in a letter to the committee chair and/or the requestor.

My opinion is that evaluators often develop extensive expertise on particular topics and I think it serves our legislatures better when we can share some of that with them. However, be aware that my office is not an "audit" office so there may be additional considerations in those offices.

From: Joel Alter, Minnesota

Policy questions are fair game for our office, particularly if they can be addressed with some sort of empirical inquiry. For example, a study looking at whether child protection removes children from their homes too quickly might include a review of a sample of cases. We might also try to systematically document the perceptions of experts—for instance, through surveys of mandated reporters of child abuse/neglect or judges who handle child protection cases. In addition, we might critique the process for making decisions for removing kids from home—for example, by examining who participates in the decision process, and considering variations in the process among different jurisdictions or offices.

In policy studies, we are more likely than in other studies to present a discussion of competing policy options, rather than making hard, fast recommendations. For example, when we were asked to evaluate ways to fund our state’s probation system, we identified criteria for a funding system and discussed how various approaches met the criteria. We also looked at possible fiscal implications that various funding alternatives would have on individual counties. Ultimately, however, we recognized that the preferred policy options would depend on the principles and philosophies of individual legislators.

So, in policy studies, I think we can help legislators by providing a FRAMEWORK FOR DECISIONS—even in cases where we don’t feel that we can justify recommending one course of action over another. We can clarify the underlying issue (that led to the policy question), sort out the evidence that might indicate a need for action, identify potential solutions, present evidence that might help legislators evaluate the merits of various options, and suggest criteria for evaluating policy choices. Then, we can step aside (but never too far away) and let the policy makers do what they were elected to do.

From: Sylvia Hensley, California

When the Legislature asks us to respond to policy issues, we do so provided we have applied appropriate audit procedures to support our conclusions or recommendations. For example, it's not that uncommon for an audit request to include questions regarding policy issues. In these cases, we design audit procedures to develop the information necessary to answer the question(s).

However, sometimes policy questions arise during briefings or hearings. In such cases, we would respond to the extent we have done work in the area. If we had not done sufficient work in the area, we would not take a position.

The only exception to the above is when the policy issue has a direct bearing on our office--questions such as, should the authority of the state auditor be expanded to include xyz?

From: Ken Levine, Texas

The Sunset Commission in Texas is a policy evaluation office. We start with the policy question of whether an agency or its functions are needed or not. If those functions are needed, is the agency under review the best place to handle those functions or will the State benefit from integrating those functions with a different agency?

Policy questions similar to the examples in your Question of the Month often are raised and evaluated during a Sunset review. For example, a few years ago we looked at whether children languish too long in foster care during our review of the Texas Child Protective Services agency. We recommended, and the Legislature implemented, statutory time frames for either returning children to their homes or moving toward a permanent placement, usually adoption.

Additional examples: During the last biennium, the Commission recommended changes in policy issues such as ensuring availability of information to make decisions on the acquisition and disposition of lands for state parks and wilderness/wildlife conservation; expanding purchasing methods to better use electronic procurement and tools like reverse auctions; and establishing a program to provide incentives to landowners and developers to clean up contaminated oil field sites.

The Sunset Commission does not often receive special assignments on policy issues. However, this cycle we have been asked, in statute, to review the need for regulation of riding stables and to look at the effectiveness of a chapter of state law regarding the placement of correctional facilities. So, we are branching out a bit into special request reviews.

During the Legislative session, we are often asked about new and additional changes to state agencies that go beyond the Commission's recommendations. If we have examined and are familiar with that area of agency operation we, of course, provide the answers. If we don't know the answer, we will request time to find out and respond. That time frame can often be a day or even less. But as a legislative service organization, we believe our responsibility is to respond to the best of our professional ability.

I hope this information helps. Does anyone else out there do these types of reviews/evaluations?

From: Heather Moss, Washington

After reading the few responses that have been posted regarding the extent to which our offices answer policy questions, it appears that most of us do, to some extent. What I would like to point out, though, is the distinction that Joel Alter made, which also reflects how we approach "policy" issues. (Keep in mind that I am writing from the perspective of an analyst that works for a legislative auditor hired by the legislature.)

Since we do work for the legislature, we are obligated to be as responsive to their requests as possible. If this means going beyond a typical performance audit or sunset review, we often work with the members to fashion a special study to get at their questions and concerns. We will, and often do, address policy questions; however, we do not offer opinions as to how the members should make policy decisions. Rather, we provide them with information about the alternatives (e.g.., their cost and programmatic implications). Then, as Joel indicated, "we can step aside...and let the policy makers do what they were elected to do." We don't answer the policy question with a single recommendation, but we provide the members with the information that will allow them to make informed decisions.

With that, I would be curious to hear from other folks out there who see it differently -- actually, some of you who have already responded to the first question make it appear that you do provide definitive opinions on policy issues. If that is the case, then how would your work be different than that of staff for the legislative policy committees (or, for that matter, the caucus staff)? I see the distinction between our offices (legislative program evaluation) and other legislative staff being that we offer information based on (hopefully) quantitative, factual information; other legislative staff use that same type of information, but then go on take a qualitative position on what is "best", "right", "wrong", or (out on a limb here) "politically safe". Is my distinction too simplistic? Any thoughts?