 |
Home | Contact Us | Press Room | Site Overview | Help | Login | Register |
 |
 |
| About NCSL | State & Federal Issues | Legislatures | Legislative Staff | Meetings | Bookstore | Legislators & Staff Only |
| NCSL Home > Legislative Staff: Staff Sections & Networks > NLPES > | Add to MyNCSL |
NLPES NewsMay 1998, No. 70 Inside the NewsChair's Corner Using Listserves as a Research Tool News Makers Chair's CornerMary Noble (CA) 
This is an exciting time for NLPES! Our committees are all working hard-refining the NLPES listserve and our Web home page, developing a brochure for new and potential members, working on our issue of New Directions (published by the American Evaluation Association), preparing the newsletter, keeping up with what other organizations are doing, coordinating panels for the annual meeting, and organizing the fall training conference. State key contacts will soon be asked about their desires for training for the fall NLPES conference here in Sacramento. We want the training to appeal to a broad group and suit your needs. So, even before you hear from an executive committee member, please e-mail or call Art Heikkila, (602) 553-9858, the chair of the training committee or me with your suggestions. Of course, I personally am excited about the fall conference. El Niño will be gone by September, so I will go out on a limb and predict good weather for the conference. To continue what has become a tradition of excellence (based on the feedback we have received), we are working to bring you a great meeting! Our Executive Committee members continue to be stolen away by state departments. First it was Tom Bardin (SC) and Craig Monson (UT) and now two more members! Sharon Patnode is leaving the Kansas state auditor's office to become an assistant to the director for the Division of Health, Kansas Department of Health and Environment. And, most recently, Jennifer Noyes is leaving the Wisconsin state auditor to become deputy director for a newly created program evaluation office in the executive branch. Obviously, state agencies have caught on that evaluators and auditors have the expertise they need! We will really miss you Sharon and Jennifer, and we wish you great success in your new jobs. NLPES Awards Update Applications are due May 15It is time to crank up the word processor, kiss the Blarney stone, and start working on applications for the 1998 NLPES awards! NLPES will offer awards in three categories: *Excellence in Program Evaluation *Excellence in Research Methodology *Excellence in Impact. The awards will be given at the July NCSL conference in Las Vegas. The criteria for each of the awards will be the same used in 1997. More information about the awards application process is available from the NLPES homepage and has been sent to NLPES key contacts as well. NLPES Executive Committee Election Applications Sought Deadline: May 15, 1998Wanted: Six new members for 1998-2000Why: A unique opportunity to improve program evaluation nationally through NLPES programs and activitiesWhat: Travel to three meetings annually, plus your time and your agency's supportHow: Send a brief bio and your goals and proposals on program evaluation (less than 200 words)Info: Jane Thesing, S.C. Legislative Audit Council, (803) 253-7612; fax: -7639Send to: Bob Boerner, NCSL, (303) 364-7700; fax: -863-8003By Mail: 1560 Broadway, Suite 700, Denver, CO 80202Office ProfileMinnesota Office of the Legislative Auditor The Program Evaluation Division of the Minnesota Legislative Auditor was established in 1975 to examine the performance and efficiency of state programs. The division also conducts studies that identify best practices of local government services. Governing Body: 20-member Legislative Audit Commission (10 House, 10 Senate; 10 Democrats, 10 Republicans). The commission usually picks five or six topics for evaluation studies in April or May of each year. 20-member Legislative Audit Commission (10 House, 10 Senate; 10 Democrats, 10 Republicans). The commission usually picks five or six topics for evaluation studies in April or May of each year.Staff Backgrounds: The division has 17 staff; five have Ph.D.s and the remainder have master's degrees, primarily in the social sciences. On average, staff have been with the office for 11 years. The division has 17 staff; five have Ph.D.s and the remainder have master's degrees, primarily in the social sciences. On average, staff have been with the office for 11 years.Recent Evaluation Reports: Since January, we have issued reports on child protective services, remedial education, transit services, state building maintenance, state trust lands, and local 9-1-1 services. Since January, we have issued reports on child protective services, remedial education, transit services, state building maintenance, state trust lands, and local 9-1-1 services.Little Known Facts About the Legislative Auditor: Jim Nobles, the Minnesota legislative auditor since 1983, went to school in Hot Springs, Arkansas, with Bill (then Billy) Clinton. Jim's brother was Governor Clinton's state trooper bodyguard for a while. Jim has not been subpoenaed by Ken Starr... well, not yet. Jim Nobles, the Minnesota legislative auditor since 1983, went to school in Hot Springs, Arkansas, with Bill (then Billy) Clinton. Jim's brother was Governor Clinton's state trooper bodyguard for a while. Jim has not been subpoenaed by Ken Starr... well, not yet.Office Mantra: Sweat the small stuff. Sweat the small stuff.Dream Assignment: One of our staff traveled to the South Pacific islands of Palau and Micronesia to help their governments establish a performance-based budgeting system. One of our staff traveled to the South Pacific islands of Palau and Micronesia to help their governments establish a performance-based budgeting system.Difficult Assignment: When the Palauan delegation made a reciprocal visit to Minnesota, our staff had a difficult time locating a Twin Cities source of betelnuts, which our visitors chewed regularly. When the Palauan delegation made a reciprocal visit to Minnesota, our staff had a difficult time locating a Twin Cities source of betelnuts, which our visitors chewed regularly.Least Flattering Description of Our Staff: We have debated which characterization was worse: the agency that compared one of our project managers to Hitler or the agency that described another manager as the Antichrist. We have debated which characterization was worse: the agency that compared one of our project managers to Hitler or the agency that described another manager as the Antichrist.Least Flattering Description of an Agency: One of our staff referred to an agency's "boneheaded" mistakes during a legislative hearing. One of our staff referred to an agency's "boneheaded" mistakes during a legislative hearing.Oddest Media Request: A TV station got a hot tip that an agency we had recently evaluated was assembling its staff for a party with strippers, and the station called us to see if staff from our office would accompany the camera crew when it tried to crash the party. The station assumed that the agency couldn't deny admission to the Legislative Auditor. (We refused the request-we were having too much fun doing our regular work.) A TV station got a hot tip that an agency we had recently evaluated was assembling its staff for a party with strippers, and the station called us to see if staff from our office would accompany the camera crew when it tried to crash the party. The station assumed that the agency couldn't deny admission to the Legislative Auditor. (We refused the request-we were having too much fun doing our regular work.)Most Curious Things in State Agency Performance Reports: The Department of Education said that one category of its customers was "learners as learners." The Department of Natural Resources proposed to increase the percentage of urbanized areas shaded by trees from 30 percent in 1992 to 50 percent in 1995 (undoubtedly, the Miracle-Gro expenditures would have been astronomical). The Capitol area police force reported trends in the number of suspicious persons in the Capitol area--we were tempted to question the validity of the numbers because lobbyists were not included. The Department of Education said that one category of its customers was "learners as learners." The Department of Natural Resources proposed to increase the percentage of urbanized areas shaded by trees from 30 percent in 1992 to 50 percent in 1995 (undoubtedly, the Miracle-Gro expenditures would have been astronomical). The Capitol area police force reported trends in the number of suspicious persons in the Capitol area--we were tempted to question the validity of the numbers because lobbyists were not included.Seasonal Activities: It's always nice to schedule site visits to northern Minnesota when the trees are changing colors. One of our staff enjoyed the experience of riding shotgun in a snowplow during a study of winter highway maintenance, but none of us enjoy the frequent threat of driving to or from site visits in blizzards. During a site visit cut short by an unexpectedly severe blizzard, one project team drove in zero visibility at low speeds for nearly four hours in windswept western Minnesota, periodically opening the door to see if they were still on the road. It was probably fortunate that they could not drive faster because the car issued by the state motor pool didn't have a passenger side seat belt. It's always nice to schedule site visits to northern Minnesota when the trees are changing colors. One of our staff enjoyed the experience of riding shotgun in a snowplow during a study of winter highway maintenance, but none of us enjoy the frequent threat of driving to or from site visits in blizzards. During a site visit cut short by an unexpectedly severe blizzard, one project team drove in zero visibility at low speeds for nearly four hours in windswept western Minnesota, periodically opening the door to see if they were still on the road. It was probably fortunate that they could not drive faster because the car issued by the state motor pool didn't have a passenger side seat belt.Best Performance by a Deputy Legislative Auditor in a Supporting Role: While helping a project team prepare for a slide presentation at a hearing, Roger Brooks accidentally dumped all of the slides on the floor shortly before the hearing began. While helping a project team prepare for a slide presentation at a hearing, Roger Brooks accidentally dumped all of the slides on the floor shortly before the hearing began.Warm-Hearted Auditors: Division staff annually organize contributions to a nearby food shelf-in December, we collected 650 pounds of food from our staff and the office's financial auditors.Not For the Faint-Hearted: One of our interns passed out during a tour of the state's maximum security prison. One of our interns passed out during a tour of the state's maximum security prison.Liaison ReviewNational State Auditors Association The National State Auditors Association (NSAA) currently is involved in a variety of activities of interest to performance auditors and program evaluators. The NSAA performance audit committee, chaired by Bill Holland of Illinois, recently released a joint performance audit project on long-term care, and has selected home health care as the new topic for the next audit. Pennsylvania is the lead state on the new topic; the scope and objectives are supposed to be firmed-up in April. The committee has been working on ways to get greater national and local exposure for these audits, including having them presented at the NCSL annual meeting. On the "emerging issues" front, state auditors have for some time followed the increase in electronic commerce (i.e., issuing food stamps through electronic cards) and how electronic programs may be audited. Other issues include the expectation gap between audit requestors and auditors, dealing with the press, and whether state auditors will have any role in auditing performance measures in federal government. NSAA's annual meeting this June in Rehoboth Beach, Delaware, will feature many program sessions that directly affect performance auditors as well as those with fairly universal appeal. Kurt Sjoberg from California is working on sessions related to the following: Performance audit practices (best practices: what works and what doesn't); how to stay ahead; and the best methodologies from program evaluation to apply to performance auditing)-Kurt is hoping to get people involved from NLPES. Hotlines and whistle blower programs. Technology issues (for example, electronic data and interchanges, computer-assisted audit tools, and electronic work papers). Staffing issues (retaining, recruiting, and training staff-both auditors and information technology people-in a booming economy). Kurt is also hoping to have tables set up for the different states to "show off" the work they do, just as NLPES did at its October 1997 training meeting. As more and more state audit shops get involved in performance auditing, there's an increased interest in what type of work is being done, who the audience is, how findings are presented, and how writing styles and report approaches vary. Given the interest among state auditors in performance auditing, I think there will be real opportunities in the future for NLPES to exchange "how to" training and ideas for conducting performance auditing. I also think it's a great opportunity to send NSAA states that aren't involved in NLPES-but that may be getting involved in performance auditing-information about our annual and fall meetings. A View from the StatesPerformance-Based Budgeting in Three States
Florida, Texas and Minnesota are three states often cited as leaders in performance-based budgeting-the authors will be the first to acknowledge that their states have a long way to go before they can truly be considered "models." This article represents a comparison of performance-based budgeting approaches in our states. What follows are some of our observations about similarities, differences, lessons learned and future prospects. Similarities Performance-based budgeting is not a new concept, but legislation was critical to recent reforms. Each of the three states has had a long history of program budgeting and efforts at performance measurement. However, the current performance-based budgeting efforts in each state were shaped by legislative directives-in Texas in 1991, in Minnesota in 1993 and in Florida in 1994.Agencies received no new funding to develop performance information. In general, agencies were expected to use existing resources to produce improved information. A Minnesota study estimated that 20 agencies in that state spent a total of $1 million developing their first set of performance reports in 1994. Two states have centralized information systems for storing performance-related data. Information on Texas' and Minnesota's automated systems is entered by agency staff. Performance information in Florida is maintained by each agency and submitted as part of the executive budget request to the legislature. Information on Texas' and Minnesota's automated systems is entered by agency staff. Performance information in Florida is maintained by each agency and submitted as part of the executive budget request to the legislature.Legislative audit agencies have played a key role in the oversight of performance information developed by agencies. The Texas State Auditor's Office certifies that agency performance measures are reliable and valid and have adequate control systems to promote accuracy in future reporting. . The Texas State Auditor's Office certifies that agency performance measures are reliable and valid and have adequate control systems to promote accuracy in future reporting.Florida's Office of Program Policy Analysis and Governmental Accountability reviews and interprets agencies' performance information, noting whether measures are valid and whether the agencies have ways to ensure the validity and reliability of their performance data. (Florida requires agency inspector generals or internal auditors to validate performance data, but lack of staff does not allow comprehensive review.) Minnesota's Office of the Legislative Auditor issues comments on each agency's performance report in each biennium, with particular emphasis on the appropriateness of measures but with little review of data accuracy. Agency "buy-in" is an important component of successful performance-based budgeting. A good performance measurement system is grown "from seed" in each agency-it is not easily transplanted, even among similar agencies. Performance-based budgeting is more likely to succeed in cases where performance measurement is viewed by agency leaders as central to the management of the agency and not just as a bureaucratic requirement. . A good performance measurement system is grown "from seed" in each agency-it is not easily transplanted, even among similar agencies. Performance-based budgeting is more likely to succeed in cases where performance measurement is viewed by agency leaders as central to the management of the agency and not just as a bureaucratic requirement.Performance information provides many potential benefits. Staff members in all three states agree that a performance-based budgeting approach can: (1) help clarify agencies' missions, goals and objectives; (2) enhance program management and strategic planning; (3) improve external accountability and oversight; and (4) improve policy and budget decision making. . Staff members in all three states agree that a performance-based budgeting approach can: (1) help clarify agencies' missions, goals and objectives; (2) enhance program management and strategic planning; (3) improve external accountability and oversight; and (4) improve policy and budget decision making.Differences Florida phased-in performance-based budgeting; Minnesota and Texas acted immediately. Florida established a seven-year schedule for implementing performance-based budgeting at all major agencies. Thus, some agencies have already produced performance information, while others have not. Florida agencies in the latter part of the schedule have benefited in some cases by the experience of agencies already under performance budgeting. In contrast, Minnesota and Texas required most major state agencies to begin reporting performance information immediately following enactment of their performance budgeting laws. Recently, Minnesota's Office of the Legislative Auditor concluded that implementation of the state's performance reporting law occurred too quickly, with too little staff training.Approval of strategic plans and performance targets differ. In Texas, executive and legislative budget offices approve agencies' strategic plans, which include goals, strategies for accomplishing them, and funding levels. In Texas and Florida, agencies develop performance "targets" for their programs, and the legislatures formally adopt performance targets during the budget process. Unlike Texas, performance targets in Florida are not derived directly from agencies' strategic plans. In Minnesota, agencies develop performance targets for each program, but the legislature rarely sets its own performance targets for programs, and it does not approve agencies' strategic plans or performance measures. In Texas, executive and legislative budget offices approve agencies' strategic plans, which include goals, strategies for accomplishing them, and funding levels. In Texas and Florida, agencies develop performance "targets" for their programs, and the legislatures formally adopt performance targets during the budget process. Unlike Texas, performance targets in Florida are not derived directly from agencies' strategic plans. In Minnesota, agencies develop performance targets for each program, but the legislature rarely sets its own performance targets for programs, and it does not approve agencies' strategic plans or performance measures.Links between performance and legislative appropriations vary. For the most part, these three states have not directly tied the size of appropriations to past performance. But beginning in fiscal year 1998, Texas law authorizes agencies to provide financial "rewards" to their employees if the agencies meet 80 percent of their performance targets and when 80 percent of measures are determined accurate by the state auditor. However, no additional funding is appropriated to the agencies for this purpose. Minnesota and Florida have developed these kinds of direct links between performance and funding on a very limited basis-mainly in higher education. Florida has used both financial and non-financial incentives and disincentives based on agency past performance. Incentives have included additional funding and flexibility in use of appropriations. For the most part, these three states have not directly tied the size of appropriations to past performance. But beginning in fiscal year 1998, Texas law authorizes agencies to provide financial "rewards" to their employees if the agencies meet 80 percent of their performance targets and when 80 percent of measures are determined accurate by the state auditor. However, no additional funding is appropriated to the agencies for this purpose. Minnesota and Florida have developed these kinds of direct links between performance and funding on a very limited basis-mainly in higher education. Florida has used both financial and non-financial incentives and disincentives based on agency past performance. Incentives have included additional funding and flexibility in use of appropriations.Reporting of performance information is the responsibility of state agencies, legislative agencies or both. In Minnesota, performance information presented to the legislature is solely the product of state agencies. In Texas, the legislative budget board examines agency performance information and twice annually reports to the legislature on performance. In Florida, agencies report performance information to the governor and legislature. In addition, the legislative Office of Program Policy Analysis and Government Accountability uses an Internet tool for reporting on performance (FGAR) including descriptions of all agency programs. In Minnesota, performance information presented to the legislature is solely the product of state agencies. In Texas, the legislative budget board examines agency performance information and twice annually reports to the legislature on performance. In Florida, agencies report performance information to the governor and legislature. In addition, the legislative Office of Program Policy Analysis and Government Accountability uses an Internet tool for reporting on performance (FGAR) including descriptions of all agency programs.Lessons Learned Executive and legislative leadership is critical. For effective performance-based budgeting, executive branch leaders must be able to articulate to staff (and foster agency discussions) about why performance budgeting is a useful part of management and decision making. Also, legislators need to be willing to devote time to reviewing agency performance information and to ask performance-related questions. When legislative committees and their staff take the time to discuss agency performance information, it can encourage skeptical agencies to take performance measurement more seriously. For effective performance-based budgeting, executive branch leaders must be able to articulate to staff (and foster agency discussions) about why performance budgeting is a useful part of management and decision making. Also, legislators need to be willing to devote time to reviewing agency performance information and to ask performance-related questions. When legislative committees and their staff take the time to discuss agency performance information, it can encourage skeptical agencies to take performance measurement more seriously.Performance measurement systems take time and hard work to develop. It takes time for people to learn the performance-based budgeting lexicon (outcomes vs. outputs, goals vs. objectives, etc.), and it takes time to develop well-crafted goals, objectives and performance measures. Perhaps most important, it takes time for agencies to collect valid, reliable data-and there will always be some programs for which cost-effective outcome measures cannot be developed. It takes time for people to learn the performance-based budgeting lexicon (outcomes vs. outputs, goals vs. objectives, etc.), and it takes time to develop well-crafted goals, objectives and performance measures. Perhaps most important, it takes time for agencies to collect valid, reliable data-and there will always be some programs for which cost-effective outcome measures cannot be developed.Performance measures should be clearly defined. To ensure consistent measurement of performance by an agency over time, a state should be sure that definitions are explicit and specific. It is also important to have consistent ways of measuring performance in cases where several agencies are reporting on the same measures. In addition, agencies that prepare performance targets should be able to explain the assumptions underlying expected levels of performance. To ensure consistent measurement of performance by an agency over time, a state should be sure that definitions are explicit and specific. It is also important to have consistent ways of measuring performance in cases where several agencies are reporting on the same measures. In addition, agencies that prepare performance targets should be able to explain the assumptions underlying expected levels of performance.States should try to identify "key" performance measures. There are many performance measures that may be useful in agencies' internal management, but some of these measures are not critical to major decisions. Texas has both key and non-key measures, but only the 3,000 key measures are included in the general appropriations act. While Texas recently reduced its total number of measures from 11,000 to 8,000, agencies have indicated that further reductions are needed. Minnesota found that its performance reports were dense and sometimes long (400 pages for one agency), so it required agencies to prepare executive summaries to help legislators focus on key performance issues. There are many performance measures that may be useful in agencies' internal management, but some of these measures are not critical to major decisions. Texas has both key and non-key measures, but only the 3,000 key measures are included in the general appropriations act. While Texas recently reduced its total number of measures from 11,000 to 8,000, agencies have indicated that further reductions are needed. Minnesota found that its performance reports were dense and sometimes long (400 pages for one agency), so it required agencies to prepare executive summaries to help legislators focus on key performance issues.Outcomes are most important, but other types of measures are also useful. States should strive to develop measures that indicate whether important goals are being achieved, but none of the three states have focused their efforts exclusively on program outcomes. In fact, some outcome measures are not cost-effective to collect on a regular basis. In addition, other types of measures (outputs, inputs and efficiency) can provide a useful context for considering program results. Performance information provides a tool for helping people make decisions, but it is important to note that values, state priorities and politics remain an important part of decision making in states that use performance-based budgeting. Performance-based budgeting offers states a way to evaluate state investments, but it may not always provide clear guidance about what is the "right amount" of funding. States should strive to develop measures that indicate whether important goals are being achieved, but none of the three states have focused their efforts exclusively on program outcomes. In fact, some outcome measures are not cost-effective to collect on a regular basis. In addition, other types of measures (outputs, inputs and efficiency) can provide a useful context for considering program results. Performance information provides a tool for helping people make decisions, but it is important to note that values, state priorities and politics remain an important part of decision making in states that use performance-based budgeting. Performance-based budgeting offers states a way to evaluate state investments, but it may not always provide clear guidance about what is the "right amount" of funding.States should address how (if at all) general measures of social well being or quality of life should be reported. For example, should the state education department report information on student achievement, even though the department's direct impact on achievement may be quite limited? In Minnesota, the legislature required this department to produce information on both the department's own performance and on the performance of the state's education system. Likewise, Florida's legislature added public schools to the state's schedule of agencies required to participate in performance-based budgeting. Also, a Florida commission developed benchmarks for assessing the state's general quality-of-life, and users of Florida's FGAR reporting system can access this information in addition to program-related performance measures. In Texas, some of the measures that are not within an agency's control have been categorized as "explanatory" measures. Agencies report the explanatory measures, but they are not held accountable for performance targets on explanatory measures. For example, should the state education department report information on student achievement, even though the department's direct impact on achievement may be quite limited? In Minnesota, the legislature required this department to produce information on both the department's own performance and on the performance of the state's education system. Likewise, Florida's legislature added public schools to the state's schedule of agencies required to participate in performance-based budgeting. Also, a Florida commission developed benchmarks for assessing the state's general quality-of-life, and users of Florida's FGAR reporting system can access this information in addition to program-related performance measures. In Texas, some of the measures that are not within an agency's control have been categorized as "explanatory" measures. Agencies report the explanatory measures, but they are not held accountable for performance targets on explanatory measures.Performance information should be considered in the budget process. Minnesota's performance reporting law divorced agency performance information from agencies' budget proposals, requiring agencies to produce separate performance reports prior to the issuance of the governor's budget. However, a task force of Minnesota executive and legislative branch officials recently concluded that it was a mistake to rely solely on "stand-alone" performance reports and recommended that performance information be presented in the governor's budget. Although performance information can also help legislators make non-budget decisions, the budget document is a focal point of legislative attention, making it a logical repository for performance information. Minnesota's performance reporting law divorced agency performance information from agencies' budget proposals, requiring agencies to produce separate performance reports prior to the issuance of the governor's budget. However, a task force of Minnesota executive and legislative branch officials recently concluded that it was a mistake to rely solely on "stand-alone" performance reports and recommended that performance information be presented in the governor's budget. Although performance information can also help legislators make non-budget decisions, the budget document is a focal point of legislative attention, making it a logical repository for performance information.Future Prospects In Florida, performance-based budgeting is becoming part of the legislative culture. Legislators are becoming accustomed to critiquing and using performance measures. They know the difference between "outcomes" and "outputs," although they are still struggling with how to use this information when making budget decisions. In Texas, both the House and Senate are reviewing the performance-based budgeting system during the interim. Legislators and state agencies were recently surveyed about how the system is working and what changes should be made. These efforts will provide an opportunity to improve the Texas system. In Minnesota, a task force of legislative and executive branch representatives recently recommended changes to Minnesota's performance reporting law. The proposed legislation would reintegrate performance reporting into the budget process. In addition, the legislation would not require the legislative auditor to review each agency's performance information every biennium, although the auditor's office would likely continue to review performance information on a more limited basis. Similarities & Differences Among Texas,
April NLPES Executive Committee MeetingExcerpts from the April 3, 1998, Seattle, Washington Meeting
Executive Committee Membership Mary Noble (CA) announced that Sharon Patnode (KS) had resigned from the executive committee because she had taken a job in the executive branch. It was expected that David Greer (LA) would fill Sharon's vacancy. Mary added that the vice chair vacancy (caused by Craig Monson's (UT) departure) would not be filled until the annual meeting. James Barber (MS) has replaced Craig on the executive committee (By rule, replacements are set according to the next highest vote-getter from the previous election.) Committee Reports Membership and Nominations Committee. Jane Thesing (SC) said that the mailing on nominations and required membership certification would be supplemented by a listserve announcement. Five positions need to be filled. Rosters and nomination statements are due back before May 15 and elections must be completed 30 days before the executive committee's July 20 meeting. Proposed NLPES Brochure. Jane and James presented a draft of an informational brochure on NLPES and its content was discussed. The brochure will be sent to all NLPES members and will be available for distribution at conferences. Without objection Jane, James and Bob Boerner (NCSL) were delegated authority to finalize the brochure, send it to all members, and prepare extra copies for the NCSL annual meeting. Jane and James presented a draft of an informational brochure on NLPES and its content was discussed. The brochure will be sent to all NLPES members and will be available for distribution at conferences. Without objection Jane, James and Bob Boerner (NCSL) were delegated authority to finalize the brochure, send it to all members, and prepare extra copies for the NCSL annual meeting.Communications and Newsletter. Craig Kinton (TX) spoke positively on the newsletter, expanded use of the NLPES listserve and the Web site. Craig cited some potential improvement areas, including putting report announcements into a searchable database and refining the works-in-progress (WIPS) searchable data base. Craig Kinton (TX) spoke positively on the newsletter, expanded use of the NLPES listserve and the Web site. Craig cited some potential improvement areas, including putting report announcements into a searchable database and refining the works-in-progress (WIPS) searchable data base.Public vs. private access to NCSL information. Craig questioned why some NCSL Web information was restricted from public access. Rich Jones (NCSL) said that decisions to limit access to NCSL data predated wide-spread use of Web sites and expressed the erstwhile belief that access should be limited to dues-paying members. Jennifer Noyes (WI) moved that Mary send a letter to the Legislative Staff Coordinating Committee recommending that all NLPES materials handled by NCSL be put in the public domain. The motion carried unanimously. Craig questioned why some NCSL Web information was restricted from public access. Rich Jones (NCSL) said that decisions to limit access to NCSL data predated wide-spread use of Web sites and expressed the erstwhile belief that access should be limited to dues-paying members. Jennifer Noyes (WI) moved that Mary send a letter to the Legislative Staff Coordinating Committee recommending that all NLPES materials handled by NCSL be put in the public domain. The motion carried unanimously. Active management of the NLPES home page. Craig raised a proposal by Gary VanLandingham (FL) to "actively manage" the NLPES home page. Currently, the NLPES Web page is passively managed, meaning that NCSL generally does not update or enhance the site, except upon request. Requests can take several weeks or months to process, depending on NCSL staff demands. Craig raised a proposal by Gary VanLandingham (FL) to "actively manage" the NLPES home page. Currently, the NLPES Web page is passively managed, meaning that NCSL generally does not update or enhance the site, except upon request. Requests can take several weeks or months to process, depending on NCSL staff demands. The Web site could be a more vital resource to NLPES members if it were actively managed. Gary suggested that NLPES members, rather than NCSL, could assume responsibility for maintaining and enhancing the site. This responsibility could be rotated among executive committee member states that have the capacity to perform this task. Several potential enhancements were discussed, as well as linking to NCSL-operated data bases. Gary volunteered OPPAGA to maintain the site. Jennifer moved that OPPAGA be authorized to develop, coordinate, maintain the NLPES Web site through the Spring 1999 meeting; that the NLPES site be provided a button link on the NCSL site; and that the current NLPES site be dropped in lieu of the Florida-maintained site. The motion was seconded by Craig. The motion was approved unanimously. Search-ability of LEGISNET. Craig stated that there are still problems with getting a comprehensive searchable LEGISNET listing of reports. He said he would work with Bob to clear up the problem. Jane suggested contacting states' key contacts to determine whether they can submit electronically formatted reports. Craig stated that there are still problems with getting a comprehensive searchable LEGISNET listing of reports. He said he would work with Bob to clear up the problem. Jane suggested contacting states' key contacts to determine whether they can submit electronically formatted reports. NLPES Listserve. There are now 140 people on listserve. However, about 32 key contacts are not on the list serve. Bob was asked to put all key contacts with e-mail addresses on the listserve. There are now 140 people on listserve. However, about 32 key contacts are not on the list serve. Bob was asked to put all key contacts with e-mail addresses on the listserve. NCSL Annual Meeting. Mary reviewed a draft of the NCSL annual meeting schedule. Monday July 20 afternoon (4:15-5:00) will be the NLPES opening round table/reception. The NLPES business/awards luncheon will be held on Wednesday, July 22 from 12:00 to 1:30. NLPES training sessions will be held on Tuesday, July 21 (8:00-9:30) and Wednesday, July 22 (9:45-11:45). Training sessions will include a Year 2000 discussion panel, correctional issues and a writing training seminar. Mary reviewed a draft of the NCSL annual meeting schedule. Monday July 20 afternoon (4:15-5:00) will be the NLPES opening round table/reception. The NLPES business/awards luncheon will be held on Wednesday, July 22 from 12:00 to 1:30. NLPES training sessions will be held on Tuesday, July 21 (8:00-9:30) and Wednesday, July 22 (9:45-11:45). Training sessions will include a Year 2000 discussion panel, correctional issues and a writing training seminar.NLPES Fall Training Conference. Gary discussed his proposal for rotating NLPES conferences among regions - South, West, Midwest and Northeast. NLPES members in the region states would be asked if they would like to host the conference. If no state volunteers, the executive committee would pick a site, and NCSL would handle the negotiation for hotels and rates, conference registration, handouts and the social activities. Pros and cons of the proposal were discussed. The current problem was lack of a site for 1999, although it was noted that West Virginia had expressed interest. It was agreed without objection that Art Heikkila (AZ) would pursue West Virginia's interest in the holding the 1999 conference. If West Virginia is unable, a state in the Northeast or Midwest will be sought. Gary discussed his proposal for rotating NLPES conferences among regions - South, West, Midwest and Northeast. NLPES members in the region states would be asked if they would like to host the conference. If no state volunteers, the executive committee would pick a site, and NCSL would handle the negotiation for hotels and rates, conference registration, handouts and the social activities. Pros and cons of the proposal were discussed. The current problem was lack of a site for 1999, although it was noted that West Virginia had expressed interest. It was agreed without objection that Art Heikkila (AZ) would pursue West Virginia's interest in the holding the 1999 conference. If West Virginia is unable, a state in the Northeast or Midwest will be sought.Art mentioned that the Fall 1998 conference will be held at the Red Lion Sacramento Inn. The theme will be "Keeping Government on Track". Mary estimated 200 to 250 attendees. Mary asked that session ideas be submitted to Art or Elaine Howell (CA). Kirk Jonas (VA) said he would seek Tor Dahl as a keynote speaker. Shared Training Manual. Joel Alter (MN) discussed the shared training manual survey to be conducted among NLPES members. Joel said he hoped the resource would be available by the summer or fall. Joel Alter (MN) discussed the shared training manual survey to be conducted among NLPES members. Joel said he hoped the resource would be available by the summer or fall.NLPES Awards. Gary mentioned that Mary Goehring (TX) has been added to the Impact award committee. Deadline for applications is May 15. Art suggested adding a "career achievement" award to recognize individual contributions. The awards committee will consider criteria for such an award. Gary mentioned that Mary Goehring (TX) has been added to the Impact award committee. Deadline for applications is May 15. Art suggested adding a "career achievement" award to recognize individual contributions. The awards committee will consider criteria for such an award.Professional Liaison. Joel said there is a need for a liaison with American Society of Public Administration (ASPA). Currently, no member of the executive committee is active in ASPA. It was mentioned that links to liaison organizations will be established soon on the NLPES home page. Joel said there is a need for a liaison with American Society of Public Administration (ASPA). Currently, no member of the executive committee is active in ASPA. It was mentioned that links to liaison organizations will be established soon on the NLPES home page.AEA Journal. Kirk said that the NLPES/AEA Journal project is proceeding smoothly. Kirk said that the NLPES/AEA Journal project is proceeding smoothly.Other Business Bob stated that the Skills Development Seminar is on track. Mary suggested that Jim Pelligrini and Max Arinder continue to do the program evaluation section. Phil Leone made a presentation on the Advisory Council on Government Auditing Standards. Phil said 80 percent of the council's time focused on issues of financial auditing, internal controls, etc. He said one issue of interest was how to handle early warning of financial problems, such as Orange County, California. There are two issues the council is beginning to grapple with: (1) What role auditors should play in the development of performance measures. (2) How performance auditing standards should be modified. Accounting firms are really pushing for yellow book performance auditing standards, he said. Phil invited comments on issues people want the council to address. Web TrappingsUsing Listserves as a Research Tool(or, how to get others to do your fieldwork) Although the Internet can provide program evaluators with valuable information, web-based research can often be extremely frustrating and time-consuming. To overcome this problem, many people join listserves (or, listserv's). A listserve is an electronic discussion group in which participants e-mail questions and comments to a central address that then forwards the discourse to all listserve members. The people on listserves usually have a common interest since all the messages revolve around a certain topic. There is a listserve for just about every conceivable subject, and more than 85,000 in all. There are two basic types of listserves. One is the discussion group format in which individuals who have joined the listserve can post messages to one another over the Internet. The NLPES listserve falls into this category. The second type of listserve is a newsletter or announcement format in which a single writer (the list owner or moderator) broadcasts an e-mail to those who have joined, and the audience does not participate directly. Both formats are excellent resources for program evaluators and allow individuals to share what they have learned in an efficient manner. As we all know, getting good information handed to you sure beats having to spend long hours digging it up yourself. Anyone with access to e-mail can join any of the many listserves available. All you have to do to get involved in a list is subscribe to it by sending an e-mail to the list subscription e-mail address with the proper command. Your e-mail address will be added to the list, and you will start receiving discussion contributions from other list members. You may reply to these messages or start new discussions at any time. You don't have to actively participate by sending messages; you can just listen to the discussion. Listserves offer the ability for the extremely busy evaluator to gain topical information and ideas, and share experiences with others in the field. If you have a question, you can send it out and, most likely, receive an answer from someone on the list. When they are used effectively, listserves can get you the right amount of information in the least amount of time and least cost. Listserves are also convenient because all the messages go directly to the member's e-mail account. A problem with listserves that you need to be aware of is that some will send you far more information than you ever wished to receive, with much of it utterly useless (a.k.a. "spam"). For example, some listserves that rely heavily on discussion groups can deluge you with announcements of the birth of kittens in other participants' households. If you aren't careful, you can find your virtual mailbox crammed with virtual junk mail. Consequently, (1) don't even think about joining all 85,000 groups; and (2) quickly cancel your registration with any listserve that isn't providing you with usable information. Two listserves that you will definitely want to join are the National Conference of State Legislatures (NCSL) listserve, and our very own NLPES listserve. Information regarding the NCSL listserve can be found here and see below for the NLPES listserve. A good directory of Listserves can be found at http://www.liszt.com/. This site has a comprehensive directory of Listserves organized by general categories. It also has a search function that will allow you to search through 85,714 groups. Good Web sites that provide information on what listserves entail, how to create a listserve, and directories of listserves that can be searched can be found at http://www.fau.edu/~rinaldi/net/dis.html, http://www.webcom.com/impulse/list.html, and http://www.lsoft.com/listserv.stm . So, surf's up! Join the crowd at the listserve beach, but watch out for the spam waves. The NLPES ListserveWell over 10 percent of our NLPES membership is now "enlisted" on our NLPES Listserve, and that number is growing. More offices are announcing their latest report offerings and there have been numerous people "efficiently and effectively" gathering the basic information necessary to more quickly advance their own research on the topics of the day. And speaking about efficient and effective, the NLPES listserve has been relatively free of "party line" banter-thanks to you; however, please remember:
Using the NLPES Listserve (1) Address an e-mail message to listserv@ncsl.org. (2) Type in the message area: Subscribe NLPES-L Firstname Lastname. (3) Wait for confirmation of your subscription-usually within 24 hours. (4) Send a message to all NLPES subscribers by e-mailing to: NLPES-L@ncsl.org.
NCSLnet: The Electronic Information Network for State Legislators and Staff
Many of you already use NCSL's public Web site (www.ncsl.org) to access a variety of materials. While the public homepage holds a great deal of information, the bulk of our online documents and our most complete services can be accessed only through the password-controlled members' area for legislators and legislative staff. This is an exciting way for legislators and staff to find the most current information on any policy issue. These exclusive resources include:
In the coming months, we will be "filling in" this framework of services, adding new kinds of services available only through password-controlled access, including:
WIPS UpdateThe Works in Progress (WIPS) records in the new Web on-line directory have been updated and loaded. You can go directly to the WIPS directory or you can access it through the NLPES homepage. If you have any questions or need assistance, contact Bob Boerner at (303) 364-7700, fax (303) 863-8003. Thank you NLPES key contacts for your assistance in getting the directory completed.
News MakersGeorgia New Management Analyst: Margaret MacDonald, currently studying for her doctorate in public administration. Margaret MacDonald, currently studying for her doctorate in public administration.Hawaii Re-engineering Faux Pax: The state auditor's first assessment of a major "re-engineering attempt," dual management and financial audits of the state library's complete privatization of selection and acquisition of materials, revealed it to be a complete flop. This innovative effort was being closely followed by librarians nationwide. The Hawaii state librarian's position will be open as of June 30, 1998. The state auditor's first assessment of a major "re-engineering attempt," dual management and financial audits of the state library's complete privatization of selection and acquisition of materials, revealed it to be a complete flop. This innovative effort was being closely followed by librarians nationwide. The Hawaii state librarian's position will be open as of June 30, 1998.Idaho New Staff: Eric Milstead, former fiscal analyst with the Kansas Legislative Research Department. Eric Milstead, former fiscal analyst with the Kansas Legislative Research Department.Kansas Moving On: Sharon Patnode, leaving the Legislative Post Audit Committee to become the assistant to the director for the Division of Health, Kansas Department of Health and Environment Sharon Patnode, leaving the Legislative Post Audit Committee to become the assistant to the director for the Division of Health, Kansas Department of Health and EnvironmentMinnesota New Staff: John Patterson, formerly with the Congressional Budget Office.Sleepless Nights: Executive committee member Joel Alter's long nights can be attributed to his new monitoring and oversight responsibility-of his new daughter Mary Clare (Feb. 17). Congrats, Joel! Executive committee member Joel Alter's long nights can be attributed to his new monitoring and oversight responsibility-of his new daughter Mary Clare (Feb. 17). Congrats, Joel!Ohio Work Inflation: Ordinarily assigned studies by its legislative education oversight committee, the oversight office was recently provided a dozen new study requests based on other legislative actions within the past year. Nice to be noticed...?Oklahoma New Fiscal Analysts: David Blatt, Appropriations Subcommittee on Health & Social Service; Robb Gray, Appropriations Subcommittee on Natural Resources & Regulatory Services.Wisconsin Moving On: Jennifer Noyes, leaving the Legislative Audit Bureau to become deputy director for a newly created executive branch office of program evaluation.NLPES CalendarDeadlines NLPES membership certification; executive committee nominations; and awards entry submission. NLPES membership certification; executive committee nominations; and awards entry submission.JULY 1998 July 20-23 (Monday-Thursday)NCSL Annual Meeting, Las Vegas, Nev. AUGUST 1998 August 3-14 (Monday-Friday)Legislative Staff Management Institute August 19-22 (Wednesday-Saturday)Skills Development Seminar Contact Bruce Feustel SEPTEMBER 1998 September 30- October 3 (Wednesday-Saturday) NLPES Training Conference Contact Bob Boerner From the EditorMy thanks to the Indiana General Assembly for ending its session two weeks early, providing the time to allow this issue to firmly gel! Appreciative nods to my regular cadre of assistants, Kristin Breen, Debra Hollon and Phyllis McCormack. This issue's content was particularly well done. If you think so too, please refer your appreciation/comments to the authors. They already have my thanks, but peer recognition goes further! Lastly, take advantage of NLPES! The work of our executive committee is only as good as the feedback it receives from you. Take a minute now to "unbury" yourself, and do one Quadrant II activity. Fill in the statement, "NLPES should do _____________ for us." Then send it! Send it to the Listserve, to Mary Noble or to me, but get it off your chest!! We like squeaky wheels, and remember, "Evaluation isn't everything, but performance is!" Small Shop Talk-Take OneWelcome to our new column focusing on issues that small evaluation offices face. We hope this column will become a regularly appearing feature of NLPES NEWS-at least until small shop issues or column authors are exhausted! Small Shop Issues Are Large . . . Why a column on "small shops?" Well, for one reason, survey results printed in the September 1997 NLPES NEWS A View From the States showed that 10 of the 47 NLPES member organizations that responded to the survey had 10 or fewer staff. That's at least one-fifth of survey respondents that must deal with certain common issues whose impact is magnified due to the small size of the office. Other evaluation shops with more staff may also be "small" in their states, and they too face some of these same issues, although perhaps to a lesser degree. Then, too, efficient ways of exchanging information are especially important in small shops. When resource pressures are acute, new and better ways of doing what we do become a matter of survival, not only improvement. Small Shop Distinctions So what are the issues that set small shops apart from big or medium-sized shops? Here is a taste of the differences to be explored: Requirements of Staff. Evaluations are conducted with fewer total resources. With a small number, staff typically cannot specialize in a given policy area, increasing evaluation "start-up" time. Also, staff must fulfill a number of different responsibilities-from interviewing to legal research, quantitative analysis to writing-during the course of a single evaluation. Managers in these offices often must contribute materially to the evaluation process in addition to fulfilling administrative responsibilities. Evaluations are conducted with fewer total resources. With a small number, staff typically cannot specialize in a given policy area, increasing evaluation "start-up" time. Also, staff must fulfill a number of different responsibilities-from interviewing to legal research, quantitative analysis to writing-during the course of a single evaluation. Managers in these offices often must contribute materially to the evaluation process in addition to fulfilling administrative responsibilities.Relations with Outside Resources. Because fewer staff translate into fewer skills in-house, small shops may be more reliant on outside expertise, adding a unique element of instability. Assistance in methodological design, editing, legal analysis, as well as project-specific expertise are all needed at times. However, staff can expend an extraordinary amount of effort in recruiting, orienting and managing short-term resources. A seemingly constant "training mode" taxes permanent office staff as they revisit office standard operating procedures with each new "expert."Types and Levels of Pressure. Small evaluation offices may experience some forms of work pressure more acutely than larger offices, including the following: Small evaluation offices may experience some forms of work pressure more acutely than larger offices, including the following:
Personnel Growth. How may a staff be structured to provide the opportunity for growth and promote teamwork in a small and flat organization? How do managers counter "interpersonal burnout" when staff must work with the same people on each project? How may a staff be structured to provide the opportunity for growth and promote teamwork in a small and flat organization? How do managers counter "interpersonal burnout" when staff must work with the same people on each project?Expectations of the Legislature. In some ways, small shops face increased pressure to demonstrate the value of their work. The Legislature is less likely to see a small evaluation shop as a critical function of government. After all, wouldn't the office be larger if it were really necessary? In some ways, small shops face increased pressure to demonstrate the value of their work. The Legislature is less likely to see a small evaluation shop as a critical function of government. After all, wouldn't the office be larger if it were really necessary?Your "Small Shop" Stories
|
© 2008 National Conference of State Legislatures, All Rights Reserved
Denver Office: Tel: 303-364-7700 | Fax: 303-364-7800 | 7700 East First Place | Denver, CO 80230 | Map
Washington Office: Tel: 202-624-5400 | Fax: 202-737-1069 | 444 North Capitol Street, N.W., Suite 515 | Washington, D.C. 20001
