 |
Home | Contact Us | Press Room | Site Overview | Help | Login | Register |
 |
 |
| About NCSL | State & Federal Issues | Legislatures | Legislative Staff | Meetings | Bookstore | Legislators & Staff Only |
| NCSL Home > Legislative Staff: Staff Sections & Networks > NLPES > | Add to MyNCSL |
NLPES NewsMay 1997, No.67 Inside the NewsChair's Corner Chair's CornerJennifer Noyes (WI) 
If my recent telephone conversations with some of you are any indication, it would seem that we are all so busy keeping up with the demands of the current legislative session in our respective states that we have little time to think of anything else. Just remember, as you strive to complete your work as effectively and efficiently as possible, your NLPES colleagues may be in a position to help you avoid reinventing an audit wheel. One fast and easy way to ask for this help is to use the electronic mail discussion group established by NCSL for NLPES. If you are not sure how to do this, please refer to the List Server instructions of this issue of the newsletter. Given how busy we all are, setting aside time for training may seem like a luxury. I would argue that training should be thought of as a necessity, rather than a luxury, and an effort should be made to take advantage of training opportunities whenever possible. That's why I am asking you now to set aside October 8-11, 1997 for our Annual Training Conference. October may seem like the distant future, but, as we all know, time has a way of flying by when we're not looking! This year's conference will be hosted by the Mississippi Joint Committee on Performance Evaluation and Expenditure Review (PEER) staff in Jackson, Miss. I am happy to report that the Legal Services Staff Section (LSSS) will be joining us in Jackson. Now, I bet you are wondering what NLPES and LSSS could possibly have in common. Actually, we have a lot more in common than one would think. Foremost, we share the common goals of improving the effectiveness and knowledge of legislative staff through the promotion and exchange of ideas and information. We also have other common interests. We are, for example, all interested in such things as what happens when our need to access information collides with an entity's (such as a foundation) interest in holding information in confidence; what rights the Legislature has to subpoena witnesses; and whether our work is "discoverable." These topics, as well as many others, will be addressed during the conference. Given the combined talents and interests of our two staff sections, I have no doubt that the training conference will be interesting and will provide all of us with a unique opportunity to get to know one another a bit better. Finally, I would like to extend all of our thanks and best wishes to Jim Carpenter, immediate past chair of NLPES. Jim retired on February 28 after many years of service to the Florida Legislature. Jim made many contributions to NLPES and NCSL over the years. He will be missed. Jim shares some of his evaluation perspectives, gained over the years in this issue. I look forward to seeing you at the Annual NCSL Meeting in Philadelphia in August. 
Welcome Bob Boerner, New NLPES Staff Liaison!A Policy Specialist in the NCSL Legislative Information Services/Systems Program, Bob covers more than 30 topic areas, specializing in telecommunications policy, civil law issues, and state government reform. With a degree in political science and a J.D., Bob performs legal data base searches as requested by the entire staff of the Conference, using Lexis/Nexis, Westlaw, and other Internet sources. ILLINOIS PROFILE History: The Illinois Office of the Auditor General was established by the Illinois Constitutional Convention of 1970 and statutorily created by the Illinois State Auditing Act in 1973. Before the establishment of the auditor general, there was an auditor of public accounts.Office Overview: The auditor general is appointed to a 10-year term by the General Assembly. All audits are submitted to the Legislative Audit Commission which is a 12-member bipartisan committee.Number of Audit Staff: The Performance Audit Division has 19 staff. The Financial/Compliance Audit Division has 27 staff, and the Information Systems Audit Division has 12 staff.Primary Software Used: Windows 95, Microsoft Office (Word, Excel, PowerPoint and Access) and GroupWise E-Mail. The Office also has its own intranet that uses Netscape.Staff Backgrounds: Accounting, Business Administration, Computer Science, Economics, Education, English, Finance, Health Services Administration, Law, Mathematics, Personnel Management, Political Science, Public Administration, Public Policy, and Statistics.Recent Performance Audits and Studies: (For a complete listing of performance audits, visit our Web site.)Unemployment at State Universities: We identified more than $43,000 in questionable unemployment benefits paid by four state universities; nine employees continued to receive unemployment benefits even after they returned to work. In light of this finding, we examined the system used by the Department of Employment Security to detect inappropriate claims. In 20 percent of the claims reviewed, employees were working at the same time they were receiving unemployment benefits charged to state accounts. The department agreed to review the criteria used by their detection system to identify questionable unemployment payments.Administrative Laws: Officials in 100 agencies reviewed 12,437 statutory provisions and identified 912 as being obsolete, unenforceable, unnecessarily burdensome, duplicative, or of no administrative benefit.Examples included a 1953 law that regulated the use of mules in Illinois coal mines and laws establishing various councils and commissions which have not met for 10 or 15 years. Legislative hearings were held on these laws and a number were subsequently repealed. Child Support Enforcement: We concluded that the present process used to establish and collect child support was lengthy and complex, with as many as 12 entities involved in the process. Less than a fifth of the cases sampled resulted in a provision to collect support. For cases with support orders, noncustodial parents were two months or more behind on payments for 64 percent of the orders reviewed. The audit made several recommendations to improve the effectiveness of the state child support enforcement system.Office Locations: Springfield and ChicagoChicago Sites and Attractions: The Sears Tower, Shedd Aquarium, Adler Planetarium, Museum of Science and Industry, Chicago Art Institute, and the Chicago Bulls.Springfield Sites and Attractions: Lincoln Home, Frank Lloyd Wright, Dana Thomas House, Old State Capitol, and the Horseshoe Sandwich. Springfield was recently named second friendliest city in the United States.Liaison ReviewJoint Committee on Standards for Educational Evaluation Legislative oversight staff members interested in a set of evaluation standards developed by a national group of evaluators now have a direct link to the committee responsible for developing, publishing, and updating those standards. NLPES recently joined 15 other sponsoring organizations as a voting member of the Joint Committee on Standards for Educational Evaluation. The joint committee, which was established in 1975, is a coalition of professional associations dedicated to the use of evaluation standards to improve evaluation practice. The joint committee currently is disseminating the 1994 edition of the Program Evaluation Standards (sometimes referred to as the "Red Book"). Evaluation training using the Program Evaluation Standards is now taking place in North America and throughout the world. The standards are used in settings related to K-12 schooling and higher education, as well as in a broad range of other settings, such as the U. S. Marine Corps, health professionals, large corporations, and several legislative oversight offices. NLPES members interested in learning more about the program evaluation standards published by the joint committee can order a copy of the Program Evaluation Standards: How to Assess Evaluations of Educational Programs from Sage Publications Inc., 2455 Teller Road, Thousand Oaks, CA 91320 or by calling (805) 499-0721. Also, training workshops on use of the standards are being scheduled nationally. The joint committee has agreed to schedule training at the NLPES Training Conference to be held in Mississippi in October 1997. Dr. James Sanders, chair of the Joint Committee on Standards for Educational Evaluation, invites NLPES member agencies to contact him about their training needs related to the standards at (616) 387-5895. Sanders recently provided in-depth training on the program evaluation standards for the Florida Legislature's Office of Program Policy Analysis and Government Accountability (OPPAGA). According to John Turcotte, OPPAGA's director, "Sanders' training session was on-target for legislative oversight staff. Jim was able to explain the standards in detail and relate them to field experiences that our analysts encounter." He added that he considers the joint committee standards relevant to OPPAGA's approach to legislative oversight. "We have found the joint committee standards more useful in our type of work than more accounting-oriented standards," he said. Because evaluators in legislative and other settings need additional training material to better apply the standards in fields other than education, several members of the joint committee have expressed interest in developing a casebook illustrating application of the standards in nonschool settings. The joint committee has agreed that if a group of associations were to develop a casebook, the joint committee would consider endorsing the product as a companion publication supplementing the 1994 standards publication. At its meeting of Oct 8, 1996, the NLPES Executive Committee agreed to participate in developing a casebook. See the insert on this page for details on the casebook and how NLPES members can help in this effort. In addition to disseminating the Program Evaluation Standards, the joint committee is revising the 1988 edition of the Personnel Evaluation Standards and initiating a new project to develop student evaluation standards. Contact Kathleen Sullivan at (601) 359-1226) for more information on either of those projects. NLPES Member Contributions NeededThe Program Evaluation Standards, a 1994 publication, presents illustrative cases about each of the Joint Committee's evaluation standards. Each case example includes a description of an education or training-related evaluation project in which a situation relevant to the standard arose. The case description is followed by an insightful analysis of the application of the standard to the illustrative case. Although these cases lend a great deal to the quality of the publication, the education and training focus of cases in the Program Evaluation Standards to some extent limits the utility of the publication in guiding the wide range of evaluation practice outside the field of education. Because many evaluators agree that the standards themselves are broadly applicable beyond the education and training settings of the case examples, a supplementary casebook applying the standards to a wider variety of evaluation settings would broaden the standards' utility for program evaluators. The proposed evaluation casebook will present case descriptions and analyses illustrating application of the joint committee standards in nonschool settings. Contributions will be sought from evaluators who practice in a variety of settings, including legislative oversight offices, and who evaluate programs on a variety of subjects. NLPES members who are willing to help in developing the casebook are asked to contact Kathleen Sullivan, Mississippi Joint Legislative Committee on Performance Evaluation and Expenditure Review (601) 359-1226. Volunteers may be asked to contribute a brief (200-word) sample case, write a case analysis, or review case descriptions and analyses. The time, talent, and evaluation experience of NLPES members are needed to ensure the success of this project.Advisory Council on Government Auditing Standards
The initial meeting of the 20-member Advisory Council on Government Auditing Standards took place in Washington, D.C., on Feb 10, 1997. Council members represent federal, state, and local governments, CPA firms, academia, and industry. After being sworn-in, members participated in a full day of discussions on the standard-setting process to be followed for future revisions of Government Auditing Standards. The standards were last issued in 1994, and are applicable to both financial and performance auditing. Council members were appointed last fall by the Comptroller General and will be involved in the identification, development, and issuance of revisions to the standards as well as guidance on their application. The broad phases of the standard setting process will include: issue identification; At this time, specifics on the issuance of the next "Yellow Book" have not been decided; however, it is possible that the council may recommend separately issued documents with a periodic codification of the standards. At least three issues were identified by the council that should be of interest to NLPES member organizations conducting performance audits in accordance with the standards. The first issue relates to whether more guidance is needed relating to the audit of performance measures. Discussion centered on when work relating to the design, development, implementation, and verification of performance measures should be considered an audit as opposed to some other type of work. Second, the council recognized that, increasingly, governmental entities are either establishing internal performance audit units, or they are contracting for performance audits or evaluations of entity programs, activities or functions. As more and more consulting firms and audit firms move into this field, there may be a need to identify situations when it would be advantageous for the governing body should require such work to be conducted in accordance with the standards. Third, GAO staff identified the issue of follow-up and requested council input as to whether, for all types of audits, auditors should be required to follow-up and report on the status of all findings and recommendations until they are effectively resolved, implemented or otherwise closed. It was the opinion of several council members familiar with performance audits that such follow-up activities should not be required. The reasoning was that performance auditors do not go back to the same program year after year as is the case with financial statement auditors, and corrective actions or program changes may not be evident in a program, in some instances, for several years. The council unanimously agreed that follow-up should not be a requirement in performance audits. In those instances where a follow-up audit is performed on the same program, the auditor should review those outstanding issues that are significant to the current audit objectives. The next council meeting is scheduled for Aug 11, 1997. All NLPES members on the council would value any comments you may have on issues that need to be addressed by the council. NLPES members on the council include Dan Kyle (LA), Phil Leone (VA), Kurt Sjoberg (CA) and Sam McCall (FL). NLPES AwardsPaul Bernard (GA) The NLPES Executive Committee is once again seeking applications for awards designed to recognize outstanding contributions to the field of legislative program evaluation. The two awards that NLPES has traditionally presented--the Award for Excellence in Program Evaluation and the Award for Excellence in Research Methodology--will, for the first time, be presented by NCSL as part of its new Legislative Staff Achievement Award Program. Another "first" is the establishment of a third award--the Impact Award--designed to recognize reports whose release have had a significant impact in any of several areas . Information about the awards, including information about how to apply for each award and the criteria used in evaluating award applications, was mailed to all key contacts the week of April 21. The deadline for submitting applications is May 15. The following reflects the general criteria to be used in evaluating submissions for each award. Award for Excellence in Program Evaluation The application for this award should include a letter and other supporting material that describe how your agency meets the above criteria. The supporting material should include two or three of your agency's best reports released in calendar year 1996. Award for Excellence in Research Methodology The application for this award should include one report released in calendar year 1996 and a brief statement that includes at least the following:
Impact Award The application for this award should include a copy of a report and a letter with supporting materials demonstrating the impact the report has had during your state's last fiscal year. In recognition of the fact that the effects of a report may often not be evident until several years after a report has been released, the report itself does not have to have been released during the previous fiscal year. The following are examples of the types of effects or impacts a report could have had during a given year and examples of how to demonstrate these effects or impacts. You can choose to highlight whatever impact you feel is the most appropriate for your report. Only one report should be submitted per office. A report must have made a demonstrable impact, such as one or more of the following:
For additional information, please contact either Paul Bernard at (404) 651-8855 or Craig Monson at (801) 538-1033. Where to begin your research?The NLPES List Server is up and running. An e-mail registration site for NLPES members, this List Server allows a message to be sent to all NLPES e-mail subscribers. You and your office can participate by posting "events" you think would be important to the membership (e.g., report publications and executive summaries, new hires, etc.), philosophical audit questions and ideas (when a muse captures a bit of your scant spare time...), and, best of all, questions regarding audits. Two auditors who ventured to use this new service asked for information on research into a specific area and were rewarded for their efforts. Don Bezruki (WI) requested information on "evaluating economic development programs." He received responses from NLPES members in Nebraska, Georgia, Mississippi and Virginia. Scott Owens (GA), on a suggestion by his division director, asked for information on auditing "educational incentive programs" and received a large list of resource contacts. Members from 14 organizations are currently registered on the NLPES List Server. As more members become familiar with using the Internet, more uses may be found for this interactive exchange among members. But, for now, tapping into other states' audit experiences will save you time and, quite possibly, save you from reinventing the audit wheel. 
A View from the StatesThe Evolution of Program Evaluation -- A Perspective
Editor's Note: Jim Carpenter recently retired from program evaluation work after 14 years with the Florida Auditor General Office. Jim can be reached by e-mail for scouting reports on FSU baseball and the NCAA tournament.Jim Carpenter: Mark asked me if I would write this article on my perspective of evaluation, based on some questions he posed. Here are some of the answers I thought you would enjoy.How has the field of program evaluation changed since you became involved? Evaluation has changed very dramatically in Florida. When I first started with the Auditor General's Office in 1982, the reports were very dry and limited in their message -- very "audit oriented". Through the years the format of reports changed, and the subject matter became more management and policy oriented. We also began making recommendations with policy orientation, (which made the auditor general a little uncomfortable) because the Legislature wanted to make use of an outside entity to initiate change. So to reach that objective, the Office of Program Policy Analysis and Government Accountability or OPPAGA was recently created. With the advent of OPPAGA, our reports continued to change as we tried to guess what our audience wanted. We shortened our long reports to four, six, eight, or, at most, 12 pages. We received good feedback from many legislative staff and worked on other staff whose lingering impression of us was formed upon "long reports that were not responsive to needs". Our audience continued to change, becoming more "budget and money" oriented, and we could hardly change fast enough. Our profession, overall, is undergoing rapid change as legislative leaders find newer ways to use the professional, unbiased information we can provide to them. What developments in the "art of managing government," in the evaluation sense, have you found interesting, e.g., zero-based budgeting, 1-800 fraud/abuse lines, etc? Zero-based budgeting was a very easy sell to the legislature. They wanted to examine the base, or they thought they wanted to. I remember trying zero-based budgeting in the late 1970s. The few agencies that cooperated and tried it were overcome with paper. The decision packets that traveled with the proposed budget, as it made its way through the process, were enormous. Now we have performance-based budgeting, and in Florida it has become a constant battle between budgeters and agency administrators in defining various programs. Since programs are budgeted on a lump-sum basis, agencies are simply after as few programs as possible. Budgeters, and often evaluators, in an effort to understand and set program objectives, are after programs with more defined goals, objectives and measures. Describe the enjoyable aspects of program evaluation you've encountered. What I enjoyed the most was the opportunity to get out into the field and be an evaluator -- to be able to find out what was really going on. One of my most memorable visits was to one of our school district offices to interview a school board member. On this occasion, we were interested in how the school board was implementing a school improvement effort. This plan was an effort to empower local school parents and teachers to seek needed changes. In the interview it became obvious that the school board member was unaware of any statewide program, much less how it was being implemented in his school district. In visiting schools ourselves, we became the catalyst for change by making parents and teachers aware of just what could be done through the school improvement process. We became conduits for change and advocates for school improvement. Describe the role NLPES has played in your career, and predict how the field of government program evaluation may change in the future. NLPES has played an important role in my career. I can remember the first NLPES meeting I attended in Seattle in the mid-1980s. As a professional organization, NLPES offers a great deal to the evaluation practitioners: training; sharing with professionals in similar and related disciplines; the opportunity to discuss unique evaluation problems; and finding ways to approach difficult evaluations. Most of all, it provides us with a unique identity among our fellow evaluators around the country. We must keep our field broad enough to encompass auditors and evaluators. Do not be surprised as more auditor shops become evaluation offices, and evaluation offices will be required to recommend more policy options to decision makers. The future for program evaluation is tremendous, but it is our responsibility to ensure that our recommendations are founded on firm ground and reflect our best judgments -- not political expediency. If our respective shops can carry this out, we will serve our constituency well. NLPES Executive Committee Meeting OverviewSan Francisco, California
General Announcements Tom Bardin announced his resignation from the Executive Committee to accept a position outside of legislative work. After discussion, the committee voted to offer Tom's seat to Art Heikkila (AZ). NCSL's Bob Boerner was introduced as the new NLPES liaison. Karen Carter, our former liaison, left NCSL to take a position with the state of Washington. Subcommittee Reports NLPES/AEA Journal Jennifer Noyes reported that Kirk Jonas (VA) says that article submissions are on track in terms of quantity and publication deadline. Kirk is spearheading this project. (Refer to the January NLPES News for more information on this project.) NLPES Awards NLPES will award a Methodology Award and Excellence Award this year. The awards will be presented by NCSL as part of the new Legislative Staff Achievement Awards program. NLPES is instituting a new award program designed to recognize reports that have had a significant impact. Paul Bernard is preparing applications for all three NLPES awards and will contact all key contacts regarding the awards. By-Laws The committee discussed service on the Executive Committee by nonlegislative branch staff. After review of the By-Laws and material provided by Bruce Prenda, it was concluded that nonlegislative branch staff could not serve as members of the Executive Committee, but could still be affiliated with NLPES. Communication/Newsletter The next NLPES NEWS will be released after the annual meeting in August. Mark Bucherl is working on standardizing the format of the newsletter. Mary Noble reiterated that only each state's key contacts will be receiving the newsletter, with additional copies to be made by each state. Craig Kinton discussed concerns with updating information on the NCSL Web site. Max Arinder raised concerns about access to LEGISNET. After a discussion of these and other issues related to on-line services, it was agreed that Craig would write a letter to the relevant NCSL staff about the committee's concerns. It was also agreed that: (1) the committee would strive to make more use of the NLPES list server member subscription function; (2) Craig and Mark will initiate the production of a Guide to Legislative Program Evaluation in conjunction with the Fiscal, Oversight, and Intergovernmental Affairs Committee of the Assembly on State Issues; and (3) training abstracts from NLPES conference sessions will be prepared and communicated through the newsletter and through the NLPES list server. Membership and Nominations Jennifer, on behalf of Joe Fiala (KY), presented the time line for electing individuals to the Executive Committee. Professional Liaison David asked the committee to consider nominations of individuals to serve as professional liaisons as well as comments on expectations on liaison responsibilities. Jennifer reviewed Sam McCall's (FL) recent letter regarding issues under consideration by the Governmental Accounting Standards Board (GASB), including GASB's recently issued exposure draft No.142-B. Training Mary presented proposed criteria for site selection for fall NLPES training conferences, which were approved. She also asked for committee consideration of the uncertain future for an NLPES Training Handbook, whose intent is to provide easy access of training, materials, and possibly instructors from states that have developed specific forms of training. The committee recommended that Mary's subcommittee again attempt to contact states that have not responded. Other Business Fall Training The committee discussed NCSL's budgetary process for NLPES training conferences and our staff section's consistent success in maintaining a positive revenue balance for its conferences. Jennifer will draft a letter to NCSL drawing attention to this success and requesting a clearer understanding of NCSL's budget planning process. Committee members related their discussions with state key contacts regarding the fall training and concluded that additional comments be sent to Max. Max reviewed progress on the next NLPES fall training conference, which will be held jointly with the NCSL Legal Services Staff Section. It is anticipated that the combination of both sections will attract ample participation. Max discussed the planned training tracks and activities and reported that Michael Scriven will be a keynote speaker and will conduct a session on critical thinking. Also invited is William Winner, former governor of Mississippi. Max asked that suggestions for one more keynote speaker be passed to him. NCSL Annual Meeting Five staff section meeting times (in addition to the luncheon) have been blocked out at the NCSL Annual Meeting. Mary Noble will be responsible for overall coordination of the Annual Meeting sessions. One of these five will be used by the new Executive Committee for its first meeting. The committee, based on information provided to it by key contacts and others, selected the following four topics for these sessions, with responsibility for organizing the sessions assigned to one member:
Yellow Book Changes Phil Leone briefed the committee on two proposed changes to the Yellow Book involving information technology and a performance standards revision. Any comments or ideas should be sent to Phil or to Sam McCall of Florida no later than the Annual Meeting in August. NLPES Calendar
From the EditorMark Bucherl (IN) I took a little more time with this edition to work on its "presentation." I hope you like it. Also, several calls to Bob Boerner have assured me that, if you are reading this, the expanded Web site edition is also available. If you're considering the Mississippi training conference, check out last year's conference "notes" on the Web for a flavor of what to expect (You can expect a lot from NLPES!). Thanks to this month's contributors -- it was great getting a spring preview from our Southern members (though it was snowing in Wisconsin when I called Jennifer Noyes). Thanks also to Indiana Legislative Services Agency staff Susan Knose for proofreading and Deb Hollon for scanning. See you in Philadelphia. P.S., if you already drive the electronic Infobahn, you do need to subscribe to the NLPES List Server, don't you?THE EVALUATORS INSTITUTE REGISTRATION DEADLINE: JUNE 16 The Institute will be held at the Georgetown University Conference Center, Washington, D.C., from July 14-19, 1997. Fees range from $350 to $995 per course. For more information, contact: The Evaluators' Institute COURSES PROGRAM EVALUATION: STEP-BY-STEP DESIGN MIXED-METHOD EVALUATIONS: POINT/COUNTERPOINT GETTING THE MOST FROM SMALL STUDIES OPTIMIZING USE OF RESEARCH AND EVALUATION IN POLICYMAKING Dr. David Cordray, Vanderbilt University. Concrete tactics to assess and improve quality of research and evaluation studies * Specific tools for reviewing the technical feature of studies involving surveys, experimental, and cost-effectiveness methods * Analytic strategies for assessing the likely influence of identified problems * New methods for reducing the uncertainties found in individual studies. PERFORMANCE MEASUREMENT Harry Hatry, Urban Institute. Identification and measurement of performance Program logic models and other methods to identify appropriate performance variables * Considerations in selecting specific indicators for measurement * Pros and cons of alternative data collection methods * Presentation of results to facilitate use and decision making. CONTROLLED FIELD EXPERIMENTS Robert Boruch. Design, execution, and analysis of data in controlled field experiments Statistical, legal/ethical, managerial, program theory, and policy problems. APPLIED SAMPLING: SIMPLE TO COMPLEX Dr. Seymour Sudman, University of Illinois, Urbana-Champaign. Day 1: Deciding how to do the study Simple and Pseudo Simple Random Sampling Determining size of sample Application. Day 2: Stratified Sampling Cluster Sampling Sampling variance estimation from complex samples Application. COST-BENEFIT AND COST-EFFECTIVENESS ANALYSIS: TOOLS AND TECHNIQUES James Edwin Kee, Senior Associate Dean, George Washington University. Identification and measurement of costs and benefits * Consideration of intangible costs and benefits * Calculation of net program benefits * Examination of the ratio of benefits to costs * Conducting a sensitivity analysis on assumptions * Understanding and handling risk factors. MAIL AND SELF-ADMINISTERED SURVEYS Dr. Don Dillman, Social and Economic Sciences Research Center, Washington State University. How and why answers to self-administered questionnaires sometimes differ from those obtained by telephone * How to structure and word survey questions that minimize measurement error * How best to lay out questionnaires * How to implement surveys to achieve high response rates. COPING WITH THE DIFFICULT MEASUREMENT PROBLEMS Dr. Lee Sechrest. Determining what aspects of problem reliability are most critical and maximizing those aspects * Recognizing and identifying latent variables &understanding their relationship to observed variables * Calibration of measures * Approaches for dealing with problems of reactivity * Basic strategies to minimize disadvantages of small samples. Press Release
|
© 2008 National Conference of State Legislatures, All Rights Reserved
Denver Office: Tel: 303-364-7700 | Fax: 303-364-7800 | 7700 East First Place | Denver, CO 80230 | Map
Washington Office: Tel: 202-624-5400 | Fax: 202-737-1069 | 444 North Capitol Street, N.W., Suite 515 | Washington, D.C. 20001
