Risk-Limiting Audits

When discussing RLAs, it is important to know and understand the terminology and concepts involved in the process. An understanding of theses terms and concepts can ensure that RLAs serve their intended purpose. Below are the primary concepts and terminology used in RLAs.

• Risk Limit: The largest chance that the audit will fail to detect and correct an incorrectly reported outcome. For example, Colorado’s first RLA had a risk limit of 9%, which meant there was a 91% chance that the audit would correct an incorrect outcome if the outcome was wrong. The risk limit is often set in administrative rule by the state or county official conducting the audit.
• Outcome: Outcome refers to the winner(s) of a contest, not the vote totals. A wrong outcome is when an originally reported outcome disagrees with a full hand count.
• Diluted Margin: This is the measure used to determine the “closeness” of the election and may factor into the number of ballots to be examined in some forms of RLAs. See below for more information on types of RLAs.
• Target Contest: The contest (race) selected for a risk-limiting audit. One or more contest can be selected for auditing but it is the target contest that determines the initial number of ballots selected for audit.
• Sample Size: The number of ballots required to be audited before an audit can be stopped. The sample size can be impacted by factors such as the contest selected, the diluted margin and the risk limit. If discrepancies are uncovered during an RLA, the sample size may increase and could lead to a full hand recount.
• Cast Vote Record (CVR): The CVR is the digital representation of all the selections on an individual ballot card that has been scanned and tabulated.
• Imprinting: Being able to locate a single ballot is necessary for conducting ballot comparison RLAs. The imprinting process prints a unique and anonymous identifier (usually a series of numbers) on each individual ballot scanned. This ensures the correct ballot can be retrieved and compared to the CVR if that ballot is selected to be audited.
• Ballot Manifest: A record cataloging all voted ballots including total quantities and where and how each physical ballot is stored. A ballot manifest is essential for all methods of RLAs and must be created by the local election official, independent of the voting system. The information included in the ballot manifest will depend on how ballots are counted (centrally or at each polling location).
• Random Sampling: RLAs rely on random sampling to ensure that all ballots are equally likely to be selected. This ensures that the audit process is not subject to manipulation or an inadvertent bias in selecting some ballots over others. The random sampling should not be predictable before the sample is selected and should provide a way to be publicly verified.
• Random Seed Number: A random number sequence that is created and used to generate the pseudorandom numbers that guide and determine which ballots are selected to be audited.

As noted previously, RLAs are one approach of conducting a postelection audit. However, there are three different methods of conducting RLAs.

• Ballot-Comparison: A ballot-level comparison audit is a type of RLA in which individual paper ballots are randomly selected, the voter markings are examined and interpreted manually, and the human interpretation of voter intent is compared to the voting system’s interpretation of the same ballot, as reflected in the corresponding cast vote records.
• Ballot-Polling: A ballot-polling audit is a type of RLA in which individual paper ballots are randomly selected, and the voter markings are examined and interpreted manually. If a large enough sample shows a large enough majority for the reported winner, the audit stops. This type of RLA cannot identify whether a specific ballot was mistabulated, but it can provide convincing evidence about whether the reported outcome is correct.
• Batch-Comparison: A batch-level comparison audit is a type of RLA that most resembles a “traditional” audit. In a batch-level comparison audit, the voting system must export subtotals for identifiable physical batches of ballots, such as all ballots cast in a precinct or all mail ballots scanned together as a batch by a particular machine. The auditors add up those batch comparison results to verify that they produce the reported contest outcomes. If so, some physical batches are selected at random. The votes in each selected batch are examined manually and tabulated, and the audit counts are compared to the voting system’s reported subtotals.

Two, large administrative considerations that may need to be addressed by states and jurisidictions are equipment and an RLA tool. 

Equipment

Although this is not a comprehensive list, these four types of equipment may dictate what kind of RLA a jurisidiction may conduct or if a jurisidiction can conduct an RLA at all.

• Voting System: All methods of RLAs require a voting system that produces a voter-verified paper audit trail.
• Scanners: Optical scanners are commonly used throughout the country to scan both hand-marked and machine-marked ballots. This function is either done at the polling location or in a central location, such as a county clerk’s office. Jurisdictions that scan and count ballots in a central location often use a few large, high-speed scanners, while jurisdictions that scan and count at each individual polling location may rely on more numerous, but smaller scanners. How and when a ballot is scanned and counted will dictate which type of RLA is possible for a jurisdiction to conduct.
• Imprinting Tool: In a ballot-comparison audit, it is critical to be able to locate a specific ballot amongst all the voted ballots. Central scanners that create and print a unique identifier on individual ballots help significantly in locating particular ballots. The printed identifier can reflect which scanner scanned the ballot, what batch of ballots it is part of and what order it is in in the batch. There is currently no precinct (voter facing) ballot scanner that can (or should) perform this function.
•  Cast Vote Record (CVR): A ballot-comparison audit requires a voting system that produces a CVR with enough information to tie a record to the original paper ballot. Not all equipment currently in use or available can produce a usable CVR for a ballot-comparison audit. However, if a jurisdiction has voter-facing scanners, a ballot comparison audit cannot be conducted.

RLA Tool

Although it is possible to conduct an RLA without the assistance of RLA software, this technology can make the process much easier and is necessary if you are conducting an RLA on a statewide level. The main purpose of an RLA tool is to help collect and manage the data needed to conduct an RLA. With the data provided by local and state officials, the RLA tool can:

• Collect local ballot manifests and create a statewide ballot manifest.
• Estimate the number of ballots that will need to be audited.
• Randomly select the ballots for audit by using a random seed number and a pseudorandom number generator.
• Record and account for discrepancies in the audited ballots.
• Determine if the scope of the audit needs to expand or if the risk limit has been met and the audit is complete.

State policymakers looking into RLAs often take into account these policy considerations/questions:

• What is the scope of the audit? What contests will be audited? The selection of contests that are to be audited is very important and will play a large role in the conduct of an RLA. How should the races be selected: randomly or at the discretion of the state election authority? What types of races (i.e. statewide races and county/local races) and how many should be included?
• How much does it cost to implement RLAs? Once established, it is possible for an RLA to save costs in jurisdictions, particularly administrative costs. Because an RLA may require the auditing of fewer ballots than a traditional audit, staff and other administrative costs could be reduced. However, in an instance where an RLA results in a full recount, administrative costs could increase.
• Will postelection timeframes need to be adjusted? Audits can be time-consuming and are only one of many things that need to be completed postelection. Does your state’s current postelection timeline allow for the inclusion of an audit? It may be necessary to permit more time postelection for the inclusion of an audit.
• How prescriptive does legislation need to be for implementing an RLA? Sometimes less is more. Providing definitions, implementation deadlines, and outlining how further rules or procedures for an RLA will be established may be sufficient for legislation. 
• Would conducting a pilot program be beneficial? Is it possible for one or several jurisdictions to conduct a pilot program to test the applicability of an RLA? This could also include provisions that require a report of the pilot program to the legislature before more jurisdictions move forward.
• Do local jurisdictions have the necessary equipment to conduct an RLA? If not, what would jurisdictions need to fully conduct one? Colorado’s RLA timeline progressed faster after many counties purchased new voting equipment. Likewise, ballot scanners and an RLA tool/interface may need to be acquired before an RLA is feasible in a jurisdiction.
• What do legislators, local election officials and the public need to know about RLAs? Understanding how RLAs work is no easy chore. Do policymakers, election officials and the public know enough about the process to understand and trust the system? What information needs to be prepared and what questions ready to be answered?