The impact of human error in vote tallies: Evidence from Mexico

Article

Published 06.08.21

Human error in vote counting leads to inconsistencies in vote tallies and threatens citizens’ trust in election institutions

The credibility of election outcomes hinges on the accuracy of vote tallies. Vote counting, however, is generally inaccurate. Whether inaccuracies are small or large, and whether they result from wilful malfeasance or unwitting error, they constitute political dynamite susceptible to exploitation for partisan ends. Disputes over the accuracy of electoral results, once the purview of the most precarious democracies, are now prominent even in time-honoured democracies such as the United States. Partisan acrimony over the accuracy of the results in the 2000 US presidential elections has now been overshadowed by conspiracy theories about those held in 2020. Other recent examples include Ecuador’s 2017 elections, where the runner-up pushed for a recount on the basis of numerical inaccuracies in the vote tally, and Mexico’s 2006 elections, where calls by a losing candidate to recount “vote by vote, precinct by precinct” generated mistrust of the electoral system that lingers to-date among his partisans.

What causes counting irregularities?

Inaccuracies in the vote count could stem from fraudulent electoral practices (Myagkov et al. 2010, Hyde 2007, Cantú 2018). However, even in a clean election, the imperfect nature of the counting process makes it impossible to guarantee the accuracy of the tally. Machine-based vote counts have been shown to be inaccurate (Alvarez et al. 2009), and the problem is likely graver when human error is involved (Ansolabehere and Reeves 2004, Goggin et al. 2012, Alvarez and Hall 2008). Yet, according to the ACE Project,1 hand-counting is the rule in almost every country with elections. In fact, hand counting is making a comeback even in places where electronic voting used to be the rule, due to concerns about foreign meddling and hacking.2 

The issue of quality of vote tallies in ‘normal’ elections – i.e. clean, routine elections where votes are counted by people – deserves much greater attention than it has received because of its potentially severe consequences. As illustrated by the cases of the US, Ecuador, and Mexico mentioned previously, inaccuracies in vote tallies are often seized upon by losing parties to impugn the credibility of election results and, in some cases, that of electoral authorities or institutions themselves. In many countries, inconsistencies provide a legal basis for recount requests, as is the case in Argentina, Austria, Brazil, Chile, Colombia, Denmark, Ecuador, Honduras, Mexico, and Spain, among others.3 Inconsistent tallies have led to major court cases in Armenia, Mali, Mexico, and the US, to name a few examples (Posner 2000, Autheman 2004). However, little systematic evidence exists on the causes and consequences of human error in vote counting. 

Measuring quality in vote tallies

To fill this gap, we conducted a systematic study (Challú, Seira, and Simpser 2020)  of the quality of vote tallies using a unique dataset covering the universe of polling stations (about 125,000 per election), poll workers (about half a million per election), and party representatives in five national elections in Mexico between 2009-2015. Additionally, we conducted an original survey of citizen attitudes towards the electoral authorities on 80,000 citizen poll workers in 2017.

Mexican law requires that votes be tallied by randomly selected citizens who function as polling-station workers. Information on inaccuracies is culled from the official document that polling-station workers must fill out by hand, on paper, at the end of Election Day after counting the ballots in their corresponding polling station. This document, known as an acta (which we translate as tally), constitutes the basic input used by the electoral authorities to compute official election results. A sample acta is shown in Figure 1. 

Figure 1 Sample acta

Our measures of inaccuracies follow the electoral authority’s own definitions of inconsistencies in the vote tallies. In any given polling station where the tally contains no inconsistencies, the following accounting / arithmetic equalities ought to hold:

  • the number of cast ballots plus the number of unused ballots should equal the initial number of ballots;
  • the number of people and party representatives who voted should equal the number of ballots extracted from the ballot box;
  • the sum of votes for parties, write-ins, and null ballots should equal the number of ballots extracted from the ballot box; and
  • the sum of people who voted and party representatives who voted should be algebraically correct.

An inconsistency is said to exist when any of the above equalities fails to hold.

Inconsistencies are common, but human errors need not be partisan

We find that inconsistencies in vote tallies are remarkably common, being present in more than two out of five actas, and in a similar proportion of polling stations. We find no evidence, however, that tally inconsistencies in the period we study are the result of partisan malfeasance. This is in line with the scholarly consensus that Mexican elections have been virtually free of many traditional forms of election fraud since at least 1997 as a result of the major electoral reforms of the mid-1990s (Molinar 1991, Domínguez and McCann 1998, Cantú 2018). 

What, then, explains variation in the quality of hand-counted vote tallies? To get at causality, we exploit electoral rules that allocate a varying number of educated polling-station workers to polling stations. We find that polling stations with more-educated poll workers yield tallies with fewer inconsistencies. An additional year of average educational attainment for the poll worker team associated with a polling station reduces the extent of inconsistencies in the tally by up to 7%. Relatedly, we find that the arithmetic difficulty of the tallying task renders inconsistencies more likely: the incidence of inconsistencies is about 17% greater when the acta requires carrying a sum than when it does not. Thus, simplifying the counting and tallying procedures might improve the accuracy of tallies, consistent with behavioural public policy guidelines. 

We also find that mistakes are proportional to the workload, understood as the number of ballots cast in a given polling station. This suggests that decreasing the number of voters allocated to each polling station (and concomitantly increasing the number of polling stations) would not decrease the total number of inconsistencies.

Human error leads to greater doubt and mistrust of institutions

Even if tally inconsistencies result from honest mistakes, politicians often use them to undermine the credibility of election results. We document that this delegitimising strategy may indeed work by eroding trust in the impartiality of electoral authorities, and therefore, we surmise, in democracy more generally. In the elections we study, inaccurate tallies are associated with a 22 percentage-point increase in the probability that the relevant polling station’s votes are recounted, and with a significant decrease in citizen trust (measured with the aforementioned survey) in the electoral authorities as impartial arbiters of elections. Comparing a precinct where no polling stations display inconsistencies with one where all polling stations do, perceptions of the Mexican electoral authority’s (INE’s) impartiality are lower in the latter by 10% of a standard deviation. Table 1 summarises the findings.

Table 1 Summary of main empirical findings

Conclusions and implications

Most democracies today count votes by hand. While hand-counting is invulnerable to hacking, it also has costs in terms of counting effort by citizens, salaries of workers, and mistakes due to human counting. Using unique data with very broad coverage, we show that inconsistencies in vote tallies are very common. Such inconsistencies do not favour any candidate or political party, which suggests that they are honest mistakes. However, the fact that tallying mistakes appear to be honest and random does not render them unimportant; politicians routinely use them to back up recount requests and to discredit specific electoral outcomes and democratic institutions in general.

Our analysis points at the educational attainment of poll workers as a major driver of tallying mistakes. This suggests that less-developed regions find themselves in a type of development trap where low levels of education feed into lower-quality vote tallies, which in turn decreases trust in elections, potentially weakening accountability and the provision of good government. Thus, choices about who counts votes – schoolteachers, citizen volunteers, or professional staff – and what training the counters receive may have important consequences for the quality of vote tallies.

References

Alvarez, R M and T E Hall (2008), Electronic Elections: The Perils and Promises of Digital Democracy, Princeton University Press.

Alvarez, R M, J N Katz, and S A Hill (2009), “Machines versus Humans: The Counting and Recounting of PreScored Punchcard Ballots”, Caltech/MIT Voting Technology Project. 

Ansolabehere, S, and A Reeves (2004), “Using Recounts to Measure the Accuracy of Vote Tabulations: Evidence from New Hampshire Elections 1946–2002”, Caltech/MIT Voting Technology Project. 

Autheman, V (2004), “The Resolution of Disputes Related to ‘Election Results’: A Snapshot of Court Practice in Selected Countries around the World,” in IFES Rules of Law Conference Paper Series. 

Cantú, F (2019), “The Fingerprints of Fraud: Evidence from Mexico’s 1988 Presidential Election”, American Political Science Review 113(3): 710-726. 

Challú , C, E Seira, and A Simpser (2020), “The Quality of Vote Tallies: Causes and Consequences”,  American Political Science Review 114(4), 1071-1085

Domínguez, J I and J A McCann (1998), Democratizing Mexico: Public Opinion and Electoral Choices, John Hopkins University Press.

Goggin, S N, M D Byrne, and J E Gilbert (2012), “Post-Election Auditing: Effects of Procedure and Ballot Type on Manual Counting Accuracy, Efficiency, and Auditor Satisfaction and Confidence”, Election Law Journal: Rules, Politics, and Policy 11(1): 36–51.

Hyde, S D (2007), “The Observer Effect in International Politics: Evidence from a Natural Experiment”, World Politics 60(1): 37–63.

Molinar, J (1991), El Tiempo de la Legitimidad: Elecciones, Autoritarismo Y Democracia en Mexico, Cal Y Arena.

Myagkov, M, P C Ordeshook, and D Shakin (2010), The Forensics of Election Fraud: Russia and Ukraine, Cambridge University Press

Posner, R A (2000), “Florida 2000: a Legal and Statistical Analysis of the Election Deadlock and the Ensuing Litigation”, The Supreme Court Review 2000: 1–60.

Endnotes

https://aceproject.org/.

2 The Netherlands switched to an all-paper ballot ahead of their 2017 election following reports of foreign interference in elections in other countries. In Germany, while electronic voted hasn’t been outlawed, the country’s Constitutional Court placed restrictions on its use without significant safeguards, effectively ending it’s usage.

3 Source: https://aceproject.org/.