Your Model Ensemble Is One Replica
The Comforting Arithmetic
There is a piece of arithmetic that sits under a lot of AI system design, usually without being said out loud.
Suppose a model answers a question correctly 80 percent of the time. Ask three of them and take the majority, and the majority is right about 90 percent of the time. Ask five and it is about 94 percent. Ask eleven and you are near 99. Keep adding voters and the majority converges on the truth.
This is the Condorcet jury theorem, from 1785. It is the intuition behind self-consistency, the technique of sampling a model several times and keeping the most common answer. It is why people build panels of LLM judges. It is why "have another model check it" feels like it should make a system safer.
The theorem has two conditions. Each voter has to be more likely right than wrong, and the voters have to be wrong independently. Language models break the second condition, and in doing so they quietly break the first one too, one question at a time.
Once you see how, a lot of what we call redundancy in AI systems turns out to be one opinion, repeated.
What Replication Assumes
Distributed systems people have a well-worn version of this lesson.
You replicate a database across three machines so that when one fails, the other two carry on. Byzantine fault tolerant protocols go further and survive machines that actively lie: with 3f+1 replicas you can tolerate f of them behaving arbitrarily. They rest on an assumption that is easy to skim past. Castro and Liskov's PBFT paper from 1999 states it plainly: "We assume independent node failures." It then adds what that takes against an attacker: each node should run different implementations of the service code and operating system, with a different root password and a different administrator.
Hardware respects that assumption better than software, though never fully. Drives from the same manufacturing batch tend to die in the same season, which is why careful operators mix batches. That is the whole lesson in miniature. Independence is not something you get for free. It is something you have to buy.
Software makes it obvious. Every replica runs the same code, so a bug triggered by some input or some date is triggered on all of them at once. The leap second added at the end of June 2012 is the classic example: a bug in how Linux handled it sent servers across the industry into runaway CPU usage within the same few minutes. The replication was fine. The replicas were the same machine, many times over.
The usual answer is to run different implementations, so they do not share bugs. That idea has a name, N-version programming, and it was tested forty years ago.
The Experiment That Should Have Warned Us
In a paper published in 1986, John Knight and Nancy Leveson described an experiment that I think every AI engineer should know.
They had 27 versions of the same program written independently from the same specification, by graduate and senior students at two universities. Different people, different places, no communication. Then they ran all 27 versions against a million test inputs.
Individually the programs were very reliable. But on the inputs where they failed, they failed together far more often than independence would predict. Independence was rejected at the 99 percent confidence level.
The programmers had not copied each other. The problem had hard parts, and different people stumble in the same places. A year earlier, Dave Eckhardt and Larry Lee had shown why that is enough to break independence. If some inputs are harder than others, then two versions that each fail independently on any given input will still fail together more often than chance, and the excess is exactly the variation in difficulty across inputs. Independent authors do not give you independent failures, because the failures come from the problem, not from the author.
In 2026, Javier Ron, Benoit Baudry and Martin Monperrus reran the experiment with coding agents in place of students, on the same specification and the same million random tests as the 1986 study: 48 implementations, from 23 models, five agent systems and three programming languages. They saw 429 coincident failures where independence predicts 115. Changing the language or the agent did not buy behavioral diversity. Many of the shared failures traced back to the parts of the specification that were hardest or most ambiguous. Where the specification needed the smallest circle enclosing three points, many agents computed the circle passing through them instead, which are the same circle only when the triangle has no obtuse angle.
The machines found the same hard parts we do.
Same Data, Same Blind Spots
Now consider what it means to sample one model five times.
Five samples of the same prompt are, in a narrow sense, independent: each is a fresh random draw. But they are draws from the same coin, and every question has its own coin. On some questions the model is right 90 percent of the time. On others, the ones that touch a misconception in its training data, a misleading premise or a gap in what it knows, it is right 30 percent of the time. Temperature changes how adventurously you sample from what the model believes. It does not give the model different beliefs.
This is Eckhardt and Lee's result again, with the model playing every programmer at once. And it changes what voting does.
On a question where the model is right 70 percent of the time, voting pushes the answer toward 100. On a question where it is right 30 percent of the time, voting pushes it toward zero. Condorcet's theorem runs in both directions. Voting is not a corrector. It is an amplifier, and it amplifies whatever the model usually says.
Strictly, self-consistency takes a plurality, not a majority, so what wins is the single answer the model gives most often. When its wrong answers scatter, the right one can win at 30 percent. Systematic errors do not scatter. A misconception produces the same wrong answer again and again, and that is the answer the vote crowns.
Here is what that looks like with numbers, on yes-or-no questions. Take a model that is right 90 percent of the time on 70 percent of the questions, and 30 percent of the time on the rest. A single sample is right 72 percent of the time. Three votes lift that to about 74.5 percent, and that is the best it ever gets. Eleven votes give 72.3. Thirty-one give 70.3, worse than asking once. With enough votes the system converges on exactly 70 percent: always right on the questions it usually gets right, and always wrong on every other one.
None of this means voting is worthless. In the original self-consistency paper, it added about eighteen points on grade-school math problems for the largest model tested, and Ron and his colleagues found that majority-voting triples of agent-written programs cut failures about threefold despite all the correlation. Some errors really are random, an arithmetic slip or an unlucky turn early in a chain of reasoning, and voting removes those. But the gain is spent on the questions the model already mostly gets right, and it is spent quickly. On the questions it mostly gets wrong, voting does not remove the error. It certifies it.
That is the dangerous part. A single wrong answer is just a wrong answer. Five agreeing wrong answers look like evidence. The vote turns a likely error into a confident one, and confidence is exactly what downstream systems and people use to decide what to trust.
Different Models, Same Mistakes
The natural fix is to use different models. Different providers, different architectures, different training runs.
The evidence says they are much less independent than we would like.
In 2025, Elliot Kim, Avi Garg, Kenny Peng and Nikhil Garg measured error correlation across more than 350 language models. On one leaderboard dataset, when two models both got a question wrong, they gave the same wrong answer about 60 percent of the time. Models from the same company were more correlated, as you would expect. The more uncomfortable finding was that larger, more accurate models had highly correlated errors with each other, even across different architectures and providers. The better the models got, the more their remaining mistakes overlapped. Kleinberg and Raghavan had named the risk in 2021: algorithmic monoculture.
It makes sense once you think about where the models come from. They are trained on overlapping slices of the same internet, optimized against similar benchmarks and tuned toward similar ideas of a good answer. Different companies, the same textbook.
Some of those shared mistakes are the benchmark being wrong, an ambiguous question or a bad answer key. That is not much comfort. In production there is no answer key, and an ambiguous question is exactly where you most need your checks to disagree.
How Much Information Is in a Panel?
A 2026 paper by Guneet Kohli put a sharp number on what this does to evaluation panels, using a tool from survey statistics called the effective sample size.
If you average n equally noisy measurements, and every pair is correlated by the same amount ρ, the average carries the information of only n / (1 + (n − 1)ρ) independent measurements. With a correlation of 0.5, three voters carry the information of 1.5. Five carry 1.67. A hundred carry 1.98. As n grows, the effective number approaches 1/ρ and stops there. At a correlation of 0.3, you can never get more than a little over three.
Kohli applied this to nine frontier models judging classification tasks together, and found they carried about 2.2 independent votes' worth of information. On those tasks, ten human annotators carried four to six, more than twice the models'. The models' panel fell roughly 8 to 22 percentage points short of what independent voting would predict, the best single judge matched or beat the whole panel, and no standard aggregation method closed more than about a tenth of the gap, even with access to the correct answers. The title says it plainly: nine judges, two effective votes.
The formula is about averages, not votes, so treat it as a gauge of how much information a panel carries, not as a prediction of its accuracy. For voting, the exact statement is the one from earlier, and it is harsher. With enough voters, the vote is right on precisely the questions where the right answer is the one the voters most often give, and wrong on every other one. You can pay for a hundred model calls and buy the information of about two.
Judges Who Recognize Themselves
The problem is sharpest when models check models.
Using an LLM as a judge has become standard, for evaluation, for filtering, and inside agents that critique and revise their own work. But a judge is just another voter, and a judge from the same family as the thing it is judging is very nearly the same voter.
It is worse than shared blind spots. Arjun Panickssery, Samuel Bowman and Shi Feng showed in 2024 that models can tell their own outputs apart from other models' and from people's, and that the better a model is at recognizing itself, the more it favors itself as a judge. A generator and a judge from the same model are one student grading their own exam and recognizing the handwriting.
Self-critique loops inherit the same problem. Asking a model to find the flaws in what it just wrote is useful for catching slips. But the critique shares the context, the reasoning and the blind spots of the draft, and reasoning makes models more alike, not less. Kohli found that asking the judges to reason step by step made the panel less independent, dropping it from 2.18 effective votes to 1.94. A shared way of thinking is a shared way of being wrong.
Where Independence Actually Comes From
If more voters do not help, what does?
The software reliability people worked this out in 1989. Bev Littlewood and Douglas Miller showed that independent development is not enough, because everyone finds the same parts hard. What helps is forcing the versions to be built differently, by different methods, so that what is hard for one is easy for another. They called it forced diversity. In principle, it can even push coincident failure below the independence level.
For AI systems, I think that means stop adding opinions and start adding mechanisms. Independence does not come from asking again. It comes from checking in ways that fail for different reasons.
The strongest checks are not models at all. A compiler, a type checker, a test suite, a schema validator or a query against the database of record each fails for reasons that have nothing to do with how the model was trained. If an agent writes code, running it is worth more than any number of models reading it. If it states a number, checking the number against its source is worth more than asking another model whether it sounds right.
When the check has to be a model, make it differ from the generator in something other than the random seed. Give it different evidence: let the verifier retrieve its own sources instead of reading the generator's. Give it a different job. Checking an answer is often easier than producing it, and when the check uses different information, it is less likely to share the generator's mistake. Confirming that a proposed fix makes the failing test pass is a different task from writing the fix. Hide the reasoning: show the verifier the claim and the evidence, not the argument that led to the claim, so it cannot be talked into agreeing. Use a different model family where you can, remembering that this helps less than it seems to.
And treat disagreement as information. Two different checks disagreeing is not a tie to be broken by a third vote. It is the system telling you it does not know. That is exactly the case to send to a person, or to answer with "I'm not sure."
Counting Sources, Not Signals
I spend a lot of my time on signal fusion, combining many noisy sources into one picture, and this problem is old news there.
The first rule of fusion is that evidence is only as independent as its errors. Two sensors seeing the same real smoke is genuine corroboration. Two sensors fooled by the same steam are one sensor. Two feeds that both repeat the same upstream report are one report, counted twice. The fusion literature has a blunt name for that last failure: data incest. A network of sources quietly re-ingests its own earlier conclusions, each pass counts them as fresh confirmation, and the system grows more confident in a belief that never had more than one source behind it.
An LLM panel is data incest by construction. The upstream report is the training data, and every model has read it.
Fusion also has an answer for the case where you cannot tell how correlated your sources are. In 1997 Simon Julier and Jeffrey Uhlmann introduced covariance intersection, a way of combining two estimates that stays honest whatever the correlation between them turns out to be. It does that by refusing to assume the sources are independent. The combined estimate is never more confident than the evidence can justify under the worst-case correlation. You give up some of the precision you would get if the sources really were independent, and in exchange your confidence never lies to you.
That is the posture I think AI systems need. When you cannot measure how independent your checks are, do not count them as independent. Combine them as if they might share every error, and let only genuinely different mechanisms raise your confidence.
Measure What Fails Together
When you can measure, measure the right thing.
Build a set of hard cases, the questions your system gets wrong or nearly wrong, with known answers. Run every candidate check on it: each model, each prompt, each verifier. Then do not just look at each check's accuracy. Look at where they fail.
For each pair of checks, ask one question. When this one fails, how often does that one fail too, and with the same wrong answer? If the answer is not much lower than one, the second check is decoration. An average correlation over the whole set hides this, because what matters is the hard cases, where the vote is close and one more independent check could change the outcome.
A check that is slightly less accurate but fails on different cases is worth more to the system than another excellent check that fails in the same places. The diversification is in how checks fail apart, not in how many you hold.
Agreement Is Not Evidence
A system that asks five models and reports that they all agree is making an implicit claim: five independent sources confirmed this. That claim is usually false, and people believe it, because agreement is one of the strongest signals of truth we have in ordinary life. When five people who do not know each other tell you the same thing, you should believe it. When five copies of the same reasoning tell you the same thing, you have learned very little.
So the most honest thing an AI system can do is be clear about where its confidence comes from. Not "verified by five models," but "verified by running the tests," or "checked against the source," or "unverified, models agree." Each of those means something different, and anyone relying on the system deserves to know which one they are getting.
Redundancy is one of the oldest and best ideas in reliable engineering. It has never been about the number of copies. It has always been about the independence of their failures. We learned that about disks, and about software, and about programmers in 1986. We are now learning it about models.
Consensus protocols promise one thing: that the replicas agree. They never promised the replicas were right. We should stop asking our models for the first and calling it the second.
The work this essay leans on: Condorcet, Essai sur l'application de l'analyse à la probabilité des décisions rendues à la pluralité des voix (1785). Eckhardt and Lee, A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors (1985). Knight and Leveson, An Experimental Evaluation of the Assumption of Independence in Multiversion Programming (1986). Littlewood and Miller, Conceptual Modeling of Coincident Failures in Multiversion Software (1989). Julier and Uhlmann, A Non-Divergent Estimation Algorithm in the Presence of Unknown Correlations (1997). Castro and Liskov, Practical Byzantine Fault Tolerance (1999). Kleinberg and Raghavan, Algorithmic Monoculture and Social Welfare (2021). Wang et al., Self-Consistency Improves Chain of Thought Reasoning in Language Models (2023). Panickssery, Bowman and Feng, LLM Evaluators Recognize and Favor Their Own Generations (2024). Kim, Garg, Peng and Garg, Correlated Errors in Large Language Models (2025). Kohli, Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels (2026). Ron, Baudry and Monperrus, N-Version Programming with Coding Agents (2026).