Circular Reasoning

In March 2009 the French composer Maurice Jarre died. He had written the music for Lawrence of Arabia, Doctor Zhivago and A Passage to India, and newspapers around the world began preparing obituaries. Within hours of the news, a 22-year-old sociology student at University College Dublin named Shane Fitzgerald did something mischievous. He added a quotation to Jarre's Wikipedia page, a lovely, wistful line about life being one long soundtrack, that Jarre had never said. Fitzgerald had made it up. He later explained that he wanted to see how journalists used the internet under deadline pressure.

Wikipedia's editors removed the line more than once, because it had no source. But it stayed up long enough. Obituary writers looking for something moving to quote found it, and it appeared in print and online, including in The Guardian. Nobody checked where it came from, because it was now everywhere. About a month later Fitzgerald emailed the publications and told them what he had done, and The Guardian published a correction and an explanation by its readers' editor.

The most unsettling part is what would have happened if he had stayed quiet. The next time someone on Wikipedia asked for a source for the quote, a helpful editor could have found one: a respected newspaper. And the newspaper's source was Wikipedia. The quote would have been holding itself up. The cartoonist Randall Munroe later drew this loop in an xkcd comic and gave it a name, citogenesis, the creation of a fact by citation.

That loop is a picture of circular reasoning. It happens when the reason given for a claim already depends on the claim being true. "This source is reliable because it says it is." "We know the model is right because the model says so." The argument appears to move from evidence to conclusion, but it is really walking in a circle back to where it started. The fallacy is also called begging the question, from the Latin petitio principii, and sometimes circulus in probando, a circle in the proving.

Aristotle described the mistake more than two thousand years ago. In the Prior Analytics and the Topics he discussed what he called asking for, or assuming, the original point: getting your opponent to grant, as a premise, the very thing you were supposed to prove. He listed it again among the fallacies in Sophistical Refutations. Medieval translators turned his phrase into petitio principii, and English turned that into "begging the question". This is why logicians wince when "begs the question" is used to mean "raises the question", as in "the results beg the question of what to do next". The everyday meaning has probably won, but the older one names something worth having a word for.

Indian logicians mapped the same territory with remarkable care. In the Nyaya tradition, debaters learned to name faults in which a thing is made to rest on itself. Atmashraya, self-dependence, is the tightest circle: A is established by A. Anyonyashraya, mutual dependence, is the two-step loop, where A rests on B and B rests on A. Chakraka, from the word for a wheel, is the longer circle, where A rests on B, B on C, and C comes back round to A. Showing that a rival's position led to one of these was a standard way of exposing it as hollow.

The longer the wheel, the harder it is to see. "Trust the policy because the company is honest. The company is honest because it follows its policies. The policies are good because we can trust them." Each step looks reasonable on its own slide. Only when you lay them end to end do you notice that the last one is holding up the first. A quieter variant hides the conclusion in a single word. Calling a proposal "reckless" and then concluding that it should be rejected has done most of the arguing inside the adjective.

Circular arguments rarely announce themselves. They usually arrive with the premise and the conclusion dressed in different words, so that "users prefer it" and "it's what users choose" feel like two separate facts rather than one fact said twice.

Repetition makes it worse. In 1977 the psychologists Lynn Hasher, David Goldstein and Thomas Toppino found that people rated statements as more likely to be true when they had seen them before, even when they had no other reason to believe them. Researchers now call this the illusory truth effect. A claim that bounces between a few sources comes back to us feeling more familiar each time, and familiarity is easy to mistake for evidence. That is exactly how the Jarre quote grew stronger without ever acquiring a real source.

The deepest reason, though, is that when we already believe a conclusion, a premise that assumes it doesn't look suspicious. It just looks obviously true. The philosopher Karl Popper told a story about this from Vienna in 1919, when he was working with the psychologist Alfred Adler. Popper described a case to Adler that did not seem to fit Adler's theory. Adler analysed it confidently in his own terms without having seen the child. When Popper asked how he could be so sure, Adler said it was because of his thousandfold experience. Popper's private thought was that, with this case, his experience had presumably become a thousand-and-one-fold. Every case was being read through the theory, and then counted as support for the theory.

Some of the most consequential circles are built into software. In 2016 the statistician Kristian Lum and the political scientist William Isaac published a study in the journal Significance called "To predict and serve?". They took a published predictive policing algorithm, the kind sold to police departments by the company PredPol, and ran it on records of drug crimes in Oakland, California. The algorithm recommended sending officers mostly to a few predominantly Black and low-income neighbourhoods.

Yet public health survey estimates suggested that drug use was spread much more widely across the city. The records did not describe where drug crime happened. They described where police had found it, which mostly meant where police had already been looking.

Now follow the loop. The model sends more patrols to an area because the records show crime there. More patrols find more crime there, because that is where people are looking. Those finds become new records. The model, retrained on the new records, is now even more confident. Lum and Isaac simulated this and showed how the extra policing would feed back into the predictions. Other researchers, including Danielle Ensign and colleagues in 2018, have described the same pattern mathematically as a runaway feedback loop.

Nobody in this story needs to be careless or biased for the circle to form. The claim "crime is concentrated here" is being supported by data that the claim itself helped to create. The output has become the evidence.

Picture a team at a learning app that has just launched an AI-powered "Picked for you" row on the home screen. A model ranks courses for each learner, and the top three appear in the row.

Four weeks later the review goes well. The slide shows that the first card in the row gets far more taps than any other course on the home screen. Someone says, "The model clearly knows what people want." A product manager adds, "And look, the courses it ranks highest are the most popular courses in the app." The team decides to retrain the model every week on what learners tap.

Walk through it slowly. The first card gets the most taps partly because it is the first card. Researchers studying search engines have known about this position effect for years. In one well-known eye-tracking study from 2005, Thorsten Joachims and colleagues secretly swapped the order of search results, and people still clicked the top result far more often. So "the top pick gets tapped" can't show the model is right. It would happen with almost any course in that slot. Then "the courses it ranks highest are the most popular": they are popular because the model put them on the home screen. And retraining on taps means next week's model learns from the choices this week's model made for people. Each step uses the model's own influence as proof of the model's judgement.

The honest version starts by admitting what the numbers can and can't show. "The top slot gets most taps. We don't yet know whether that's the model or the slot." Then it breaks the circle with evidence the model didn't produce. Keep a small random group whose row is shuffled, so you can see how much the slot alone is worth. Measure something further down the line that the ranking doesn't directly cause, such as whether people finish the course or come back to it. Record the position of every tap, so the retraining can allow for it. None of this is exotic. It just refuses to let the system mark its own homework.

Not every loop is a fallacy. Definitions are circular by design. "A bachelor is an unmarried man" isn't a weak argument, because it isn't an argument at all. It only becomes a problem when a definition is passed off as a discovery, as in "our power users are the ones who use advanced features, and we've found that power users love advanced features".

In the nineteenth century the philosopher John Stuart Mill made a sharper objection, in A System of Logic (1843). He pointed out that in any valid deduction the conclusion is already contained, in a sense, in the premises. "All humans are mortal, Socrates is human, so Socrates is mortal" only works if you already accept that every human, Socrates included, is mortal. If that counts as begging the question, all of logic does. The better test, which many modern logicians use, is about the listener. An argument begs the question when the only way to accept the premise is to already accept the conclusion. Someone who doubted that Socrates would die could still be persuaded by the premises, because they can be checked separately.

There are also virtuous circles. The philosopher John Rawls described a method he called reflective equilibrium, in which you adjust general principles and particular judgements against each other until they fit. Designers do something similar when they move between the whole layout and its details. And when several independent lines of evidence point the same way, that's not circular, it's convergence. The key word is independent. The Jarre quote seemed to have many sources, but they all led back to one.

The first question I ask is: would this reason convince someone who doesn't already agree? If the only people who find the premise believable are the ones who already believe the conclusion, the argument hasn't moved anyone anywhere.

The second habit is to trace evidence back to where it first came from. A number in a deck came from a dashboard, the dashboard from an event we chose to log, the event from a design we chose to ship. If the trail ends at our own decision, our own model or our own earlier slide, I've found a loop, and I need something from outside it.

The third is to restate the premise and the conclusion in the plainest words I can find, side by side. "People choose it because it's what they prefer" becomes "people choose it because they choose it". Once the synonyms are gone, a circle has nowhere to hide.

Shane Fitzgerald broke his own loop by owning up. Most loops don't come with a confession, so the job falls to us. In the next post, on anecdotal evidence, I'll look at a different way evidence can mislead: not by pointing back at itself, but by being one vivid story asked to stand in for many.

Further reading: Aristotle, Prior Analytics, Book II, chapter 16 · Kristian Lum and William Isaac, "To predict and serve?" (2016) · Karl Popper, Conjectures and Refutations (1963) · Douglas Walton, Begging the Question: Circular Reasoning as a Tactic of Argumentation (1991) · Danielle Ensign and colleagues, "Runaway Feedback Loops in Predictive Policing" (2018)