(Roughly) Daily

David Krakauer (President of the Sante Fe Institute and Professor of Complex Systems there) argues that it takes intelligence to get things spectacularly wrong…

I have never heard a rock described as stupid. And the same would be true of a river, a hurricane, and even a thermostat. Stupidity seems to be a sophisticated form of behavior despite its ignominious associations.

Human beings can land autonomous rovers on Mars, sequence a genome in hours, and engineer nanometer circuits. And yet conspiracy theories, anti-scientific political movements, and institutional hatred proliferate on a scale that might embarrass the meager success record of a medieval alchemist.

One might say that stupidity implies a capacity for getting things right before it can get them spectacularly wrong. Stupidity is not the opposite of intelligence but its evil twin, the dissimulating Cain to a cerebral Abel. And perhaps surprisingly, the degree of stupidity available to any system scales directly with the intelligence that system possesses—more intelligence begets greater feats of stupidity. It would be a stretch to call a bacterium stupid, and we know that cats and dogs achieve modest feats of it. But human beings, equipped with language, abstraction, technology, institutions, and ideology, can be stupid on a truly civilizational scale. This is not a joke; it is close to a law of nature. A law that might very well be our undoing.

We have thousands of research programs on intelligence, and not all of them are intelligent, including studies of IQ, AI, animal cognition, and collective problem-solving. These take place in celebrated departments and are published in prestigious journals devoted to understanding how minds make hard problems easy. These days we cannot take a step without crashing into another article on intelligence and AI. Researchers from all fields without any knowledge of intelligence research and its history have become self-declared “thought leaders” in natural and artificial intelligence.

Yet stupidity gets almost no attention at all. It is treated as a mere absence, as if once we subtract intelligence what remains is stupidity. It is likened to a form of psychological darkness experienced after you have switched off the lights of deliberation. But this is a mistaken belief. Darkness does not do anything pernicious in the way that stupidity does. Stupidity takes an easy problem and, with great effort and misdirected ingenuity, makes it hard. That effort is the key to grasping stupidity, in that you need sophisticated machinery to be genuinely, consequentially stupid.

Here is how I like to think about this from a scientific perspective. If intelligence means making a problem of difficulty X easier, by deploying tools, using mathematics, and adopting strategies that reduce its cost, then stupidity means making a problem of difficulty X harder. And contrary to expectations, the most reliable way to make an easy problem hard is to bring to bear an impressive apparatus of complicated theories, elaborate beliefs, and sophisticated algorithms that sound tremendously convincing but perform worse than doing nothing. A person who does not know the answer to a question is merely ignorant. A person who constructs an ingenious hundred-page argument for the wrong answer is stupid. As great writers, artists, and philosophers throughout time have understood, such constructions require intelligence of a high order…

[Krakauer explores a variety of types of human stupidity, then turns to AI and draws his conclusion…]

… Artificial intelligence is by design the most powerful cognitive artifact ever created. It is engineered to minimize user effort by performing tasks that humans would otherwise find time consuming or impossible. The problem of course is that the better the tool gets the less the user needs to think for themselves. And in a vicious spiral, the less the user thinks the more dependent they become on their tools. That is until the tool disappears and the whole system collapses.

If intelligence is a necessary precondition for stupidity, and intelligence and stupidity scale together such that it takes real intelligence to be spectacularly stupid, then super-intelligence will be the opening act to an era of super-stupidity.

AI hallucination might be the first evidence of this dynamic. Large language models produce fluent, confident, detailed text that is, with some regularity, factually wrong. And this is not a simple bug but a structural feature of systems that optimize for appeal and plausibility rather than truth. And the danger is not that the AI will be wrong, after all, humans are wrong all the time, but knowing this, humans have invented means to detect and correct errors. We call this the scientific method.

The danger is that an AI will be wrong in ways humans can no longer detect because the very capacities that would catch the error have been outsourced to the machine or exceed the capacities of human minds. We face the prospect of a stupidity so sophisticated that it becomes indistinguishable, to its beneficiaries, from intelligence. This is the parable of Douglas Adams’ The Hitchhiker’s Guide to the Galaxy, where the answer to the ultimate question, the meaning of life, the universe, and everything, is 42.

I would like to make a modest proposal and suggest that we need a science of stupidity as rigorous as our emerging sciences of intelligence. This will not require billions of dollars of investment. It would involve inquiries into the mechanisms by which intelligent systems produce stupid outcomes. It would include studying the evolutionary dynamics that maintain stupidity despite its selective costs. It would promote the development of design principles that distinguish tools which enhance cognition from tools which replace it. And it would include surveying the institutional conditions under which collective intelligence degrades into collective stupidity.

Stupidity is not what remains when intelligence is subtracted, it is an active mechanism with its own logic, its own dynamics, and a capacity for unbounded growth parasitic on ingenuity. In a world obsessed with ever more powerful cognitive technologies, understanding stupidity is not merely an academic exercise, it might prove to be the most intelligent thing we do…

An essay on our undoing: “What Makes Humans Stupid,” from @sfiscience.bsky.social and @nautil.us.

Apposite: “How we meet the future” (“If we couldn’t make generalisations, we would be paralysed by the world’s complexity. Does that ever excuse stereotyping?”)

* Albert Einstein

###

As we investigate imbecility, we might recall that it was on this date in 2008 that the Trump International Hotel and Tower in Chicago “topped out“; construction was completed over the next few months. Donald Trump had announced in 2001 that the skyscraper would become the tallest building in the world, but after the September 11 attacks that same year, the architects scaled back the building’s plans, and its design underwent several revisions. When opened in 2009, it became the second-tallest building in the U.S. It surpassed the city’s John Hancock Center as the building with the highest residence (apartment or condo) in the world and briefly held that title until the completion of the Burj Khalifa several months later. The property has been beset by problems from its opening; they continue to this day.

source