Posts Tagged ‘cognition’
“In nature, as in work of art, the isolation of an object tends to invest it with absoluteness, to endow it with that more-than-symbolic meaning which is identical with being”*…
Does AI threaten our humanity? Nathan Gardels, the editor-in-chief of Noema Magazine, suggests that the horse has already left the barn…
From Silicon Valley to the Vatican, across kitchen tables and in the halls of power, the world is astir over the question of whether humans will be able to control the AIs we’ve created. But that begs the question: What does it mean to be “human”?
One place to start sorting this all out is what Confucian philosophy calls “rectification” — naming things for what they actually are.
To N. Katherine Hayles [here], author of Bacteria to AI: Human Futures with our Non-human Symbionts [here], we are already “post-human.”
As she explained to Claire Webb in a recent Futurology podcast, the concept of “human” is culturally and historically contingent. These days, she says, there is still the dominant view that arose with the Enlightenment of the human as an autonomous, singular entity with “rational free will.”
For Hayles, that has never been true because humans have never existed as an entity apart but have always been “woven in the web” of a surrounding environment, or “Umwelt.” It is the symbiotic interactions within that interdependence which determine the signs we sense, the meaningful interpretation of the information we process, the decisions we make, and the actions we take that define who we are.
The most obvious example of our relational interdependence with non-human life is the gut bacteria within us, the biological functions of which we’d perish without.
The false notion of self-enclosed human autonomy is even less true today, she argues, because “we are already in a deeply symbiotic relationship with computational media — chips, routers, devices, microwaves, etc. They have become the controllers, processors and disseminators of a flow of information” in which we are inextricably enmeshed — and, in advanced societies, without which we couldn’t survive. If it all crashed, we couldn’t transport food, fly airplanes or operate the stock market. We’d quite literally live in the dark.
As Hayles describes it, this condition amounts to “distributed cognition and distributed decision-making” between human and non-human participants. She calls these “cognitive assemblages” or “networked intelligence.”
In short, we are “post-human” because we are inseparably joined in an “integrated cognitive framework” with computational media.
This post-human condition would seem to place us some distance along the continuum toward Singularity, the full fusion of humans and intelligent machines. By definition, this symbiosis blurs the line of who or what is in control.
In a related Futurology podcast, Kevin Kelly [here] tells Nils Gilman that “we are already in the Singularity.” From his northern California perch, Kelly has been surfing the waves of technological innovation and cultural transformation for decades, going back to his time as co-founder of The Whole Earth Catalog with Stewart Brand and then Wired magazine.
What Kelly means by “already in” is that human/machine fusion is a presently ongoing process that doesn’t suddenly cross some rapturous threshold, but incrementally unfolds with the accelerated deepening of the “integrated cognitive framework” Hayles talks about.
We can’t clearly see the massive change taking place now in its pixelated form, but will only be able to recognize the fuller picture of what has taken place from a future retrospective standpoint. Our perspective is clouded, says Kelly, by the fact that “we are living inside the event horizon.”
For Kelly, the Singularity is not a closed process with a fixed endpoint, but perpetually open to the emergence of new creative forms and unexpected symbiotic combinations, what the complexity scientist Stuart Kauffman calls the “adjacent possible” that can’t be foretold. “The Singularity is always near,” as Kelly puts it, but forever remains on the elusive horizon.
Hayles departs from the transhumanist vision of Singularity where human consciousness one day is uploaded to an inorganic substrate other than the biological body.
She makes the point, as do many others, that artificial intelligences on their own cannot attain consciousness.
This is because they lack the “natural intelligence” that can only emerge from the extraordinary complexity of the millennia-long evolution of embodied humans adapting to their environment to survive and thrive. No matter how powerful the large language models may be, language which inherently infers that distinctly human experience, cannot be meaningfully transferred to an inorganic substrate lacking the commensurate context.
AIs cannot know the “ground truth” of embodied intelligence, which, at bottom, is the homeostatic quest for survival that neuroscientist Antonio Damasio argues is the root of emotion and self-awareness.
AIs may use chain-of-reasoning logic to achieve objectives by stringing together the language they’ve been trained on, but the meaning is not the same. Precisely the danger of AIs is that they become empowered as actors through “recursive self-improvement” ungrounded in lived experience.
As such, they lack the ingrained primal training that informs moral reasoning. In Pope Leo XIV’s words, “So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean.”
Though AI models may “imitate language” and “simulate empathy and understanding,” he says, “they do not understand what they produce.”
What humans invaluably bring to the table of an “integrated cognitive framework” is, in the parlance of the pontiff, the “integral” nature of the human as a “whole person” in the full set of relations with others and nature that constitute him or her. “The person,” he says, “is not a system of algorithms: he or she is a creature, relationship, mystery.”
In this sense, retaining “human control” of AIs means purposefully and insistently instantiating the integral element drawn from lived human experience into our entwinement with intelligent machines.
The challenge is not just about “disarming” the capacity of AI agents to act on their own, as the Pope has called for, but also about “arming” the integral conception of the human within the integrated cognitive framework in which we already dwell…
From an integrated to an integral cognitive framework: “We Are Already Post-Human,” from @noemamag.com.
* Aldous Huxley, The Doors of Perception
###
As we contemplate connection, we might recall that it was on this date in 1973 that a US Federal Judge signed his decision, following a lengthy trial, declaring the ENIAC patent invalid and belatedly credited physicist John Atanasoff with developing the first electronic digital computer, the Atanasoff- Berry Computer (or simply, “the ABC”). Built in 1937-42 at Iowa State University (then Iowa State College) by Atanasoff and a graduate student, Clifford Berry, it introduced the ideas of binary arithmetic, regenerative memory, and logic circuits.
As it happened, these ideas were communicated from Atanasoff to John Mauchly, who used them in the design of the better-known ENIAC built and patented several years later.
“They will read many things without instruction and will therefore seem to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise.”*…
Socrates was worried about the impact of a new technology– writing– on effetive intelligence of its users. Similar concerns have surfaced with the rise of other new communications technologies: moveable-type printing, photography, radio, television, and the internet. As Erik Hoel reminds us, AI is next on that list…
Unfortunately, there’s a growing subfield of psychology research pointing to cognitive atrophy from too much AI usage.
Evidence includes a new paper published by a cohort of researchers at Microsoft (not exactly a group predisposed to finding evidence for brain drain). Yet they do indeed see the effect in the critical thinking of knowledge workers who make heavy use of AI in their workflows.
To measure this, the researchers at Microsoft needed a definition of critical thinking. They used one of the oldest and most storied in the academic literature: that of mid-20th century education researcher Benjamin Bloom (the very same Benjamin Bloom who popularized tutoring as the most effective method of education).
Bloom’s taxonomy of critical thinking makes a great deal of sense. Below, you can see how what we’d call “the creative act” occupies the top two entries of the pyramid of critical thinking, wherein creativity is a combination of the synthesis of new ideas and then evaluative refinement over them.
To see where AI usage shows up in Bloom’s hierarchy, researchers surveyed a group of 319 knowledge workers who had incorporated AI into their workflow. What makes this survey noteworthy is how in-depth it is. They didn’t just ask for opinions; instead they compiled ~1,000 real-world examples of tasks the workers complete with AI assistance, and then surveyed them specifically about those in all sorts of ways, including qualitative and quantitative judgements.
In general, they found that AI decreased the amount of effort spent on critical thinking when performing a task…
… While the researchers themselves don’t make the connection, their data fits the intuitive idea that positive use of AI tools is when they shift cognitive tasks upward in terms of their level of abstraction.
We can view this through the lens of one of the most cited papers in all psychology, “The Magical Number Seven, Plus or Minus Two,” which introduced the eponymous Miller’s law: that working memory in humans caps out at 7 (plus or minus 2) different things. But the critical insight from the author, psychologist George Miller, is that experts don’t really have greater working memory. They’re actually still stuck at ~7 things. Instead, their advantage is how they mentally “chunk” the problem up at a higher-level of abstraction than non-experts, so their 7 things are worth a lot more when in mental motion. The classic example is that poor Chess players think in terms of individual pieces and individual moves, but great Chess players think in terms of patterns of pieces, which are the “chunks” shifted around when playing.
I think the positive aspect for AI augmentation of human workflows can be framed in light of Miller’s law: AI usage is cognitively healthy when it allows humans to mentally “chunk” tasks at a higher level of abstraction.
But if that’s the clear upside, the downside is just as clear. As the Microsoft researchers themselves say…
While GenAI can improve worker efficiency, it can inhibit critical engagement with work and can potentially lead to long-term over-reliance on the tool and diminished skill for independent problem-solving.
This negative effect scaled with the worker’s trust in AI: the more they blindly trusted AI results, the more outsourcing of critical thinking they suffered. That’s bad news, especially if these systems ever do permanently solve their hallucination problem, since many users will be shifted into the “high trust” category by dint of sheer competence.
The study isn’t alone. There’s increasing evidence for the detrimental effects of cognitive offloading, like that creativity gets hindered when there’s reliance on AI usage, and that over-reliance on AI is greatest when outputs are difficult to evaluate. Humans are even willing to offload to AI the decision to kill, at least in mock studies on simulated drone warfare decisions. And again, it was participants less confident in their own judgments, and more trusting of the AI when it disagreed with them, who got brain drained the most…
… Admittedly, there’s not yet high-quality causal evidence for lasting brain drain from AI use. But so it goes with subjects of this nature. What makes these debates difficult is that we want mono-causal universality in order to make ironclad claims about technology’s effect on society. It would be a lot easier to point to the downsides of internet and social media use if it simply made everyone’s attention spans equally shorter and everyone’s mental health equally worse, but that obviously isn’t the case. E.g., long-form content, like blogs, have blossomed on the internet.
But it’s also foolish to therefore dismiss the concern about shorter attention spans, because people will literally describe their own attention spans as shortening! They’ll write personal essays about it, or ask for help with dealing with it, or casually describe it as a generational issue, and the effect continues to be found in academic research.
With that caveat in mind, there’s now enough suggestive evidence from self-reports and workflow analysis to take “brAIn drAIn” seriously as a societal downside to the technology (adding to the list of other issues like AI slop and existential risk).
Similarly to how people use the internet in healthy and unhealthy ways, I think we should expect differential effects. For skilled knowledge workers with strong confidence in their own abilities, AI will be a tool to chunk up cognitively-demanding tasks at a higher level of abstraction in accordance with Miller’s law. For others… it’ll be a crutch.
So then what’s the take-away?
For one, I think we should be cautious about AI exposure in children. E.g., there is evidence from another paper in the brain-drain research subfield wherein it was younger AI users who showed the most dependency, and the younger cohort also didn’t match the critical thinking skills of older, more skeptical, AI users. As a young user put it:
It’s great to have all this information at my fingertips, but I sometimes worry that I’m not really learning or retaining anything. I rely so much on AI that I don’t think I’d know how to solve certain problems without it.
What a lovely new concern for parents we’ve invented!
Already nowadays, parents have to weather internal debates and worries about exposure to short-form video content platforms like TikTok. Of course, certain parents hand their kids an iPad essentially the day they’re born. But culturally this raises eyebrows, the same way handing out junk food at every meal does. Parents are a judgy bunch, which is often for the good, as it makes them cautious instead of waiting for some finalized scientific answer. While there’s still ongoing academic debate about the psychological effects of early smartphone usage, in general the results are visceral and obvious enough in real life for parents to make conservative decisions about prohibition, agonizing over when to introduce phones, the kind of phone, how to not overexpose their child to social media or addictive video games, etc.
Similarly, parents (and schools) will need to be careful about whether kids (and students) rely too much on AI early on. I personally am not worried about a graduate student using ChatGPT to code up eye-catching figures to show off their gathered data. There, the graduate student is using the technology appropriately to create a scientific paper via manipulating more abstract mental chunks (trust me, you don’t get into science to plod through the annoying intricacies of Matplotlib). I am, however, very worried about a 7th grader using AI to do their homework, and then, furthermore, coming to it with questions they should be thinking through themselves, because inevitably those questions are going to be about more and more minor things. People already worry enough about a generation of “iPad kids.” I don’t think we want to worry about a generation of brain-drained “meat puppets” next.
For individuals themselves, the main actionable thing to do about brain drain is to internalize a rule-of-thumb the academic literature already shows: Skepticism of AI capabilities—independent of if that skepticism is warranted or not!—makes for healthier AI usage.
In other words, pro-human bias and AI distrust are cognitively beneficial.
It’s said that first we shape our tools, then they shape us. Well, meet the new boss, same as the old boss… Just as, both as individuals and societies, we’ve had to learn our way into effective use of new technologes before, so we will with AI.
The enhancement and atrophy of human cognition go hand in hand: “brAIn drAIn,” from @erikphoel.
Pair with a broad and thoughtful view from Robin Sloan: “Is It OK?“
* “For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise.” – Socrates, in Plato’s dialogue Phaedrus 14, 274c-275b
###
As we think about thinking, we might send carefully-considered birthday greetings to Alfred North Whitehead; he was born on this date in 1861. Whitehead began his career as a mathematician and logician, perhaps most famously co-authoring (with his former student, Bertrand Russell), the three-volume Principia Mathematica (1910–13), one of the twentieth century’s most important works in mathematical logic.
But in the late teens and early 20s, Whitehead shifted his focus to philosophy, the central result of which was a new field called process philosophy, which has found application in a wide variety of disciplines (e.g., ecology, theology, education, physics, biology, economics, and psychology).
“There is urgency in coming to see the world as a web of interrelated processes of which we are integral parts, so that all of our choices and actions have consequences for the world around us.”
“Zero is powerful because it is infinity’s twin. They are equal and opposite, yin and yang.”*…

… and like infinity, zero can be a cognitive challenge. Yasemin Saplakoglu explains…
Around 2,500 years ago, Babylonian traders in Mesopotamia impressed two slanted wedges into clay tablets. The shapes represented a placeholder digit, squeezed between others, to distinguish numbers such as 50, 505 and 5,005. An elementary version of the concept of zero was born.
Hundreds of years later, in seventh-century India, zero took on a new identity. No longer a placeholder, the digit acquired a value and found its place on the number line, before 1. Its invention went on to spark historic advances in science and technology. From zero sprang the laws of the universe, number theory and modern mathematics.
“Zero is, by many mathematicians, definitely considered one of the greatest — or maybe the greatest — achievement of mankind,” said the neuroscientist Andreas Nieder, who studies animal and human intelligence at the University of Tübingen in Germany. “It took an eternity until mathematicians finally invented zero as a number.”
Perhaps that’s no surprise given that the concept can be difficult for the brain to grasp. It takes children longer to understand and use zero than other numbers, and it takes adults longer to read it than other small numbers. That’s because to understand zero, our mind must create something out of nothing. It must recognize absence as a mathematical object.
“It’s like an extra level of abstraction away from the world around you,” said Benjy Barnett, who is completing graduate work on consciousness at University College London. Nonzero numbers map onto countable objects in the environment: three chairs, each with four legs, at one table. With zero, he said, “we have to go one step further and say, ‘OK, there wasn’t anything there. Therefore, there must be zero of them.’”
In recent years, research started to uncover how the human brain represents numbers, but no one examined how it handles zero. Now two independent studies, led by Nieder and Barnett, respectively, have shown that the brain codes for zero much as it does for other numbers, on a mental number line. But, one of the studies found, zero also holds a special status in the brain…
Read on to find out the ways in which new studies are uncovering how the mind creates something out of nothing: “How the Human Brain Contends With the Strangeness of Zero,” from @QuantaMagazine.
Pair with Percival Everett’s provocative (and gloriously entertaining) Dr. No.
* Charles Seife, Zero: The Biography of a Dangerous Idea
Scheduling note: your correspondent is sailing again into uncommonly busy waters. So, with apologies for the hiatus, (R)D will resume on Friday the 25th…
###
As we noodle on noodling on nothing, we might send carefully-calculated birthday greetings to Erasmus Reinhold; he was born on this date in 1511. A professor of Higher Mathematics (at the University of Wittenberg, where he was ultimately Rector), Reinhold worked at a time when “mathematics” included applied mathematics, especially astronomy– to which he made many contributions and of which he was considered the most influential pedagogue of his generation.
Reinhold’s Prutenicae Tabulae (1551, 1562, 1571, and 1585) or Prussian Tables were astronomical tables that helped to disseminate calculation methods of Copernicus throughout the Empire. That said, Reinhold (like other astronomers before Kepler and Galileo) translated Copernicus’ mathematical methods back into a geocentric system, rejecting heliocentric cosmology on physical and theological grounds. Both Reinhold’s Prutenic Tables and Copernicus’ studies were the foundation for the Calendar Reform by Pope Gregory XIII in 1582… and both made copious use of zeros.

“Criticism may not be agreeable, but it is necessary. It fulfills the same function as pain in the human body. It calls attention to an unhealthy state of things.”*…
The estimable Henry Farrell on why, on average, we’re better at criticizing others than thinking originally ourselves…
… our individual reasoning processes are biased in ways that are really hard for us (individually) to correct. We have a strong tendency to believe our own bullshit. The upside is that if we are far better at detecting bullshit in others than in ourselves, and if we have some minimal good faith commitment to making good criticisms, and entertaining good criticisms when we get them, we can harness our individual cognitive biases through appropriate group processes to produce socially beneficial ends. Our ability to see the motes in others’ eyes while ignoring the beams in our own can be put to good work, when we criticize others and force them to improve their arguments. There are strong benefits to collective institutions that underpin a cognitive division of labor.
This superficially looks to resemble the ‘overcoming bias’/’not wrong’ approaches to self-improvement that are popular on the Internet. But it ends up going in a very different direction: collective processes of improvement rather than individual efforts to remedy the irremediable. The ideal of the individual seeking to eliminate all sources of bias so that he (it is, usually, a he) can calmly consider everything from a neutral and dispassionate perspective is replaced by a Humean recognition that reason cannot readily be separated from the desires of the reasoner. We need negative criticisms from others, since they lead us to understand weaknesses in our arguments that we are incapable of coming at ourselves, unless they are pointed out to us…
…
… It’s not about a radical individual virtuosity, but a radical individual humility. Your most truthful contributions to collective reasoning are unlikely to be your own individual arguments, but your useful criticisms of others’ rationales. Even more pungently, you are on average best able to contribute to collective understanding through your criticisms of those whose perspectives are most different to your own, and hence very likely those you most strongly disagree with. The very best thing that you may do in your life is create a speck of intense irritation for someone whose views you vigorously dispute, around which a pearl of new intelligence may then accrete…
…
… One of my favourite passages from anywhere is the closing of Middlemarch, where Eliot says of Dorothea:
“Her full nature, like that river of which Cyrus broke the strength, spent itself in channels which had no great name on the earth. But the effect of her being on those around her was incalculably diffusive: for the growing good of the world is partly dependent on unhistoric acts; and that things are not so ill with you and me as they might have been, is half owing to the number who lived faithfully a hidden life, and rest in unvisited tombs.”
Striving to be a Dorothea is a noble vocation, and likely the best we can hope for in any event; sooner or later, we will all be forgotten. In the long course of time, all of our arguments and ideas will be broken down and decomposed. At best we may hope, if we are very lucky, that they will contribute in some minute way to a rich humus, from which plants that we will never see or understand might spring.
Eminently worth reading in full: “In praise of negativity,” from @henryfarrell.
* Winston Churchill
###
As we contemplate the constructive, we might recall that it was on this date in 1871 that a discipline wholly dependent on incorporating corrective critique into its methods was founded: Cleveland Abbe became the founding chief scientist– effectively the head– of the newly formed U.S. Weather Service (later named the Weather Bureau; later still, the National Weather Service).
Abbe had started the first private weather reporting and warning service (in Cincinnati) and had been issuing weather reports or bulletins since 1869 and was the only person in the country at the time who was experienced in drawing weather maps from telegraphic reports and forecasting from them. The first U.S. meteorologist, he is known as the “father of the U.S. Weather Bureau,” where he systemized observation, trained personnel, and established scientific methods. He went on to become one of the 33 founders of the National Geographic Society.









You must be logged in to post a comment.