(Roughly) Daily

The estimable Robin Sloan on the challenge of keeping our language– our words and our use of them– up to the task of wrestling with our present and our future…

The overloading of common words is well underway: new language models have “thinking” modes, “reasoning” capabilities! What this means, in practice, is that they’ve learned to produce a special kind of text, the conversion of the linguistic if-then into a dynamo that spins and spins and, often, magically — yes, it is magical — produces useful results.

Here is one distinction among several: this process can only compound — the models can only “think” by spooling out more text — while human thinking often does the opposite: retreats into silence, because it doesn’t have words yet to say what it wants to say.

Human thinking often washes the dishes, then goes for a walk.

So, if you redefine “thinking” to mean “arriving at a solution through an iterative linguistic loop” … yes, that’s what these models do. That definition is IMHO pretty thin.

We talk about humans thinking harder, which is not the same as thinking longer. I think most people know from experience that thinking longer generally just makes you anxious. But that’s what the models do, and not only longer, but in parallel, all those step-by-step monologues spilling out simultaneously, somewhere in the dark of a data center. “Quantity has a quality all its own,” said Stalin, maybe … 

Well, okay — what does it mean for a human to think harder? Reasonable people will disagree (and in interesting ways) but, for my part, I think it means prospecting new analogies; pitching your inquiry out away from the gravitational attractors of protocol and cliché; turning the workpiece around to inspect it from new angles; and especially bringing more senses into the mix — grounding yourself in reality. You’ll note these moves are challenging or impossible for systems that operate only on/with/inside language.

A couple of years ago, when I wondered if language models are in hell, I expressed some hope about the richness of multimodal training. So far, this hasn’t panned out. Rather than images anchoring text in a richer, more embodied realm, the marriage seems to have gone the opposite direction. The models chop images into sequences of tokens — big bright pictures become spindly threads, a bit sad — and feed them in along with everything else.

We are going to lose this word — we might already have lost it — but/and we can put a marker down; a gravestone, you might call it; for a kind of thinking that used to mean more than “more”.

Other useful words, still with us, include: imagination, ingenuity, insight. Clarity, most of all. Clarity is what Einstein was seeking when he sat and thought hard about the relative motion of magnets and conductors. He wanted to push through language, beyond it, beyond even the formalism of physics — because there wasn’t physics yet for the things he wanted to understand.

I am still waiting for models that aspire to pack complex systems — whole economies — into high-dimensional space, “hold it all in their heads”, then make observations and predictions way out beyond the if-then of “reasoning” language.

Think harder!

“Thinking modes,” from Sloan’s wonderful newsletter.

Pair with “Horseless Carriages, Digital Paint, AI,” Quentin Hardy‘s meditation on the ways in which new technologies shape both our language(s) and the ways we think (from Hardy’s also-wonderful newsletter).

[Image above: Rodin, “The Thinker” (source)]

* Albert Einstein

###

As we ponder pondering, we might recall that it was on this date in 1963 that “Louie Louie” by the Kingsmen entered the Billboard Hot 100.

For more on how the record came to be (and the ruckus over language that followed), see here (and here and here).