(Roughly) Daily

Further (and in some ways, contra) to yesterday’s post, Marek Poliks, Roberto Alonso Trillo, Tyler Dunn, Hugh Scott-Douglas, and Jay Springett ponder one extrapolation of AI…

Imagine the most extreme possible factory: a factory that drives itself, a factory that determines its own objectives and optimizes itself to accomplish them. It is something like a fully automated firm, a company with no people inside, a recursively self-optimizing body that decides what to make, how to make it, and why. This is a very special kind of factory, a factory built with the intention of producing something strictly in excess of its creator’s abilities, something so productive, so efficient, and so capable that it exceeds any possibility of its prescription in advance, something that belongs to the set of possible factories that are, at present, unthinkable. How might one build such a thing? What would it mean to steer or govern such a thing? We call this factory a superdark factory, a factory whose insides churn too quickly to comprehend, whose logics branch into tangles too dense to reconstruct, and whose standards of performance are too ephemeral to direct. We believe that it is possible to build such a factory now, with existing tools, should one decide to do so.

Toward such an end, we propose the dark stack—a collection of design principles an architect could use to bring about such a factory and, further, to contribute to this factory’s ongoing governance after its initialization, all while remaining unable to understand almost anything about it. The most significant intervention available to an architect building a superdark factory with the dark stack is a single committed move, made before the factory is ever turned on, that sets four key principles into motion: (1) an array of structural primitives that enable the factory to learn; (2) a versioning regime that measures what the factory does rather than what it is; (3) an adversarial evaluation layer that serves as the factory’s primary sensory organ; and (4) a charter that functions as a negotiated site of governance, cowritten with the factory itself. After this initial move is complete, the dark stack’s most consequential implications involve the foreclosure of human control from factory design and the subsequent instatement of the human architect as a game-theoretical strategic player engaged with the emerging totality of production…

The most extreme possible automated factory- how would you build it? How would you manage/govern it? “The Superdark Factory,” from https://x.com/antikythera_xyz and @mitpress.bsky.social.

See also: “What is a black box? A computer scientist explains what it means when the inner workings of AIs are hidden” (from whence, the image above)

* Widely attributed to Virginia Woolf’s private journal

###

As we feel our way forward, we might recall that it was on this date in 1716 that the first lighthouse in America, Boston Light, was lit on Little Brewster Island, marking the entrance to the Boston, Massachusetts harbor. (Canada’s first lighthouse, the Louisbourg Lighthouse, located on Lighthouse Point at the entrance to Louisbourg Harbour on Cape Breton Island, was lit in 1734.)

During the American Revolution, the original lighthouse was held by British forces and was attacked and burnt on two occasions by American forces. As the British forces withdrew in 1776, they blew up the tower and completely destroyed it.

The lighthouse was eventually reconstructed in 1783, to the same 75-foot height as the original tower. The reconstruction is the second oldest working lighthouse in the United States (after Sandy Hook Lighthouse in New Jersey) and is the only lighthouse still actively staffed by the U. S. Coast Guard (despite its automation in 1998).