Posts Tagged ‘organizations’
“Something is happening here / But you don’t know what it is / Do you, Mister Jones?”*…
Further to the post of Septmeber 15: on trying to understand what might lie on the other side of a transformation driven by AI (and related technologies). Microsoft’s Deputy CTO, Sam Schillace, suggests that organizations are about to change completely…
… Here’s a nerdy TL;DR if you want it: I think AI will enable a switch from what is essentially circuit switching today, in all kinds of organizations, to something that will look and feel a lot more like packet switching. This will happen partially because it can now (AI is to this problem what fast processors were to distributed networking protocols), and because networks are almost always more powerful, flexible, and scalable. This idea is going to feel super uncomfortable and be rejected by business leaders, employers and academics in much the same way that network engineers rejected the idea of packet-switched networks…
… In 1965, making a long-distance phone call in America was a feat of engineering coordination. A call from New York to Los Angeles required a dedicated copper path, physically reserved, for the duration of the conversation, and AT&T had to employ armies of switching engineers to manage this. The Bell System was one of the great technical achievements of the twentieth century, and the people who built it were not fools. They worked ferociously for what they were optimizing for: guaranteed, high-quality, reliable connections between two points.
It seemed fine – expensive, but that’s what everyone was used to. If you are my age, you remember when long-distance was even a thing, and those calls cost more and were precious. I still get a tiny bit nervous talking to my parents 3000 miles away in Michigan, that was so deeply ingrained. Part of me still can’t believe that phone calls all cost the same.
Then a small group of researchers, largely funded by ARPA, mostly working in universities, proposed something that sounded to the Bell engineers like a joke. Instead of reserving a dedicated path, you would break messages into small pieces, label each piece with a destination, and release them into the network to find their own way. Different pieces of the same message might travel completely different routes. They might arrive out of order. There was no guarantee that any particular piece would arrive at all. A phone call, under this scheme, would be like mailing a letter by tearing it into a hundred pieces and dropping each one in a different mailbox, hoping they’d all show up. (Kind of like training a giant AI model on lots of small fragments of language (tokens) and hoping that somehow intelligence would emerge from putting them back together randomly – which largely worked).
The Bell engineers had a name for this general idea: chaos. They had a specific critique too, which was that you couldn’t make any guarantees about quality of service. Voice needs low, consistent latency: you can’t have some syllables arriving seconds after others. And of course, data needs reliability: a corrupted file is useless. The engineers were right about all of this. Packet switching, in its early forms, was genuinely worse at the things circuit switching was designed to do.
It also, within a few decades, made the Bell System irrelevant. The internet didn’t improve on the telephone network; it made the telephone network a minor application running on top of something much larger. Today your phone calls travel as packets (which is why they all cost the same – there’s no such thing as a “long circuit” now), just like everything else. The Bell engineers optimized for the wrong thing. I love that this also has a flavor of “worse is better”. The Bell engineers were optimizing for “correct, complete, but complex and rigid” and the internet is more “simple and recoverable beats correct and complex”.
What actually happened was a philosophical shift about where intelligence should live. Circuit switching is a centralized control philosophy. Someone, somewhere, reserves the path before any message travels it. The network takes responsibility for delivery. The endpoints are dumb; they just send and receive. All the complexity, all the guarantee-making, lives in the infrastructure itself. This works, and it scales up to a point, and then it hits a wall, because the central system has to know everything about every connection, and that knowledge becomes impossible to manage as the network grows. (Sound familiar? This is how we build companies these days. We’ll come back to that idea)
Packet switching inverts this entirely. The network is dumb and the endpoints are smart. A router doesn’t understand the content of the packets it handles – it reads a destination address and forwards the packet one hop closer. The intelligence about what the message means, whether it arrived correctly, what to do if it didn’t – all of that lives at the edges, in the devices doing the communicating. The protocol defines what the endpoints must do. The network just moves bits.
You have seen this argument before, in a different domain. Central planning versus markets is the same disagreement. The Soviet planning apparatus tried to do for an economy what circuit switching does for a telephone network: reserve resources, guarantee outcomes, manage the whole system from the center. It failed not because the planners were stupid but because the information required to run a complex system centrally exceeds what any center can hold (it also failed because many of them were corrupt, and centralized systems are vulnerable to this, since there’s no way to build in mechanisms to correct, outcompete or otherwise reverse that corruption).
Prices, in a market, are packets. They carry information about supply and demand from the edges without anyone at the center having to understand or coordinate it. Markets are chaotic, wasteful, frequently unfair, and they consistently outperform central planning at scale for the same reason the internet outperformed the Bell System. The network always wins, eventually, because the network is where the information actually lives.
Let’s think about companies now. Companies love to do re-orgs. Or at least, they have to do them. They’re a lot of work, just like it was a lot of work to organize and change the old Bell network, and for the same reasons. Companies have to deal with the pain of organizational overhead, because to a large extent, they try to do central planning – circuit switching.
A manager, in an org chart, is a leased line. From an organizational perspective, they’re like a circuit between strategy and execution, reserved whether or not signal is flowing. The org chart is the circuit map. It tells you the paths that exist, which means it also tells you the paths that don’t. If you need to get information or work from point A to point B and there’s no line between them on the chart, you have a problem – you need to go up to a common node and back down, or you need to get someone to lay a new line. When a company needs to change how it behaves, it rewires the circuit map, which we call a reorg.
Why do we do this? This is not because companies are stupid or because the people running them lack imagination (hopefully). It’s because circuit-switched organizations are the only kind that could be managed without the infrastructure that packet-switched organizations require (which isn’t unlike what happened with the internet – we needed cheap processors to be able to build routers that could handle packets fast enough. Before we had that, we had no choice but to set up dedicated circuits).
If you can’t do the things an organization needs – observing and judging work, finding capabilities, holding people accountable – without the overhead of a fixed org chart, well, you just have to have it.
The cost of this solution is enormous and mostly invisible because it’s so normalized…
[Schillace expands on this idea…]
… When packet switching was proposed, the experts who objected most loudly were the most qualified people in the room. They were “right” in an engineering sense, but wrong in the systems sense. And they did what most people do, asking a “why not” question (against the old design and constraints) instead of a “what if” one: what if the new processors could enable a new network design?
Most people looking at AI are making the same mistake, and I predict we will see a lot of this from the orthodox “business advisor” community. I hear this from engineers and leaders already, and it sounds just like the Bell folks: You can’t guarantee accountability in a decentralized system. You can’t audit a network that routes around hierarchy. Regulators will never accept this. Sophisticated investors will raise concerns about governance.
All of these objections will be correct in the short term and beside the point in the long term. They are right from some kind of local engineering perspective but wrong from an emergent systems perspective. The question is not whether packet-switched organizations perform better on the metrics that circuit-switched organizations were built for. They won’t, at least at first, just like packet-switched networks didn’t at first. The question is whether the new architecture unlocks things the old one couldn’t do at all. And I think it will – I think we will get levels of scale, creativity and responsiveness by using these new tools to push intelligence to the edge of all kinds of organizations – not just businesses but political, social, educational and creative ones…
…There is a useful heuristic here, which is that disruptive ideas tend to produce a room that’s roughly split in half. Usually something like half the people think the idea is obviously correct (the folks asking “what if”), and half think it’s obviously wrong (the “why not” crowd). I’ve written about this kind of bifurcation before – this is the sign of something that is genuinely disruptive, and the split is between people who are rejecting it because it feels threatening, and people who have internalized the new world view and are beginning to extrapolate it…
[Schillace considers earlier, unsuccessful attempts at “packet-switched organizations,” concluding that “AI is [the] missing piece – cheap enough intelligence to put “everywhere” and enable a bunch of these ideas for real. It’s the equivalent of those cheap processors [that enabled packet switching] finally showing up, except it’s a much more complete stack arriving all at once” and that the next few years will consist in discovering the new protocols that will allow “packet-switched organizations” to succeed…]
… The Bell engineers who resisted packet switching were not, in the end, wrong about what they said. Voice quality did suffer in early VoIP, packets did get dropped a lot, and for a while, quality was worse. But they were wrong about what would matter in ten years, and catastrophically wrong about what would be possible in thirty.
Email was not a better telegram, the web was not a better encyclopedia, and streaming was not a better cable network. Each of these looks, in retrospect, like an obvious evolution, but none of them were conceivable from inside the circuit-switched world (I remember the freak out about how video was going to break even the highly scalable new internet. It wasn’t even thinkable in the circuit switched world). They required the new infrastructure before anyone could imagine what to build on it.
The organizational equivalent is similarly hard to see in advance. It is likely that packet-switched, AI-native organizations will not just do existing organizational things more efficiently, but that they will do things that are not currently possible. There might be organizations that form around specific problems and dissolve when the problem is solved, rather than persisting as structures in search of purpose, or expertise that routes to where it is needed rather than being allocated by headcount. We might have large groups with a degree of strategic flexibility and sophistication that have never existed together – we might get political or creative movements in radically different shapes because of that. We are definitely going to see an evolution in the creation, management and use of institutional memory. Training will probably look very different, as will job descriptions.
And almost certainly: we will get things we cannot name yet. Nobody in 1975 could have described Google, or Facebook, or VOIP, or even Google Docs.
The phone company’s engineers built something extraordinary. The Bell System was a genuine engineering achievement, and the people who designed it were some of the best technical minds of their era. They lost anyway. Networks are more robust, once the infrastructure to support them exists, and the right protocols to create and maintain them are adopted.
My bet is that the organizations of the future, the ones that compete and thrive, will make use of AI to build networks the same way the internet made use of those cheap processors and new protocols to build huge scale and new capabilities.
This will feel controversial, and might even be worse at first, but networks always win. The age of centrally planned organizations is coming to an end…
Eminently worth reading in full. Chaos that scales: “The Network Always Beats the Castle,” via @timoreilly.bsky.social.
See also: Benedict Evans‘ “AI, tools and transformation“:
… With every new technology, we start by using it for the work we already have, and we just do that more and faster. But then, over time, you make entirely new things. We will use AI to automate broad classes of stuff inside existing workflows and existing companies (although, as I’ve outlined above, that will be enormously more trouble and work than just giving everybody a model). But with every previous platform shift, the stuff that actually mattered was the stuff that wasn’t even possible before and that no-one even imagined…
And by way of keeping all of this in perspective, see Henry Farrell‘s “Machine god metaphors eat your brain” and Max Read‘s “Between 5 and 14 thoughts about AI and the discourse cycle.”
(Image above: source)
* Bob Dylan, “Ballad of a Thin Man” (from Highway 61 Revisited, 1965)
###
As we try to peer over the horizon, we might send carefully-calculated birthday greetings to a man whose work has helped propel the advances we’re pondering: James Hardy Wilkinson; he was born on this date in 1919. A mathematician, he is best known for his pioneering work in numerical analysis and numerical linear algebra, especially in the context of high-speed digital computing. In 1970 he won the ACM Turing Award.
“Bureaucracy defends the status quo long past the time when the quo has lost its status”*…
… which is one of the reasons that they’re hard to update. Kevin Baker describes a 1998 visit to the IRS Atlanta Service Center and ponders its lessons…
… the first thing you’d notice would be the wires. They ran everywhere, and the building obviously hadn’t been constructed with them in mind. As you walked down a corridor, passing carts full of paper returns and rows of “tingle tables,” you would tread over those wires on a raised metal gangway. Each work area had an off-ramp, where both the wires and people would disembark…
… The desks were covered with dot matrix paper, cartons of files, and Sperry terminals glowing a dull monochromatic glow. These computers were linked to a mainframe in another room. Magnetic tapes from that mainframe, and from mainframes all over the country, would be airlifted to National Airport in Washington DC. From there, they’d be put on trucks to a West Virginia town of about 14,000 people called Martinsburg. There, they’d be loaded into a machine, the first version of which was known colloquially—and not entirely affectionately—as the “Martinsburg Monster.” This computer amounted to something like a national nerve center for the IRS. On it programs called the Individual Master File and the Business Master File processed the country’s tax records. These programs also organized much of the work. If there were a problem at Martinsburg, work across the IRS’s offices spanning the continent could and frequently did shut down.
Despite decades of attempts to kill it, The IRS’s Individual Master File, an almost sixty-year old accumulation of government Assembly Language, lives on. Part of this strange persistence can be pegged squarely on Congress’s well-documented history of starving the IRS for funding. But another part of it is that the Individual Master File has become so completely entangled in the life of the agency that modernizing it resembles delicate surgery more than a straightforward software upgrade. Job descriptions, work processes, collective bargaining agreements, administrative law, and technical infrastructure all coalesce together and interface with it, so that a seemingly technical task requires considerable sociological, historical, legal, and political knowledge.
In 2023, as it was in the 1980s, the IRS is a cyborg bureaucracy, an entangled mass of law, hardware, software, and clerical labor. It was among the first government agencies to embrace automatic data processing and large-scale digital computing. And it used these technologies to organize work, to make decisions, and to understand itself. In important ways, the lines between the digital shadow of the agency—its artificial bureaucracy—and its physical presence became difficult if not impossible to disentangle….
Baker is launching a new Substack, devoted to exploring precisely this kind tangle– and what it might portend…
This series, called Artificial Bureaucracy, is a long-term project looking at the history of government computing in the fifty-year period between 1945-1995. I think this is a timely subject. In the past several years, promoters and critics of artificial intelligence alike have talked up the possibility that decision-making and even governance itself may soon be handed over to sophisticated AI systems. What draws together both the dreams of boosters and the nightmares of critics is a deterministic orientation towards the future of technology, a conception of technology as autonomous and somehow beyond the possibility of control.
These visions mostly ignore the fact that the computerization of governance is a project at least seventy years in the making, and that project has never been determined, in the first instance or the last, primarily by “technological” factors. Like everything in government, the hardware and software systems that make up its artificial bureaucracy were and are subject to negotiation, conflict, administrative inertia, and the individual agency of its users.
Looking at government computing can also tell us something about AI. The historian of computing, Michael Mahoney has argued that studying the history of software is the process of learning how groups of people came to put their worlds in a machine. If this is right—and I think it is—our conceptions of “artificial intelligence” have an unwarranted individualistic bias; the proper way to understand machine intelligence isn’t by analogy to individual human knowledge and decision-making, but to methods of bureaucratic knowledge and action. If it is about anything, the story of AI is the story of bureaucracy. And if the future of governance is AI, then it makes sense to know something about its past…
Is bureaucracy the future of AI? Check it out the first post in Artificial Bureaucracy, from @kevinbaker@mastodon.social.
* Laurence J. Peter
###
As we size up systems, we might recall that it was on this date in 1935 that President Franklin D. Roosevelt signed the Social Security Act. A key component of Roosevelt’s New Deal domestic program, the Act created both the Social Security program and insurance against unemployment.
“Resistance to the organized mass can be effected only by the man who is as well organized in his individuality as the mass itself”*…
Thomas R. Wells on something to remember in these times of international conflict…
As any map will show you, the world is divided by political borders into spaces called countries. People and things can live in, come from, or go to these places.
But countries are not any more than that.
Firstly and most obviously, countries are merely a social construction. They are collectively produced fictions (like money, or religions) rather than mind-independent objects (like stones). Being fictional does not mean that countries do not matter, but it does mean that they only exist so long as enough people agree to act as if they do.
Secondly and more significantly, countries are places not agents. Places on a map cannot have interests or goals or take actions to achieve them. To think otherwise is to confuse the properties of one kind of thing with another. This category error infects not only general talk, but also much otherwise careful journalism and even academic analysis. For example, the influential Realistschool of international relations is founded on the axiom that countries do (or ought to) act only in their national interest. This trades on two category errors: that countries (rather than governments) can act and that they have interests. The result is confusing and unfalsifiable nonsense about buffer zones, access to resources and so forth that is about as helpful for understanding, predicting, and managing conflicts as an astrological map.
What lies behind this error is the eliding of spaces on a map with the organisations that rule them. Organisations are collective agents like armies or corporations in which groups of human individuals are converted into a hierarchically coordinated and powerful actor in their own right. Unlike countries, organisations are a kind of collectively produced fiction about which it does make sense to attribute interests and which can actually do things, often very significant things. What we call governments are a particular kind of organisation, one that has achieved the power to make and enforce rules over the inhabitants of a country, for example by hurting those who dare to disagree with it and by preventing outsiders from entering. In Max Weber’s famous definition, it “successfully claims a monopoly of the legitimate use of violence”. This power is called sovereignty and it is an attribute of governments, not countries.
…
People live in countries and are ruled over by governments. It is important to keep each of these three elements distinct and clear so that we can prevent the relentless category errors that confuse public discussion of international affairs. In particular, we should pay more credence to actual people and less to the organisations who claim to be their legitimate representatives merely because they have the power to hurt them. There are many tyrannical governments in this world. Their leaders may declare that they act in the name of the populations and territories they rule but they remain the ones responsible, the ones who should be held to account…
Being careful in how how we apportion blame in these fraught geopolitical times: “There Is No Such Thing As Countries,” from @Philos_Beard in @3QD. Eminently worth reading in full.
* Carl Jung
###
As we steer clear of stereotypes, we recall that it was on this date in 1602 that Vereenigde Oost-Indische Compagnie (VOC, or The Dutch East India Company, as it’s known in the Anglophone world) was born. Generally considered the world’s first trans-national corporation and the first publicly to issue stocks and bonds (and the first company to be ever actually listed on an official stock exchange), it began with a 21-year monopoly on the Dutch spice trade. The VOC also prefigured the mega-corporation of today in that it had quasi-governmental powers, including the ability to wage war, imprison and execute convicts, negotiate treaties, strike its own coins, and establish colonies. Considered by many to be the greatest corporation in history, the VOC eclipsed all of its rivals in international trade (and many nations in power) for almost 200 years.
“Tradition is not the worship of ashes, but the preservation of fire”*…
In the year 578AD Germanic tribes were warring over the remains of the Roman Empire, an eight-year-old boy named Muhammad was growing up in Mecca, the Mayan Empire was flourishing in Central America, and the world’s longest continuously operated business was founded in Japan. When Prince Shōtoku Taishi (572–622) commissioned the construction of Japan’s first Buddhist temple, Shitennō-ji, Japan was predominantly Shinto and had no miyadaiku(carpenters trained in the art of building Buddhist temples), so the prince hired three skilled men from Baekje, a Buddhist state in what is now Korea. Among them was Shigetsu Kongō, whose work would become the foundation of the construction firm Kongō Gumi.
In the centuries that followed, the maintenance, repair and reconstruction of Shitennō-ji (ravaged a number of times by wars and natural disasters) provided Kongō Gumi’s main source of income, but as Buddhism spread throughout Japan the scope of the company’s work also expanded to include contributions to other major temple complexes such as Hōryū-ji (607) and Koyasan (816), as well as Osaka Castle (1583). Kongō Gumi would continue to flourish under the Tokugawa shogunate (1603–1867), a period during which Buddhist temples received substantial financial support. The company weathered the pro-Shinto Meiji Period (1868–1912) and its often violent efforts to eradicate Buddhism from Japan, which included the destruction of tens of thousands of Buddhist temples. Kongō Gumi also survived the Shōwa Financial Crisis of 1927, keeping pace with economic and technological developments until it finally succumbed to financial difficulties and became a subsidiary of Takamatsu Kensetsu in 2006, after more than 1,400 years of independent operation.
Although Japan boasts six of the world’s oldest companies and an estimated 20,000 firms over 100 years old, Kongō Gumi’s longevity is certainly remarkable and worthy of study. Fortunately, the principles that guided the company over the centuries have been preserved by the Kongō family itself. The 32nd leader of the company, Yoshisada Kongō, writing during the Meiji Period, left a creed, later titled Shokuke kokoroe no koto, or ‘family knowledge of the trade’, a list of 16 precepts distilled from the company’s successful past and intended to guide and preserve the family’s operations into the future. Western observers might be surprised to discover that while the creed addresses ‘business’ subjects such as quality control and customer satisfaction, it puts equal emphasis on ‘personal’ issues such as how to dress (in keeping with one’s station), how much to drink (in moderation) and how to treat others (with utmost respect). Indeed, the first article of the creed states that minding the precepts of Confucianism, Buddhism and Shinto, and training to use the carpenter’s rule are ‘our most important duty’, suggesting that the standards against which a Kongō measures his life are as critical to success as the instrument by which he measures his work…
Learning from the long-lived: “Building on Tradition — 1,400 Years of a Family Business.”
See also: “The Data of Long-Lived Institutions” from @zander at The Long Now Foundation.
* Gustav Mahler
###
As we take the long view, we might recall that it was on this date in 1911 that RMS Titanic was launched from the boatyard in Belfast in which it was built, the largest passenger ship of its day. A state-of-the-art steamship, it set sail from Southampton on its maiden voyage on march 10th of the following year, bound for New York City. Four days later, after calls at Cherbourg in France and Queenstown (now Cobh) in Ireland, the “unsinkable” Titanic collided with the iceberg that sent it under in the North Atlantic, 375 miles south of Newfoundland.
You must be logged in to post a comment.