Destroyer of Worlds
An Oppenheimer moment for software engineering
Thirty years ago, my friend S– told me in the schoolyard that he’d been channel-flipping and had come across the strangest thing: a documentary, in French, about a French electronic musician whose single aim it was to deconstruct music, gradually stripping back and stripping out every feature of what he wrote - tone, melody, rhythm - until eventually he would have his audience dancing, he claimed, to a single uninterrupted note.
For some reason this piece of TV flotsam stayed with me. Elements of its aesthetic provocation, audacity, and absurdity found echoes in my day-to-day experience down the decades. More recently though, that late-night TV spectacle has transmuted into a template through which it’s possible to see the politics of the past ten years - or perhaps more accurately, the mix of politics, business, and culture of our particular moment. The common element of the template is a deep impatience with everything except destruction.
The strange thing to my mind is not that politics or business would want to destroy, not create, despite manifestos proclaiming the opposite: anger and destruction are clearly the primary emotion and desired outcome of the people, and disruption is a central pillar of how the Valley works. Rather, the strange thing is that the ladder of education that allowed for the construction of the cathedral of modern software engineering is being kicked away so willingly by the technocrats who built it.
It didn’t start this way. The pre-social media era Google had a lot of people who were deeply enthusiastic about the mission of “organising the world’s information and making it universally accessible and useful” - classic bookish nerd types who saw their role as providing a search result as close as practical to the truth, or at least had strong connections to reality. A notable number of the first few waves of early Amazon hires were also book nerds and valued organisation and structure. Bezos’s public reflections on his business, as outlined in his shareholder letters, are similarly thoughtful and well-written. Zuckerberg may well have called his early users dumb fucks for sharing information with him - he was right - but he also knew how to court the driven, algorithmically-obsessed engineers he needed to convert that information into a product. That he achieved by valuing their work, their analysis, and their output - or, in other words, instilling a culture that valued engineering.
But it’s hard to see how many of those driven engineers will actually be happy with the outcome of their feted skills, intelligence, and excellence. Not because of objections to the conjoined surveillance and security state (about which the Valley is increasingly relaxed), but because every value and behaviour inherent in the scientific disciplines relies on the supposition of an objective reality, the tractability of its analysis by rigorous methods, and the supremacy of facts over feelings. In short, the old information environment, for those of us old enough and lucky enough to have experienced it.
The new information environment - created in the main by graduates of the old one - is nothing like this. For a start, the cost of communication has been driven almost to zero; as a result, a lot more communication happens. When the cost of broadcast communication was high, it hid the costs of high-quality communication - research, etc - under the costs of access to the medium in general. But now that the cost of broadcast communication is effectively zero, low-quality communication is cheaper and faster to output, and spreads faster. The bad drives out the good. In Thinking Fast and Slow (2011) Kahneman & Tversky provide evidence for human behaviour - cognition, really - happening in two distinct modes: the first is fast, instinctive, and emotional (system 1), and the second is slow, deliberative, and rational (system 2). These are broad distinctions, but have some scientific validity, and those of us who have used public transport at any point in the last ten years will have seen system 1 in action: image after image flicks past, most of them lasting less than a second, as the viewer rummages through the floating petals on the Internet’s bough in search of something to arouse their passions. But this mode of interacting with information, now the overwhelmingly dominant one, works almost completely within the domain of system 1, and people are thereby trained to seek novelty and instinctive reaction in ways they themselves rarely understand.
In writing the above, I realize I add myself to a not particularly distinguished long line of complainers about “the current state of affairs”. Alas, it is possible to argue that what is going on now is in fact notable and distinct from previous eras. We see it in education, in politics, in this pervasive sense of fear and fragmentation surrounding every social interaction. More than any other political, economic, or social explanation, I find it plausible to see where we are as a function of the general population being pushed into system 1 thinking almost all of the time, soaked as they are in the blatting raindrops of thumbnailed provocations. Scandals, complex money flows, obviously stupid or horrific actions all flow by as but more droplets in the stream. None of it matters, not because the population has gotten more nihilistic - though it has, and will become increasingly so - but because the next image is less than a second away, and maybe that image will provide the satisfaction that the previous thousand did not. The cruelty is not the point — well, it is, but perhaps not an endpoint in itself; the cruelty is the point because it compels a momentary pause in the scrolling, and in that power also extracts a vote.
It is a fact, though generally not expressed this way, that this new information environment is one of the most profound achievements of modern industry. Not just the skill, knowledge, and innovation that went into the screens on which the images rain, but also the vast infrastructure supporting the services displaying the images, tracking the taps, figuring out who you are based on your actions and information, selecting the appropriate ad, charging for same, and feeding the resulting behaviour back into the machine-learning models to stimulate even more tapping, scrolling, and buying. Often these systems, particularly the older or larger ones, have a set of architecture documents larger than any one person could ever read, never mind understand. Making a significant change is an act of heroism - essaying a journey amongst the foothills of words and lines of code until eventually the high goal is achieved. It is startling that the pinnacle of what the old information environment achieved was its own obsolescence - the literate society begetting the post-literate one. Engineers must hope that their future colleagues, for as long as that turns out to be a useful construct, do not consume their own product too much.
It is also notable that so much of this has been built by people whose interests it is now not serving. For example, though I have no formal data about the distribution of trans people in the tech industry - in my experience more prevalent in the tech industry than the general population - infrastructure built by them now has the political purpose of carrying anti-trans messages. It brings to mind Koestler’s image of "sparrows chattering on telegraph wires, while wire flashes telegram that all sparrows must die", from his book of WW2 experiences, The Scum of the Earth. Continuing the trend, my plumber, in an isolated, rural area of the country, was radicalized into 9/11 and Pearl Harbor truthership by YouTube, and had a finely developed theory around how the US and Russia had agreed to carve up Ukraine between them. (Personally speaking, it seems hard to look at US policy towards Ukraine and understand it as anything other than mostly improvised rather than agreed behind the scenes.)
Regardless, though the 2016 US elections, the Brexit referendum, and indeed Russia’s nihilistic war on the long-suffering Ukrainians as well as its own people are all separately treatable, it is possible to understand them as aspects of the same thing — intimately bound up with, relying on, and propagated by the new information environment, feeding on and feeding up sheer misery in a feedback loop optimised to keep itself going. How sustainable is all this?
We have only his own account, but it is apparently true that Oppenheimer, upon witnessing the Luciferean detonation of the first atomic bomb, quoted from Indian epic - I am become death, destroyer of worlds - feeling, in the physical shock of detonation, the direct consequences of his own physics. That detonation set the stage for most of the decades afterwards, and the world that was destroyed was the old world of Europe, its empires, and the pre-WW2 order. The order that was reassembled was founded on the continual threat of annihilation.
My industry faces its own Oppenheimer moment. All around us are the consequences of our detonation: the return of fascism as a mainstream political project, thoughtless and continual hatred, spiralling unaddressable climate change, and elites comfortably immune - or believing they are - to whichever of the multiple potential kinds of annihilation will get the rest of us first. If we learn anything from Oppenheimer, please let us learn from what he did next: an extended period pushing for peace, discredited by his government, but nonetheless continually arguing against nuclear proliferation and the consequent arms race. As reliant as we are in the industry on the literate society and the pre-truth information environment, it is as yet unclear who amongst us would be brave enough to risk their career to defend it.
Alan Craige has worked in the technology sector in the US and in Ireland.
