This is a hotly debated topic in so-called ‘tech’ circles and the academic and media groups that orbit that world like one of Jupiter’s radiation blasted moons. I dropped the phrase, ‘can large language models reason’ into Google, (that rusting machine) and got this result:
This is only a small sample. According to Google there are “About 352.000.000 results.” We can safely conclude from this, and the back and forth that endlessly repeats on Twitter in groups that discuss ‘AI’ that there is a lot of interest in arguing the matter: pro and con. Is this debate, if indeed it can be called that, the least bit important? What is at stake?
***
According to ‘AI’ industry enthusiasts, nearly everything is at stake; a bold new world of thinking machines is upon us. What could be more important? To answer this question, let’s do another Google search, this time, for the phrase, Project Nimbus:
The first result returned was a Wikipedia article, which starts with this:
Project Nimbus (Hebrew: פרויקט נימבוס) is a cloud computing project of the Israeli government and its military. The Israeli Finance Ministry announced in April 2021, that the contract is to provide “the government, the defense establishment, and others with an all-encompassing cloud solution.” Under the contract, the companies will establish local cloud sites that will “keep information within Israel’s borders under strict security guidelines.”
What sorts of things does Israel do with the system described above? We don’t have precise details but there are clues such as what’s described in this excerpt from the +972 Magazine article, ‘A mass assassination factory’: Inside Israel’s calculated bombing of Gaza’ –
According to the [+972 Magazine] investigation, another reason for the large number of targets, and the extensive harm to civilian life in Gaza, is the widespread use of a system called “Habsora” (“The Gospel”), which is largely built on artificial intelligence and can “generate” targets almost automatically at a rate that far exceeds what was previously possible. This AI system, as described by a former intelligence officer, essentially facilitates a “mass assassination factory.”
I wrote about Habsora in this essay. When I think of the ways algorithmic systems are used – increasingly to not only control but also to kill, my interest in the LLM debate fades, like the memory of a dream. It seems light and airy, unsuitable for the age.
***
History, and legend tell us that in ancient Athens there was a place called the Lyceum, founded by Aristotle, where the techniques of the Peripatetic school were practiced. Peripatetic means, more or less, ‘walking about’ which reflects the method: philosophers and students, mingling freely, discussing ideas. There are centuries of accumulated hagiography about this school. No doubt it was nice for those not subject to the slave system of ancient Greece.
Similarly, debates about whether or not LLMs can reason are nice for those of us not subject to hellfire missiles, fired by Apache helicopters sent on their errands based on targeting algorithms. But, I am aware of the pain of people who are subject to those missiles. I can’t unsee the death facilitated by computation.
This is why I have to leave the debating square, the social media crafted lyceum. Do large language models reason? No. But even spending time debating the question offends me now. A more pressing question is what the people building the systems killing our fellow human beings are thinking. What is their reasoning?
On September 14, 2023, while touring Twitter the way you might survey the ruins of Pompey, I came across a series of posts responding to this statement from the EU Commission account:
Mitigating the risk of extinction from AI should be a global priority…
What attracted critical attention was the use of the phrase, ‘risk of extinction‘ a fear of which, as Dr. Timnit Gebru alerts us (among others, mostly women researchers I can’t help but notice) lies at the heart of what Gebru calls the ´TESCREAL Bundle.’ The acronym, TESCREAL, which brings together the terms Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism and Longtermism, describes an interlocked and related group of ideologies that have one idea in common: techno-utopianism (with a generous helping of eugenics and racialized ideas of what ‘intelligence’ means mixed in to make everything old new again).
Risk of extinction. It sounds dramatic, doesn’t it? The sort of phrase you hear in a Marvel movie, Robert Downey Jr, as Iron Man stands in front of a green screen and turns to one of his costumed comrades as some yet to be added animated threat approaches and screams about the risk of extinction if the animated thing isn’t stopped. There are, of course, actual existential risks; asteroids come to mind and although climate change is certainly a risk to the lives of billions and the mode of life of the industrial capitalist age upon which we depend, it might not be ‘existential’ strictly speaking (though, that’s most likely a distinction without a difference as the seas consume the most celebrated cities and uncelebrated communities).
The idea that what is called ‘AI’ – which, when all the tech industry’s glittering makeup is removed, is revealed plainly to be software, running on computers, warehoused in data centers – poses a risk of extinction requires a special kind of gullibility, self interest, and, as Dr, Gebru reminds us, supremacist delusions about human intelligence to promote, let alone believe.
***
In the picture posted to X, Ursula von der Leyen, President of the European Commission, is standing at a podium before the assembled group of commissioners, presumably in the EU Commission building (the Berlaymont) in Brussels, a city I’ve visited quite a few times, regretfully. The building itself and the main hall for commissioners, are large and imposing, conveying, in glass, steel and stone, seriousness. Of course, between the idea and the act there usually falls a long shadow. How serious can this group be, I wondered, about a ‘risk of extinction’ from ‘AI’?
***
To find out, I decided to look at the document referenced and trumpeted in the post, the EU Artificial Intelligence Act. There’s a link to the act in the reference section below. My question was simple: is there a reference to ‘risk of extinction’ in this document? The word, ‘risk’, appears 71 times. It’s used in passages such as the following, from the overview:
The Commission proposes to establish a technology-neutral definition of AI systems in EU law and to lay down a classification for AI systems with different requirements and obligations tailored on a ‘risk-based approach’. Some AI systems presenting ‘unacceptable’ risks would be prohibited. A wide range of ‘high-risk’ AI systems would be authorised, but subject to a set of requirements and obligations to gain access to the EU market.
The emphasis is on a ‘risk based approach’ which seems sensible at first look but there are inevitable problems and objections. Some of the objections come from the corporate sector, claiming, with mind-deadening predictability, that any and all regulation hinders ‘innovation’ a word that is invoked like an incantation only not as intriguing or lyrical. More interesting critiques come from those who see risk (though, notably, not existential) and who agree something must be done but who view the EU’s act as not going far enough or going in the wrong direction.
Here is the listing of high-risk activities and areas for algorithmic systems in the EU Artificial Intelligence Act:
o Biometric identification and categorisation of natural persons
o Management and operation of critical infrastructure
o Education and vocational training
o Employment, worker management and access to self-employment
o Access to and enjoyment of essential private services and public services and benefits
o Law enforcement
o Migration, asylum and border control management
o Administration of justice and democratic processes
Missing from this list is the risk of extinction; which, putting aside the Act’s flaws, makes sense. Including it would have been as out of place in a consideration of real-world harms as adding a concern about time traveling bandits.. And so, now we must wonder, why include the phrase, “risk of extinction” in a social media post?
***
On March 22, 2023, the modestly named Future of Life Institute, an organization initially funded by the bathroom fixture toting Lord of X himself, Musk (a 10 million USD investment in 2015) whose board is as alabaster as the snows of Antarctica once were, kept afloat by donations from other tech besotted wealthies, published an open letter titled, ‘Pause Giant AI Experiments: An Open Letter.’ This letter was joined by similarly themed statements from OpenAI (‘Planning for AGI and beyond’) and Microsoft (‘Sparks of Artificial General Intelligence: Early experiments with GPT-4’).
Each of these documents has received strong criticism from people, such as yours truly, and others with more notoriety and for good reason: they promote the idea that the imprecisely defined Artificial General Intelligence (AGI) is not only possible, but inevitable. Critiques of this idea – whether based on a detailed analysis of mathematics (‘Reclaiming AI as a theoretical tool for cognitive science’) or of computational limits (The Computational Limits of Deep Learning) have the benefit of being firmly grounded in material reality.
But as Freud might have warned us, we live in a society shaped not only by our understanding of the world as it is but also, in no small part by dreams and fantasies. White supremacists harbor the self congratulating fantasy that any random white person (well, man) is an astounding genius when compared to those not in that club. This notion endures despite innumerable and daily examples to the contrary because it serves the interests of certain individuals and groups to persist in delusion and impose this delusion on the world. The ‘risk of extinction’ fantasy has caught on because it builds on decades of fiction, like the idea of an American Dream and adds spice to an otherwise deadly serious and grounded business: controlling the tech industry’s scope of action. Journalists who ignore the actual harms of algorithmic systems rush to write stories about a ‘risk of extinction’ which is far sexier than talking about the software now called ‘AI’ that is used to deny insurance benefits or determine criminal activity.
The European Union’s Artificial Intelligence Act does not explicitly reference ‘existential risk’ but the social media post using this idea is noteworthy. It shows that lurking in the background, the ideas promoted by the tech industry – by OpenAI and its paymaster Microsoft and innumerable camp followers – have seeped into the thinking of decision makers at the highest levels.
And how could it be otherwise? How flattering to think you’re rescuing the world from Skynet, the fictional, nuclear missile tossing system featured in the ‘Terminator’ franchise, rather than trying, at long last, to actually regulate Google.
Usually, I try to start these essays with an anecdote to lead you, the esteemed reader, into my topic. These anecdotes lead me into a subject too; a warm up to get the writing process flowing.
For this brief essay, which is about yet another video posted to Twitter about a supposedly autonomous killing machine, I’m thinking of the classic shell game, which is described in this (obligatory) Wikipedia article:
The shell game (also known as thimblerig, three shells and a pea, the old army game) is often portrayed as a gambling game, but in reality, when a wager for money is made, it is almost always a confidence trick used to perpetrate fraud. In confidence trick slang, this swindle is referred to as a short-con because it is quick and easy to pull off. The shell game is related to the cups and balls conjuring trick, which is performed purely for entertainment purposes without any purported gambling element.
A confidence trick, a fraud. What do I mean when I use these words to describe the machine gun toting, robot ‘dog’?
After all, there is the machine and it’s gun. Where is the game?
Let’s look at the original post:
Tweet showing video of robot dog
The video shows a machine, very similar to Boston Dynamics’ Spot (which I discuss in this essay, Boston Dynamics, A Brief Inquiry) moving on a course with its spindly legs, firing an automatic weapon.
It’s implied that what we’re seeing is, as in that often cited Black Mirror episode ‘Metal Head‘, an autonomous machine which can roam on its own, killing people using some form of silicon intelligence, tuned for lethality.
What we’re really seeing is a remote controlled system, whose true purpose is obscuring the bullet’s source, the hand pulling the trigger.
The evidence is in the video.
The Controller and The Controlled
Take a close look at what we see in this still excerpt; a military transport vehicle, sitting idly by for no apparent reason. This is the controller, hidden in plain sight (the operator could just as easily have been outside) to give the appearance of autonomy.
A grounded, materialist, less science fiction informed examination of this image tells the story (well that, and the fact there is no machine Intelligence and certainly not within the form factor of this device): this is a remote controlled device, a drone. What appears to be a VHF whip antenna is clearly visible along with control interfaces and a camera for navigation.
One additional bit of information can be found in this image:
This shot, probably by accident because someone thought it was cool, reveals what’s behind the curtain: the camera’s view is directly of the gunsight which is certainly what the controller, sitting in the military transport vehicle, sees via a display. The robot ‘dog’ though it exhibits dangerous potential, is not the harbinger of a new form of self-directed killbot but rather, the harbinger of a new class of remote controlled drone, designed, like their UAV cousins, to obscure culpability.
What is the True Danger?
The 21st century isn’t going well.
In addition to climate change, the lingering possibility of nuclear war and the unraveling of neoliberal capitalism which, at the height of its power as a social form, was sold as being history’s last stage, we face the coming to earth of the military drone, long a menace to people around the world and arriving, as all military ideas eventually do, to a street near you.
So, we should agree there is a danger. But it’s not the science fiction danger of sinister machines, free of human control. It’s the danger of remote operated systems, used to harass and kill people while obscuring the source of this harassment and death. It’s easy to imagine a scenario: someone is killed by a police officer but the tools of body cams and eyewitness testimony are removed; the device from which the bullets flew is controlled by an unseen operator, indemnified from responsibility like the drone operators remotely flying machines over contested territory.
Earlier I mentioned the shell game which is this: the sleight of hand, performed via carefully shot marketing material, which leads our thoughts away from who is pulling the trigger into talking endlessly, and in terrified circles, about the same, tired science fiction tropes.
It’s time to put Black Mirror away to see the true danger taking shape, right before our eyes.
What follows is a sketch, the foundation of a propaganda model, focused on what I’ll call the ‘AI Industrial Complex‘. By the term AI Industrial Complex, (AIIC) I mean the combination of technological capacity (or the lack thereof) with marketing promotion, media hype and capitalist activity that seeks to diminish the value of human labor and talent. I use this definition to make a distinction between the work of researchers and practical technologists and the efforts of the ownership class to promote an idea: that machine cognition is now, or soon will be, superior to human capabilities. The relentless promotion of this idea should be considered a propaganda campaign.
If There’s No AI, What is Being Promoted?
It’s my position there is no existing technology that can be called ‘artificial intelligence’ (how can we engineer a thing we haven’t yet decisively defined?) and that, at the most sophisticated levels of government and industry, the actually existing limitations of what is essentially pattern matching, empowered by (for now) abundant storage and computational power, are very well understood. The existence of university departments and corporate divisions dedicated to ‘AI’ does not mean AI exists; it’s evidence there’s powerful memetic value attached to using the term, which has been aspirational since it was coined by computer scientist John McCarthy in 1956. Once we filter for hype inspired by Silicon Valley hustling (the endless quest to attract investment capital and gullible customers) we are left with promotion intended to shape common perception about what’s possible with computer power.
It’s tempting to see this as a temporarily embarrassing bit of overreach by an enthusiastic field luminary – yet another example of familiar hype but let’s go deeper and ask questions about the political economy underpinning this messaging excess.
Hinton on Radiology in 2016
Radiologists are expensive and, in the US, very much in demand (indeed, there’s a shortage of qualified people). Labor shortages typically lead to higher wages and better working conditions and form the material conditions that create what some call labor aristocracies. In the past, such shortages were addressed via pushes for training and incentives to workers (such as the lavish perks that were common in the earlier decades of the tech era).
If this situation could be bypassed via the use of automation, that would devalue the skilled labor performed by radiologists, solving the shortage problem while increasing the power of owners over the remaining staff.
The promotion of the idea of automated radiology – regardless of actually existing capabilities – is attractive to the ownership class because it holds the promise of weakening labor’s power and increasing – via workforce cost reduction and greater scalability – profitability. I say promotion, because there is a large gap between what algorithmic systems are marketed as being capable of, and reality. This gap, which, as I stated earlier is well understood by the most sophisticated individuals in government and industry, is unimportant to the larger goal of convincing the general population their work efforts can be replaced by machines. The most important outcome isn’t thinking machines (which seems to be a remote goal if possible at all) but a demoralized population, subjected to a maze of crude automated systems which are described as being better than the people forced to navigate life through these systems.
A Factor Among Factors
Technological systems – and the concepts attached to them – emerge from, and reflect the properties of the societies that create those systems. Using the Hegelian (and later, Marxist) philosophy of internal relations, we can analyze both real algorithmic systems and the concept of ‘AI’ as being a part of the interplay of factors that comprise global capitalist dynamics – both actor and acted upon. From this point of view, the propaganda effort promoting ‘AI’ should not be considered in isolation, but as one aspect of a complex.
Hype vs. Propaganda
What defines hype and what differentiates standard industry hype from a propaganda campaign?
Hype (such as marketing material that makes excessive claims – for example, AI reading emotions) is narrowly designed to attract investment capital and customers. Hype should be considered a species of advertisement. Propaganda has a broader aim, which is described by Jacques Ellul in his work, Propaganda.
Describing one of the four elements of propaganda, and bridging from advertising to propaganda, Ellul writes…
“Public and human relations: These must necessarily be included in propaganda. This statement may shock some readers, but we shall show that these activities are propaganda because they seek to adapt the individual to a society, to a living standard, to an activity. They serve to make him conform, which is the aim of all propaganda. In propaganda we find techniques of psychological influence combined with techniques of organization and the envelopment of people with the intention of sparking action.”
A Propaganda Model: Foundational Concepts
As the model of AI as propaganda is constructed, the works of three thinkers will provide key guidance:
Jacques Ellul: Propaganda
As already noted, Ellul’s key work on propaganda (which, I think, was the first to apply sociology and psychology to the topic) is a critical source of inspiration:
“Propaganda is first and foremost concerned with influencing an individual psychologically by creating convictions and compliance through imperceptible techniques that are effective only by continuous repetition. Propaganda employs encirclement on the individual by trying to surround man by all possible routes, in the realm of feelings as well as ideas, by playing on his will or his needs through his conscious and his unconscious, and by assailing him in both his private and his public life.
The propagandist also acknowledges the most favorable moment to influence man is when an individual is caught up in the masses. Propaganda must be total in that utilizes all forms of media to draw the individual into the net of propaganda. Propaganda is designed to be continuous within the individual’s life by filling the citizen’s entire day. It is based on slow constant impregnation that functions over a long period of time exceeding the individual’s capacities for attention or adaptation and thus his capabilities of resistance”
The relentless promotion of the idea that automation is on the verge of replacing human labor can be interpreted as being part of an effort to create a conviction (there is artificial intelligence’, it cannot be stopped) and compliance (resistance to ‘AI’ is retrogressive Luddism).
Noam Chomsky/Edward S. Herman: The Propaganda Model
In their book, ‘Manufacturing Consent’ Chomsky and Herman present a model of propaganda via media:
“The third of Herman and Chomsky’s five filters relates to the sourcing of mass media news:
The mass media are drawn into a symbiotic relationship with powerful sources of information by economic necessity and reciprocity of interest. Even large media corporations such as the BBC cannot afford to place reporters everywhere. They concentrate their resources where news stories are likely to happen: the White House, the Pentagon, 10 Downing Street and other central news “terminals”. Although British newspapers may occasionally complain about the “spin-doctoring” of New Labour, for example, they are dependent upon the pronouncements of “the Prime Minister’s personal spokesperson” for government news. Business corporations and trade organizations are also trusted sources of stories considered newsworthy. Editors and journalists who offend these powerful news sources, perhaps by questioning the veracity or bias of the furnished material, can be threatened with the denial of access to their media life-blood – fresh news. Thus, the media has become reluctant to run articles that will harm corporate interests that provide them with the resources that they depend upon.
The dependence of news organizations on press releases from Google and other tech giants that promote the idea of ‘AI’ can be interpreted as being an example of the ‘symbiotic relationship, based on reciprocity of interest’ Chomsky and Herman detail.
The concept of “artificial intelligence” is aspirational (like ‘warp drive’) and does not describe any existing or likely to exist computational system. Despite this, the concept is promoted to attract investment capital and customers but also, more critically for my purposes, devalue the power of labor – if not in fact than in perception (which, in turn, becomes fact). For this reason, I assert that ‘AI’, as a concept, is part of a propaganda campaign.
Key Characteristics of AI Propaganda
The promotion of the concept of AI, as a propaganda effort, has several elements:
* Techno-optimism: The creation of thinking machines is promoted as being possible, with little or no acknowledgement of limitations.
* Techno-determinism: The creation of thinking machines is promoted as being inevitable and beyond human intervention, like a force of nature
* An Elite Project: Although individual boosters, grifters, techno enthusiasts and practitioners may contribute within their circles (for ex. social media) to hype, the propaganda campaign is an elite project designed to effect political economy and the balance of power between labor and capital.
* Built on, but not limited to, hype: There is a relationship between hype and propaganda. Hype is of utility to the propaganda campaign but the objective of that campaign is broader and targeted towards changing societal attitudes and norms.
I use the term attack mannequins to describe this complex – lifeless things, presented as being lifelike, used to assault the position and power of ordinary people.
UPDATE: 2 NOVEMBER 2021
In this video, YouTube Essayist Tom Nicholas details the efforts Waymo has made to convince people – via the use of YouTube ‘educators’ – that autonomous vehicles are a perfected technology, superior to human drivers and a solution to traffic safety and congestion issues.
Nicholas makes the point that inasmuch as the Waymo ‘autonomous’ taxi service (supported by a large staff of people behind the scenes) only operates in a subsection of the suburbs of Phoenix, Arizona USA, the PR campaign’s goal can’t be explained as advertising; it’s part of a broad effort to change minds.
Computational Impacts 