This is a hotly debated topic in so-called ‘tech’ circles and the academic and media groups that orbit that world like one of Jupiter’s radiation blasted moons. I dropped the phrase, ‘can large language models reason’ into Google, (that rusting machine) and got this result:
This is only a small sample. According to Google there are “About 352.000.000 results.” We can safely conclude from this, and the back and forth that endlessly repeats on Twitter in groups that discuss ‘AI’ that there is a lot of interest in arguing the matter: pro and con. Is this debate, if indeed it can be called that, the least bit important? What is at stake?
***
According to ‘AI’ industry enthusiasts, nearly everything is at stake; a bold new world of thinking machines is upon us. What could be more important? To answer this question, let’s do another Google search, this time, for the phrase, Project Nimbus:
The first result returned was a Wikipedia article, which starts with this:
Project Nimbus (Hebrew: פרויקט נימבוס) is a cloud computing project of the Israeli government and its military. The Israeli Finance Ministry announced in April 2021, that the contract is to provide “the government, the defense establishment, and others with an all-encompassing cloud solution.” Under the contract, the companies will establish local cloud sites that will “keep information within Israel’s borders under strict security guidelines.”
What sorts of things does Israel do with the system described above? We don’t have precise details but there are clues such as what’s described in this excerpt from the +972 Magazine article, ‘A mass assassination factory’: Inside Israel’s calculated bombing of Gaza’ –
According to the [+972 Magazine] investigation, another reason for the large number of targets, and the extensive harm to civilian life in Gaza, is the widespread use of a system called “Habsora” (“The Gospel”), which is largely built on artificial intelligence and can “generate” targets almost automatically at a rate that far exceeds what was previously possible. This AI system, as described by a former intelligence officer, essentially facilitates a “mass assassination factory.”
I wrote about Habsora in this essay. When I think of the ways algorithmic systems are used – increasingly to not only control but also to kill, my interest in the LLM debate fades, like the memory of a dream. It seems light and airy, unsuitable for the age.
***
History, and legend tell us that in ancient Athens there was a place called the Lyceum, founded by Aristotle, where the techniques of the Peripatetic school were practiced. Peripatetic means, more or less, ‘walking about’ which reflects the method: philosophers and students, mingling freely, discussing ideas. There are centuries of accumulated hagiography about this school. No doubt it was nice for those not subject to the slave system of ancient Greece.
Similarly, debates about whether or not LLMs can reason are nice for those of us not subject to hellfire missiles, fired by Apache helicopters sent on their errands based on targeting algorithms. But, I am aware of the pain of people who are subject to those missiles. I can’t unsee the death facilitated by computation.
This is why I have to leave the debating square, the social media crafted lyceum. Do large language models reason? No. But even spending time debating the question offends me now. A more pressing question is what the people building the systems killing our fellow human beings are thinking. What is their reasoning?
The IDF assault on Nasser hospital in Southern Gaza joined a long and growing list of bloody infamies committed by Israel since Oct 7, 2023. During a Democracy Now interview, broadcast on Feb 15, 2024, Dr. Khaled Al Serr, who was later kidnapped by the IDF, described what he saw:
“Actually, the situation here in the hospital at this moment is in chaos. All of the patients, all the relatives, refugees and also the medical staff are afraid because of what happened. We could not imagine that at any time the Israeli army will bomb the hospital directly, and they will kill patients and medical personnel directly by bombing the hospital building. Yesterday also, Israeli snipers and Israeli quadcopters, which is a drone, carry on it an AR, and with a sniper, they shot all over the building. And they shot my colleague, Dr. Karam. He has a shrapnel inside his head. I can upload for you a CT for him. You can see, alhamdulillah, it was superficial, nothing serious. But a lot of bullets inside their bedroom and the restroom.”
The Israeli military is using quadcopters, armed with sniper rifles, as part of its assassination arsenal. These remote operated drones, which possess limited but still important automatic capabilities (flight stability, targeting persistence) are being used in the genocidal war in Gaza and the war between Russia and Ukraine to name two, prominent examples. They are likely to make an appearance near you in some form, soon enough.
I haven’t seen reporting on the type of quadcopter used but it’s probably the Smash Dragon, a model produced by the Israeli firm Smart Shooter which, on its website, describes its mission:
SMARTSHOOTER develops state-of-the-art Fire Control Systems for small arms that significantly increase weapon accuracy and lethality when engaging static and moving targets, on the ground and in the air, day and night.
Here is a promotional video for the Smash Dragon:
Smart Shooter’s product, and profit source are the application of computation to the tasks of increasing accuracy and automating weapon firing. One of their ‘solutions’ (solving, apparently, the ‘problem’ of people being alive) is a fixed position ‘weapon station’ called the Smash Hopper that enables a distant operator to target-lock the weapon on a person, initiating the firing of a constant stream of bullets. For some reason, the cartoonish word, ‘smash’ is popular with the Smart Shooter marketing team.
‘AI’, as used under the current global order, serves three primary purposes: control via sorting, anti-labor propaganda and obscuring culpability. Whenever a hospital deploys an algorithmic system, rather than healthcare worker judgment, to decide how long patients stay, sorting is being used as a means of control, for profit. Whenever a tech CEO tells you that ‘AI’ can replace artists, drivers, filmmakers, etc. the idea of artificial intelligence is employed as an anti-labor propaganda tool. And whenever someone tells you that the ‘AI’ has decided, well, anything, they are trying to hide the responsibility of the people behind the scenes, pushing algorithmic systems on the world.
The armed quadcopter brings all of these purposes together, wrapped in a blood stained ribbon. Who lives and who dies is decided via remote control while the fingers pulling the trigger, and the people directing them are hidden from view. These systems are marketed as using ‘AI’ implying machines are making life and death decisions rather than people.
In the introduction to his 2023 book, The Palestine Laboratory, which details Israel’s role in the global arms trade and use of the Palestinians as lethal examples, journalist Anthony Lowenstein describes a weapons demonstration video attended by Andrew Feinstein in 2009:
“Israel is admired as a nation that stands on its own and is unashamed in using extreme force to maintain it. [Andrew Feinstein is] a former South African politician. journalist, and author. He told me about attending the Paris Air Show in 2009, the world’s largest aerospace industry and air show exhibitions. [The Israel-based defense firm Elbit Systems] was showing a promotional video about killer drones, which have been used in Israel’s war against Gaza and over the West Bank.
The footage had been filmed a few months before and showed the reconnaissance of Palestinians in the occupied territories. A target was assassinated. […] Months later, Feinstein investigated the drone strike and discovered that the incident featured in the video had killed a number of innocent Palestinians, including children. This salient fact wasn’t featured at the Paris Air Show. “This was my introduction to the Israeli arms industry and the way it markets itself.”
The armed quadcopter drone, one of the fruits of an industry built on occupation and death, can be added to the long list of the harms of computation. ‘Keep watching the skies!’ someone said at the end of a 1950s science fiction film whose name escapes me. Never mind though, the advice stands.
Confirmed: Dr. Khaled Al Serr, who took a lead role in trying to inform western press of the consequences of Israel’s attack on Nasser Hospital, has been abducted. Statement: https://t.co/ETI2hbcegzpic.twitter.com/emMypy29bD
This is written in the spirit of the Request for Comments memorandums that shaped the early Internet. RFCs, as they are known, are submitted to propose a technology or methodology and gather comments/corrections from relevant and knowledgeable community members in the hope of becoming a widely accepted standard.
Purpose
This is a consideration of the information technology options for politically and socially active organizations. It’s also a high level overview of the technical landscape. The target audience is technical decision makers in groups whose political commitments challenge the prevailing order, focused on liberation. In this document, I will provide a brief history of past patterns and compare these to current choices, identifying the problems of various models and potential opportunities.
Alongside this blog post there is a living document posted for collaboration here. I invite a discussion of ideas, methods and technologies I may have missed or might be unaware of to improve accuracy and usefulness.
Being Intentional About Technology Choices
It is a truism that modern organizations require technology services. Less commonly discussed are the political, operational, cost and security implications of this dependence from the perspective of activists. It’s important to be intentional about technological choices and deployments with these and other factors in mind. The path of least resistance, such as choosing Microsoft 365 for collaboration rather than building on-premises systems, may be the best, or least terrible choice for an organization but the decision to use it should come after weighing the pros and cons of other options. What follows is not an exhaustive history; I am purposefully leaving out many granular details to get to the point as efficiently as possible.
A Brief History of Organizational Computing
By ‘organizational computing’ I’m referring to the use of digital computers arranged into service platforms by non-governmental and non-military organizations. In this section, there is a high level walk through of the patterns which have been utilized in this sector.
Mainframes
IBM 360 in Computer Room – mid 1960s
The first use of digital computing at-scale was the deployment of mainframe systems as centrally hosted resources. User access, limited to specialists, was provided via a time sharing method in which ‘dumb’ terminals displayed results of programs and enabled input (punch cards were also used for inputting program instructions). One of the most successful systems was the IBM 360 (operational from 1965 to 1978). Due to expense, the typical customer was large banks, universities and other organizations with deep pockets.
Client Server
Classic Client Server Architecture (Microsoft)
The introduction of personal computers in the 1980s created the raw material for the development of networked, smaller scale systems that could supplement mainframes and provide organizations with the ability to host relatively modest computing platforms that suited their requirements. By the 1990s, this became the dominant model used by organizations at all scales (mainframes remain in service but the usage profile became narrower – for example, to run applications requiring greater processing capability than what’s possible using PC servers).
The client server model era spawned a variety of software applications to meet organizational needs such as email servers (for example, Sendmail and Microsoft Exchange), database servers (for ex. Postgres and SQL Server), web servers such as Apache and so on. Companies such as Novell, Cisco, Dell and Microsoft rose to prominence during this time.
As the client server era matured and the need for computing power grew, companies like VMWare sold platforms that enabled the creation of virtual machines (software mimics of physical servers). Organizations that could not afford to own or rent large data centers could deploy the equivalent of hundreds or thousands of servers within a smaller number of more powerful (in terms of processing capacity and memory) computing systems running VMWare’s ESX software platform. Of course, the irony of this return to something like a mainframe was not lost on information technology workers whose careers spanned the mainframe to client server era.
Cloud computing
Cloud Pattern (Amazon Web Services)
Virtualization, combined with the improved Internet access of the early 2000s, gave rise to what is now called ‘cloud.’ Among information technology workers, it was popular to say ‘there is no cloud, it’s just someone else’s computer.’ Overconfident cloud enthusiasts considered this to be the complaint of a fading old guard but it is undeniably true.
The Cloud Model
There are four modes of cloud computing:
Infrastructure as a service – IaaS: (for example, building virtual machines on platforms such as Microsoft Azure, Amazon Web Services or Google Cloud Platform)
Platform as a service – PaaS: (for example, databases offered as a service utility eliminating the need to create a server as host)
Software as a Service – SaaS: (platforms like Microsoft 365 fall into this category)
Function as a Service – FaaS: (focused on deployment using software development – ‘code’ – alone with no infrastructural management responsibilities)
A combination of perceived (but rarely realized) convenience, marketing hype and mostly unfulfilled promises of lower running costs have made the cloud model the dominant mode of the 2020s. In the 1990s and early 2000s, an organization requiring an email system was compelled to acquire hardware and software to configure and host their own platform (the Microsoft Exchange email system running on Dell server or VMWare hardware was a common pattern). The availability of Office 365 (later, Microsoft 365) and Google’s G-Suite provided another, attractive option that eliminated the need to manage systems while providing the email function.
A Review of Current Options for Organizations
Although tech industry marketing presents new developments as replacing old, all of the pre-cloud patterns mentioned above still exist. The question is, what makes sense for your organization from the perspectives of:
Cost
Operational complexity
Maintenance complexity
Security and exposure to vulnerabilities
Availability of skilled workers (related to the ability to effectively manage all of the above)
We needn’t include mainframes in this section since they are cost prohibitive and today, intended for specialized, high performance applications.
Client Server (on-premises)
By ‘on-premises’ we are referring to systems that are not cloud-based. Before the cloud era, the client server model was the dominant pattern for organizations of all sizes. Servers can be hosted within a data center the organization owns or within rented space in a colocation facility (a business that provides rented space for the servers of various clients).
Using a client server model requires employing staff who can install, configure and maintain systems. These skills were once common, indeed standard, and salaries were within the reach of many mid-size organizations. The cloud era has made these skills harder to come by (although there are still many skilled and enthusiastic practitioners). A key question is, how much investment does your organization want to make in the time and effort required to build and manage its own system? Additional questions for consideration come from software licensing and software and hardware maintenance cycles.
Sub-categories of client server to consider
Virtualization and Hyper-converged hardware
As mentioned above, the use of virtualization systems, offered by companies such as VMWare, was one method that arose during the heyday of client server to address the need for more concentrated computing power in a smaller data center footprint.
Hyper-converged infrastructure (HCI) systems, combining compute, storage and networking into a single hardware chassis, is a further development of this method. HCI systems and virtualization reduce the required operational overhead. More about this later.
Hybrid architectures
A hybrid architecture uses a mixture of on-premises and off-site, typically ‘cloud’ based systems. For example, an organization’s data might be stored on-site but the applications using that data are hosted by a cloud provider.
Cloud
Software as a Service
Software as a Service platforms such as Microsoft 365 are the most popular cloud services used by firms of all types and sizes, including activist groups. The reasons are easy to understand:
Email services without the need to host an email server
Collaboration tools (SharePoint and MS Teams for example) built into the standard licensing schemes
Lower (but not zero) operational responsibility
Hardware maintenance and uptime are handled by the service provider
The convenience comes at a price, both financial, as licensing costs increase and operational inasmuch as organizations tend to place all of their data and workflows within these platforms, creating deep dependencies.
Build Platforms
The use of ‘build platforms’ like Azure and AWS is more complex than the consumption model of services such as Microsoft 365. Originally, these were designed to meet the needs of organizations that have development and infrastructure teams and host complex applications. More recently, the ‘AI’ hype push has made these platforms trojan horses for pushing hyperscale algorithmic platforms (note, as an example, Microsoft’s investment in and use of OpenAI’s Large Language Model kit) The most common pattern is a replication of large-scale on-premises architectures using virtual machines on a cloud platform.
Although marketed as superior to, and simpler than on-premises options, cloud platforms require as much, and often more technical expertise. Cost overruns are common; cloud platforms make it easy to deploy new things but each item generates a cost. Even small organizations can create very large bills. Security is another factor; configuration mistakes are common and there are many examples of data breaches produced by error.
Private Cloud
The potential key advantage of the cloud model is the ability to abstract technical complexity. Ideally, programmers are able to create applications that run on hardware without the requirement to manage operating systems (a topic outside of the scope of this document). Private cloud enables the staging of the necessary hardware on-premises. A well known example is Openstack which is very technically challenging. Commercial options include Microsoft’s Azure Stack which extends the Azure technology method to hyper converged infrastructure (HCI) hosted within an organization’s data center.
Information Technology for Activists – What is To Be Done?
In the recent past, the answer was simple: purchase hardware and software and install and configure it with the help of technically adept staff, volunteers or a mix. In the 1990s and early 2000s it was typical for small to midsize organizations to have a collection of networked personal computers connected to a shared printer within an office. Through the network (known as a local area network or LAN) these computers were connected to more powerful computers called servers that provide centralized storage and the means through which each individual computer could communicate in a coordinated manner and share resources. Organizations often hosted their own websites which were made available to the Internet via connections from telecommunications providers.
Changes in the technology market since the mid 2000s, pushed to increase the market dominance and profits of a small group of firms (primarily, Amazon, Microsoft and Google) have limited options even as these changes appear to offer greater convenience. How can these constraints be navigated?
Proposed Methodology and Doctrines
Earlier in this document, I mentioned the importance of being intentional about technology usage. In this section, more detail is provided.
Let’s divide this into high level operational doctrines and build a proposed architecture from that.
First Doctrine: Data Sovereignty
Organizational data should be stored on-premises using dedicated storage systems rather than in a SaaS such as Microsoft 365 or Google Workspace
Second Doctrine: Bias Towards Hybrid
By ‘hybrid’ I am referring to system architectures that utilize a combination of on-premises and ‘cloud’ assets
Third Doctrine: Bias Towards System Diversity
This might also be called the right tool for the right job doctrine. After consideration of relevant factors (cost, technical ability, etc) an organization may decide to use Microsoft 365 (for example) to provide some services but other options should be explored in the areas of:
Document management and related real time collaboration tooling
Online Meeting Platforms
Database platforms
Email platforms
Commercial platforms offer integration methods between platforms that make it possible to create an aggregated solution from disparate tools.
These doctrines can be applied as guidelines for designing an organizational system architecture:
The above is only one option. More are possible depending on the aforementioned factors of:
Cost
Operational complexity
Maintenance complexity
Security and exposure to vulnerabilities
Availability of skilled workers (related to the ability to effectively manage all of the above)
I invite others to add to this document to improve its content and sharpen the argument.
Activist Documents and Resources Regarding Alternative Methods
Counter Cloud Action Plan – The Institute for Technology In the Public Interest
On September 14, 2023, while touring Twitter the way you might survey the ruins of Pompey, I came across a series of posts responding to this statement from the EU Commission account:
Mitigating the risk of extinction from AI should be a global priority…
What attracted critical attention was the use of the phrase, ‘risk of extinction‘ a fear of which, as Dr. Timnit Gebru alerts us (among others, mostly women researchers I can’t help but notice) lies at the heart of what Gebru calls the ´TESCREAL Bundle.’ The acronym, TESCREAL, which brings together the terms Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism and Longtermism, describes an interlocked and related group of ideologies that have one idea in common: techno-utopianism (with a generous helping of eugenics and racialized ideas of what ‘intelligence’ means mixed in to make everything old new again).
Risk of extinction. It sounds dramatic, doesn’t it? The sort of phrase you hear in a Marvel movie, Robert Downey Jr, as Iron Man stands in front of a green screen and turns to one of his costumed comrades as some yet to be added animated threat approaches and screams about the risk of extinction if the animated thing isn’t stopped. There are, of course, actual existential risks; asteroids come to mind and although climate change is certainly a risk to the lives of billions and the mode of life of the industrial capitalist age upon which we depend, it might not be ‘existential’ strictly speaking (though, that’s most likely a distinction without a difference as the seas consume the most celebrated cities and uncelebrated communities).
The idea that what is called ‘AI’ – which, when all the tech industry’s glittering makeup is removed, is revealed plainly to be software, running on computers, warehoused in data centers – poses a risk of extinction requires a special kind of gullibility, self interest, and, as Dr, Gebru reminds us, supremacist delusions about human intelligence to promote, let alone believe.
***
In the picture posted to X, Ursula von der Leyen, President of the European Commission, is standing at a podium before the assembled group of commissioners, presumably in the EU Commission building (the Berlaymont) in Brussels, a city I’ve visited quite a few times, regretfully. The building itself and the main hall for commissioners, are large and imposing, conveying, in glass, steel and stone, seriousness. Of course, between the idea and the act there usually falls a long shadow. How serious can this group be, I wondered, about a ‘risk of extinction’ from ‘AI’?
***
To find out, I decided to look at the document referenced and trumpeted in the post, the EU Artificial Intelligence Act. There’s a link to the act in the reference section below. My question was simple: is there a reference to ‘risk of extinction’ in this document? The word, ‘risk’, appears 71 times. It’s used in passages such as the following, from the overview:
The Commission proposes to establish a technology-neutral definition of AI systems in EU law and to lay down a classification for AI systems with different requirements and obligations tailored on a ‘risk-based approach’. Some AI systems presenting ‘unacceptable’ risks would be prohibited. A wide range of ‘high-risk’ AI systems would be authorised, but subject to a set of requirements and obligations to gain access to the EU market.
The emphasis is on a ‘risk based approach’ which seems sensible at first look but there are inevitable problems and objections. Some of the objections come from the corporate sector, claiming, with mind-deadening predictability, that any and all regulation hinders ‘innovation’ a word that is invoked like an incantation only not as intriguing or lyrical. More interesting critiques come from those who see risk (though, notably, not existential) and who agree something must be done but who view the EU’s act as not going far enough or going in the wrong direction.
Here is the listing of high-risk activities and areas for algorithmic systems in the EU Artificial Intelligence Act:
o Biometric identification and categorisation of natural persons
o Management and operation of critical infrastructure
o Education and vocational training
o Employment, worker management and access to self-employment
o Access to and enjoyment of essential private services and public services and benefits
o Law enforcement
o Migration, asylum and border control management
o Administration of justice and democratic processes
Missing from this list is the risk of extinction; which, putting aside the Act’s flaws, makes sense. Including it would have been as out of place in a consideration of real-world harms as adding a concern about time traveling bandits.. And so, now we must wonder, why include the phrase, “risk of extinction” in a social media post?
***
On March 22, 2023, the modestly named Future of Life Institute, an organization initially funded by the bathroom fixture toting Lord of X himself, Musk (a 10 million USD investment in 2015) whose board is as alabaster as the snows of Antarctica once were, kept afloat by donations from other tech besotted wealthies, published an open letter titled, ‘Pause Giant AI Experiments: An Open Letter.’ This letter was joined by similarly themed statements from OpenAI (‘Planning for AGI and beyond’) and Microsoft (‘Sparks of Artificial General Intelligence: Early experiments with GPT-4’).
Each of these documents has received strong criticism from people, such as yours truly, and others with more notoriety and for good reason: they promote the idea that the imprecisely defined Artificial General Intelligence (AGI) is not only possible, but inevitable. Critiques of this idea – whether based on a detailed analysis of mathematics (‘Reclaiming AI as a theoretical tool for cognitive science’) or of computational limits (The Computational Limits of Deep Learning) have the benefit of being firmly grounded in material reality.
But as Freud might have warned us, we live in a society shaped not only by our understanding of the world as it is but also, in no small part by dreams and fantasies. White supremacists harbor the self congratulating fantasy that any random white person (well, man) is an astounding genius when compared to those not in that club. This notion endures despite innumerable and daily examples to the contrary because it serves the interests of certain individuals and groups to persist in delusion and impose this delusion on the world. The ‘risk of extinction’ fantasy has caught on because it builds on decades of fiction, like the idea of an American Dream and adds spice to an otherwise deadly serious and grounded business: controlling the tech industry’s scope of action. Journalists who ignore the actual harms of algorithmic systems rush to write stories about a ‘risk of extinction’ which is far sexier than talking about the software now called ‘AI’ that is used to deny insurance benefits or determine criminal activity.
The European Union’s Artificial Intelligence Act does not explicitly reference ‘existential risk’ but the social media post using this idea is noteworthy. It shows that lurking in the background, the ideas promoted by the tech industry – by OpenAI and its paymaster Microsoft and innumerable camp followers – have seeped into the thinking of decision makers at the highest levels.
And how could it be otherwise? How flattering to think you’re rescuing the world from Skynet, the fictional, nuclear missile tossing system featured in the ‘Terminator’ franchise, rather than trying, at long last, to actually regulate Google.
In this essay, published for The Nation magazine, I argue that the primarily purpose of the application of generative methods to the production of images (what some insist upon calling AI art) is the creation of an assembly line.
Several years ago, there was a mini-trend of soft documentaries depicting what would happen to the built environment if humans somehow disappeared from the Earth. How long, for example, would untended skyscrapers punch against the sky before they collapsed in spectacular, downward cascading showers of steel and glass onto abandoned streets? These are the sorts of questions posed in these films.
As I watched these soothing depictions of a quieter world, I sometimes imagined a massive orbital tombstone, perhaps launched by the final rocket engineers, onto which was etched: Wasted Potential.
While I type these words, billions of dollars have been spent on and barely tabulated amounts of electrical power, water and human labor (barely tabulated, because deliberately obscured) have been devoted to large language model (LLM) systems such as ChatGPT. If you follow the AI critical space you’re familiar with the many problems produced by the use and promotion of these systems – including, on the hype end, the most recent gyration, a declaration of “existential risk” by a collection of tech luminaries (a category which, in a Venn diagram, overlaps with carnival barker). This use of mountains of resources to enhance the profit objectives of Microsoft, Amazon and Google, among other firms not occupying their olympian perches, is wasted potential in frenetic action.
But what of alternative visions? They exist, all is not despair. The dangerous nonsense relentlessly spewing from the AI industry is overwhelming and countering it is a full time pursuit. But we can’t stay stuck, as if in amber, in a state of debunking and critique. There must be more. I recommend the DAIR Institute and Logic(s) magazine as starting points for exploring other ways of thinking about applied computation. Ideologically, AI doomerism is fueled in large measure by dystopian pop sci-fi such as Terminator. You know the story, which is a tale as old as the age of digital computers: a malevolent supercomputer – Skynet (a name that sounds like a product) – launches, for some reason, a war on humanity, resulting in near extinction. The tech industry seems to love ripping dystopian yarns. Judging by the now almost completely forgotten metaverse push (a year ago, almost as distant as the pleistocene in hype cycle time), inspired by the less than sunny sci-fi novel Snow Crash, we can even say that dystopian storylines are a part of business plans (what is the idea of sitting for hours wearing VR goggles if not darkly funny?).
There are also less terrible, even hopeful, fictional visions, presented via pop science fiction such as Star Trek´s Library Computer Access/Retrieval System – LCARS.
In the Star Trek: The Next Generation episode, “Booby Trap” the starship Enterprise is caught in a trap, composed of energy sapping fields, that prevents it from using its most powerful mode of propulsion, warp drive. The ship’s chief engineer, Geordi LeForge, is given the urgent task of finding a solution. LeForge realizes that escaping this trap requires a re-configuration, perhaps even a new understanding, of the ship’s propulsion system. That’s the plot but most intriguing to me is the way LeForge goes about trying to find a solution.
The engineer uses the ship’s computer – the LCARS system – to do a retrieval and rapid parsing of the text of research and engineering papers going back centuries. He interacts with the computer via a combination of audio and keyboard/monitor. Eventually, LeForge resorts to a synthetic, holo mockup of the designer of the ship’s engines, Dr. Leah Brahms, raising all manner of ethical issues but we needn’t bother with that plot element.
I’ve created a high level visualisation of how this fictional system is portrayed in the episode:
The ability to identify text via search, to summarize and read contents (with just enough contextual capability to be useful) and to output relevant results is rather close, conceptually, to the potential of language models. The difference between what we actually have – competing and discrete systems owned by corporations – and LCARS (besides many orders of magnitude of greater sophistication in the fictional system) is that LCARS is presented as an integrated, holistic and scoped system. LCARS’ design is to be a library that enables access to knowledge and retrieves results based on queried criteria.
There is a potential, latent within language models and hybrid systems – indeed, within almost the entire menagerie of machine learning methods – to create a unified computational model for a universally useful platform. This potential is being wasted, indeed, suppressed as oceans of capital, talent and hardware is poured into privately owned things such as ChatGPT. There are hints of this potential found within corporate spaces; Meta’s LLaMA, which leaked online, shows one avenue. There are surely others.
Among a dizzying collection of falsehoods, the tech industry’s greatest lie is that it is building the future. Or perhaps, I should sharpen my description: the industry may indeed be building the future but contrary to its claims, it is not a future with human needs centered. It is possible however, to imagine and build a different computation and we needn’t turn to Silicon Valley’s well thumbed library of dystopian novels to find it. Science fiction such as Star Trek (I’m sure there are others) provide more productive visions
What should we think about AI? To corporate boosters and their camp followers (an army of relentless shouters) , so-called artificial intelligence is a world altering technology, sweeping across the globe like a wave made from the plots of forgotten science fiction novels. Among critics, thoughts are more varied. Some focus on debunking hyped claims, others, on the industry’s racist conceptions (such as the presentation of a cohort of men, mostly White, who work with ‘code’ as being the pinnacle of human achievement) and still others, on the seldom examined ideology of ‘intelligence’ itself.
For Dan McQuillan, author of the taut (seven chapters) yet expansive book, ‘Resisting AI: An Anti-Facist Approach to Artificial Intelligence’ AI, is, under current conditions but not inherently, the computational manifestation of ever present fascist ideologies of control, categorization and exclusion. McQuillan has written a vital manifesto, the sort of work which, many years from now, may be recalled, if we’re fortunate, as being among the defining calls to arms of its age. In several interviews (including this one for Machine Learning Street Talk) McQuillan has described the book’s origin as a planned, scholarly review of the industry that, as its true state became clearer to him, evolved into a warning.
We can be glad he had the courage to follow the evidence where it led.
Both In and Of the World
“The greatest trick the Devil ever pulled” the saying goes, “was convincing the world he doesn’t exist.” The tech industry, our very own Mephistopheles (though lacking the expected fashion sense) has pulled a similar trick with ‘AI’ convincing us that, alone among technical methods, it exists as a force disconnected from the world’s socio-political concerns. In short order, McQuillan dispenses with this in the introduction:
It would be troubling enough if AI was a technology being tested in the lab or applied in a few pioneering startups, but it already has huge institutional and cultural momentum. […] AI derives a lot of its authority from its association with methods of scientific analysis, especially abstraction and reduction, an association which also fuels the hubris of some of its practitioners. The roll out of AI across swathes of industry doesn’t so much lead to a loss of jobs as to an amplification of casualized and precarious work. [emphasis mine] Rather than being an apocalyptic technology, AI is more aptly characterized as a form of supercharged bureaucracy that ramps up everyday cruelties, such as those in our systems of welfare. In general, […] AI doesn’t lead to a new dystopia ruled over by machines but an intensification of existing misery through speculative tendencies that echo those of finance capital. These tendencies are given a particular cutting edge by the way Al operates with and through race. AI is a form of computation that inherits concepts developed under colonialism and reproduces them as a form of race science. This is the payload of real AI under the status quo. [Introduction, pg 4]
Rather than acting as the bridge to an unprecedented new world, AI systems (really, statistical inference engines) are the perfect tool for the continuance of existing modes of control, intensified and excused by the cover of supposed silicon impartiality.
Later, in chapter two, titled, ‘AI Violence’ McQuillan sharpens his argument that the systems imposed on us are engines of automated abuse.
AI operationalizes [a] reductive view through its representations. […] , Aľ’s representations of the world consist of the set of weights in the [processing] layers plus the model architecture of the layers themselves. Like science, Al’s representations are presented as distinct from that which they claim to represent. In other words, there is assumed to be an underlying base reality that is independent of the practices by which such representations are constructed. But […] the entities represented by AI systems- the ‘careful Amazon driver’ or the ‘trustworthy citizen’- are partly constructed by the systems that represent them. AI needs to be understood not as an instrument of scientific measurement but as an apparatus that establishes ‘relations of becoming between subjects and representations. The subject co-emerges along with the representation. The society represented by AI is the one that it actively produces.
We are familiar with the categories McQuillan highlights such as ‘careful drivers’ from insurance and other industries and government agencies which use the tagging and statistical sorting of discrete attributes to manage people and their movements within narrow parameters. AI, as McQuillan repeatedly stresses, supercharges already existing methods and ways of thinking, embedded within system logic. We don’t get a future, we are trapped in a frozen present, in which new thinking and new arrangements are inhibited via the computational enforcement of past structures.
Necropolitics
For me, the most powerful diagnostic section of the book is chapter 4, ‘Necropolitics.’ Although McQuillan is careful to not declare AI systems fascist by nature (beginning the work of imagining other uses for computational infrastructure in Chapter 5, ‘Post Machinic Learning’) he does make the critical point that these systems, embedded within a fraying political economy, are being promoted and made inescapable at a moment of mounting danger:
Al is entangled with our systems of ordering society. […] It helps accelerate a shift towards far-right politics. AI is emerging from within a convolution of ongoing crises, each of which has the potential to be fascism-inducing, including austerity, COVID-19 and climate change. Alongside these there is an internal crisis in the ‘relations of oppression’, especially the general destabilization of White male supremacy by decolonial, feminist, LGBTQI and other social movements (Palheta, 2021). The enrollment of AI in the management of these various crises produces ‘states of exception’ – forms of exclusion that render people vulnerable in an absolute sense. The multiplication of algorithmic states of exception across carceral, social and healthcare systems makes visible the necropolitics of Al; that is, its role in deciding who should live and who should be allowed to die.
As 20th century Marxists were fond of saying, it is no accident that as the capitalist social order faces ever more significant challenges, ranging from demands from the multitudes subjected to its tyranny to the growing instability of nature itself as climate change’s impacts accelerate, there is a turn, by elites, to a technology of command and control to reassert some sense of order. McQuillan’s urgency is born of a recognition of global emergency and the ways the collection of computational methods called ‘AI’ is being marshalled to meet that emergency using what can clearly be identified as fascist approaches. There’s much more to say but I will leave it here so you can explore on your own. Resisting AI: An Anti-Facist Approach to Artificial Intelligence, is an important and necessary book.
As the hype, indeed, propaganda about AI and its supposed benefits and even dangers (such as the delusions about ‘superintelligence’ a red herring) are broadcast ever more loudly, we need a collectivity of counterbalancing ideas and voices. McQuillan has provided us with a powerful contribution.
[In this post, I imagine that I’m writing to a researcher who, disappointed, and perhaps confused by the seemingly unstoppable corporate direction their field is taking, needs a bit of, well, not cheering up precisely but, something to help them understand what it all means and how to resist]
My friend,
Listen, I know you’ve been thrown by the way things have been going for the past few years – really, the past decade; a step by step privatization of the field you love and education pursued at significant financial cost (you’re not a trust funder) because of your desire to understand cognition and just maybe, build systems that, through their cognitive dexterity, aid humanity (vainglorious but, why not aim high?) You thought of people such as McCarthy, Weizenbaum, Minsky and Shannon and hoped to blaze trails, as they did.
When OpenAI hit the scene in 2015, with the promise – in its very name – to be an open home for advanced research, you celebrated. Over wine, we argued (that’s too strong, more like warmly debated with increasing heat as the wine flowed) about the participation of sinister figures such as Musk and Thiel. At the time, Musk was something of a hero to you and Thiel? Well, he was just a quirky VC with deep pockets and an overlooked penchant for ideas that are a bit Goebbels-esque. “Form follows function,” I said, “and the function of these people is to find ways to generate profit and pretend they’re gods.” But we let that drop over glasses of chardonnay.
Here we are, in 2023… which for you, or more pointedly your dreams, has become an annus horribilis, a horrible year. OpenAI is now married to Microsoft and the much anticipated release of GPT-4 is, in its operational and environmental impact details, shrouded in deliberate mystery. AI ethics teams are discarded like used tissues – there is an air of defeat as the idea of the field you thought you had joined dies the death of a thousand cuts.
Now is the time to look around and remember what I told you all those years ago: science and engineering (and your field contains both these things) do not exist outside of the world but are very much in it and are subject to a reality described by the phrase you’ve heard me say a million times: political economy. Our political economy – or, I should say, the political economy (the interrelations of law, production, custom and more) we’re subject to, is capitalist. What does this mean for your field?
It means that the marriage between OpenAI and Microsoft, the integration of large language models with the Azure cloud and the M365 SaaS platforms, the elimination of ethics teams whose work might challenge or impede marketing efforts, the reckless proliferation of algorithmically enacted harms is all because the real goal is profit, which is at the heart of capitalist political economy.
And we needn’t stop with Microsoft; there is no island to run to, no place that is outside of this political economy. No, not even if your team and leadership are quite lovely. This is a totalitarian (or, if you’re uncomfortable with that word, hegemonic) system which covers the globe in its harsh logic.
Oh but now you’re inclined to debate again and it’s too early for wine. I can hear you saying, ‘We can create an ethical AI; it’s possible. We can return to the research effort of years past’ I won’t say it’s impossible, stranger things have presumably happened in the winding history of humanity, but taking the whole fetid situation into account – yes, the relationship between access to computation and socio-technical power, the political economy, it’s not probable. So long as you continue believing in something that the structure of the society we live in does not support, you will continue to be disappointed.
Unless, that is, that structure is changed.
What is to be done?
I don’t expect you to become a Marxist (though it would be nice, we could compare obscure notes about historical materialism) but what I’m encouraging you to consider is that the world we grew up in and, quite naturally take for granted as immutable – the world of capitalist social relations, the world which, among other less than fragrant things, has all but completely absorbed your field into its profit engine is not the only way to organize human society.
Once you accept that, we can begin to talk about what might come next.
A few years ago, I started this very blog, devoted, as the subtitle reads, to “AI industry analysis without hype and techbro-ism.” Writing, when seriously pursued (whether money is exchanged or not) is a demanding activity, requiring time and often, a reduced number of social interactions, things that are becoming ever scarcer in our decaying world of enforced busy-ness and endlessly distracting ‘discourse.’
Considering the difficulty, why bother writing? And why bother writing about the tech industry generally, and its so-called ‘AI’ incarnation specifically? Until very (very) recently, the unchallenged cultural consensus was that Silicon Valley is populated by a wondrous horde of luminous creatures, the brilliant young who, armed only with wafer thin laptops, dreams, and that sorcerer’s wand, code, were building a vibrant future of robot taxis, chatbot friends and virtual worlds filled with business meetings attended by cartoon dinosaur avatars.
Who could resist this vision, this nirvana of convenience? Well, as it happens, yours truly.
It was while watching a left-leaning (and at the time, supposedly Marxist) YouTube show that I realized there was an acute need for a pitiless, materialist critique of the tech industry. One of the show’s co-hosts opined that it would not be long before robot trucks replaced actual truckers, changing the political economy of logistics in the US. This is not remotely close to happening (as one of that program’s guests, a trucker, pointed out) and so, I wondered why this idea was asserted with the same confidence of a Tesla press release about full self-driving…happening, any day now.
The reason is a lack of understanding of how actually existing computational systems work. This isn’t a sin; the world is complex and we all can’t be experts in everything (though there’s a large army of men who assume they can, for example, perform surgery, fly fighter jets and wrestle bears – the scientific term for such men is idiot). As it happens, my decades of experience with computation, combined with an unequivocally Marxist (therefore, materialist) understanding of capitalism seemed to make me qualified to fill this niche from a unique perspective – not from the distance of academics but feeling the cold chill of data centers.
And so, I started this blog, a sisyphean effort, of unknown utility but necessary, if only to help me achieve some measure of clarity.
But, how to write about the tech industry? What ‘voice’, to lean on a cliche, should be used? In the beginning, I wrote like a war correspondent (or at least, what I supposed to be the attitude of a war correspondent) : urgent, sparse, accessibly technical. The enemy was clearly identified, the stark facts countering mythology plainly stated. There was no time for leisurely applied words. In an earlier age, when fedoras and smoking on planes were common, this might have been called a ‘muscular’ style (which evokes the image of a body builder, busily typing on a keyboard after leg day at the gym). I imagined myself in a smart, yet disheveled suit, sitting on-set with Dick Cavett in a forever 1969 Manhattan, a Norman Mailer of tech critique, though without the nasty obsession with performative manliness.
Something pulls at me, another ‘voice’ which has moved me, by degrees, away from reports from the front to an even sharper-edged approach, one informed by a combination of disdain for the target – an intrusive and destructive industry – and deep concern for its victims: all of us, nearly everywhere. This writing personna is closer to my day to day self – not a perfect mirror, but more recognizable.
This person at the keyboard, this version of Dwayne who tries to convey to you, esteemed reader, the true danger posed by the tech industry and the various illusions it promotes, is a man who refuses to be fooled or, at least, to walk into delusion willingly, without struggle.
Raymond Chandler – 1943
Now, as I write, my thoughts turn to an essay about detective fiction Raymond Chandler penned for The Atlantic magazine in 1950 titled, ‘The Simple Art of Murder.’ About writing, as a craft, Chandler wrote:
“The poor writer is dishonest without knowing it, and the fairly good one can be dishonest because he doesn’t know what to be honest about.”
Honesty. This is the goal; an honest accounting of the situation we’re in and what we’re up against – capitalist political economy, supply chains, resources extraction and data centers as a form of sociotechnical power – a rejection of the Californian Ideology; no, not just a rejection, but a hard boiled reaction to it, a Noir response.
To close this, which is a work in progress, let’s return to Chandler’s essay about detective fiction, “The Simple Art of Murder” –
“It is not a very fragrant world, but it is the world you live in, and certain writers with tough minds and a cool spirit of detachment can make very interesting and even amusing patterns out of it. It is not funny that a man should be killed, but it is sometimes funny that he should be killed for so little, and that his death should be the coin of what we call civilization. All this still is not quite enough.”
The world itself may be lovely but the world the tech industry has built and which it seeks to entrench is ‘not very fragrant’ indeed; in fact, it is a nightmare. Resistance requires passion but also, as Chandler wrote of his fictional hero, Phillip Marlowe, a tough mind and cool spirit of detachment. No, I will not celebrate AI and each gyration of an industry whose goal is to act as the means through which labor’s power is suppressed.
Enough wide eyed belief; time for productive cynicism.
I have avoided writing about ChatGPT as one might hurriedly walk past a group of co-workers, gathered around a box of donuts who’re talking about a popular movie or show; to avoid being drawn into the inevitable.
In some circles, certainly the circles I travel in, ChatGPT is the relentless talk of the town. Everyone from LinkedIn hucksters who claimed to be making millions from the platform, only moments after it was released, to the usual ‘AI’ enthusiasts who take any opportunity to sweatily declare a new era of machine intelligence upon us – and of course, a scattering of people carefully analyzing the actually existing nuts and bolts – everyone seems to be promoting, debating and shouting about ChatGPT.
You can imagine me, dear reader, in the midst of this drama, quietly sitting in a timeworn leather chair, slowly sipping a glass of wine while a stream of text, video and audio, all about ChatGPT, that silicon, would-be Golem, washes over me
What roused me from my torpor was the news Microsoft was investing 10 billion dollars in OpenAI, the organization behind ChatGPT and other ballyhooed large language model systems (see: “Microsoft’s $10bn bet on ChatGPT developer marks new era of AI”). Even for Microsoft, that’s a lot of money. Behind all this, is Microsoft’s significant investment in what it calls purpose built, AI supercomputers such as VOYAGER-EUS2 to train and host platforms such as ChatGPT. Although tender minded naifs believe corporations are using large scale computation to advance humanity, more sober minds are inclined to ask fundamental questions such as, why?
The answer came from the Microsoft article, “General availability of Azure OpenAI Service expands access to large, advanced AI models with added enterprise benefits.” Note that phrase, ‘enterprise benefits.’ The audience for this article is surely techie and techie adjacent (and here, I must raise my hand) but even if neither of these categories describes you I suggest giving it a read. There’s also an introductory video, providing a walkthrough of using the OpenAI tooling that’s mediated via the Microsoft Azure cloud platform.
Microsoft Video on OpenAI Platforms, Integrated with Azure
As I watched this video, the purpose of all those billions and the hardware it bought became clear to me; Microsoft and its chief competitors, Amazon and an apparently panicked Google (plus, less well known organizations) are seeking to extend the rentier model of cloud computing, which turns computation, storage and database services into a rented utility and recurring revenue source for the cloud firm that maintains the hardware – even for the largest corporate customers – into the ‘AI’ space, creating super rentier platforms which will spawn subordinate, sub-rentier platforms:
Imagine the following…
A San Francisco based startup, let’s give it a terrible name, Talkist, announces it has developed a remarkable, groundbreaking chat application (and by the way, ‘groundbreaking’ is required alongside ‘next generation’) which will enable companies around the world to replace customer service personnel with Talkist’s ‘intelligent’, ‘ethical’ system. Talkist, which only consists of a few people (mostly men) and a stereotypical, ‘visionary’ leader, probably wearing a thousand dollar t-shirt, doesn’t have the capital, or the desire to build the computational infrastructure required to host such a system.
This is where the Azure/OpenAI complex of systems comes to the rescue of our plucky band of well-funded San Franciscans. Instead of diverting precious venture capital into purchasing data center space and the computers to fill it, that money can be poured into creating applications which utilize Microsoft/OpenAI cloud services. Microsoft/OpenAI rent ‘AI’ capabilities to Talkist who in turn, rent ‘AI’ capabilities to other companies who think they can replace people with text generating, pattern matching systems (ironically, OpenAI itself is dependent on exploited labor as the Time Magazine article, “OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic” shows).
What a time to be alive.
Of course, the uses (and from the perspective of profit-driven organizations, cost savings) don’t end with chatty software. We can imagine magazines and other publications, weary of having to employ troublesome human beings with their demands for salaries, health care and decent lives (The gall! Are there no workhouses? Are there no prisons?) rushing to use these systems to ‘write’ – or perhaps we should say, mechanistically assemble, articles and news stories, reducing the need for writers who are an annoying class (I wink at you dear reader for I am the opposite of annoying – being a delightful mixture of cologne, Bordeaux and dialectical analysis). Unsurprisingly, and let’s indulge our desire for a bit of the old schadenfreude, amusingly there are problems such as those detailed in the articles “CNET Is Reviewing the Accuracy of All Its AI-Written Articles After Multiple Major Corrections. and, “CNET’s AI Journalist Appears to Have Committed Extensive Plagiarism.”
…
Of all the empires that have stalked the Earth, the tech imperium is, perhaps, the bullshitiest. The Romans derived their power from myths, yes, but also, roads, aqueducts and organized violence – real things in a real world. The US empire has its own set of myths, such as a belief that sitting in a car, in traffic, is the pinnacle of freedom and in meritocracy (a notion wielded by the most mediocre minds to explain their comforts). Once again however, real things, such as possessing the world’s reserve currency and the capacity for ultra-violence lurk behind the curtain.
The tech empire, by contrast, is built, using the Monorail maneuver detailed in this Simpsons episode, on false claims prettily presented. It has inserted itself between us and the things we need – information, memories, creativity. The tech industry has hijacked a variety of commons and then rents us access to what should be open. In its ‘AI’ incarnation, the tech industry attempts to replace human reason with computer power, a fool’s errand, which computer scientist Joseph Weizenbaum dissected almost 50 years ago, but a goal motivated by a desire to increase the rate of profit in an era of creeping stagnation by reducing the need for labor.
Rather than being a refutation of Marx and Engel’s analysis as some, such as Yanis Varoufakis with his ‘cloudalist’ hypothesis bafflingly claim, we are indeed, still very much dealing with the human grinding workings of capitalist logics, wearing a prop, science fiction film costume, claiming to have come in peace.
ChatGPT isn’t a research platform or the herald of a new age of computation; it is the embodiment of the revenue stream dreams of the tech industry, the super-rentier.
Computational Impacts 