Tag Archives: ai

Pygmalion Displacement – A Review

Posted on by
Reply

From the beginning, like a fast talking shell game huckster, the computer technology industry has relied on sleight of hand. 

First, in the 1950s and 60s, to obscure its military origins and purposes by describing early electronic computers as ‘electronic brains’ fashioned from softly glowing arrays of vacuum tubes. Later, by the 1980s, as the consumer electronics era was launched, the industry presented itself as the silicon wielding embodiment of ideas of ‘freedom’ and ‘self expression’ that are at the heart of the Californian Ideology (even as it was fully embedded within systems of command, control and counter-insurgency).

The manic, venture capitalist funded age of corporate ‘AI’ we’re currently subjected to has provided the industry with new opportunities for deception; we are encouraged to believe large language models and other computationally enacted, statistical methods are doing the same things as minds. Earlier, I called this deception but as Lelia A. Erscoi, Annelies Kleinherenbrink, and Olivia Guest, describe in their paper, “Pygmalion Displacement: When Humanising AI Dehumanises Women“, a more precise term is, displacement.

Uniquely for the field of AI critique, ‘Pygmalion Displacement’ identifies the specific ways women have been theorized and thought about within Western societies and how these ideas have persisted into, and shaped the computer age. 

The paper’s abstract introduces the reader to the authors’ concept:

We use the myth of Pygmalion as a lens to investigate the relationship between women and artificial intelligence (AI). Pygmalion was a legendary king who, repulsed by women, sculpted a statue, which was imbued with life by the goddess Aphrodite. This can be seen as a primordial AI-like myth, wherein humanity creates life-like self-images. The myth prefigures gendered dynamics within AI and between AI and society. Throughout history, the theme of women being replaced by automata or algorithms has been repeated, and continues to repeat in contemporary AI technologies. However, this pattern—that we dub Pygmalion displacement—is under-examined, due to naive excitement or due to an unacknowledged sexist history of the field. As we demonstrate, Pygmalion displacement prefigures heavily, but in an unacknowledged way, in the Turing test: a thought experiment foundational to AI. With women and the feminine being dislocated and erased from and by technology, AI is and has been (presented as) created mainly by privileged men, subserving capitalist patriarchal ends. This poses serious dangers to women and other marginalised people. By tracing the historical and ongoing entwinement of femininity and AI, we aim to understand and start a dialogue on how AI harms women.

Pygmalion Displacement: When Humanising AI Dehumanises Women – Pg 1

Like all great theoretical frameworks (such as Marx’s dialectical and historical materialism), Pygmalion Displacement provides us with a toolkit, the Pygmalion Lens, which can be applied to real world situations and conditions, sharpening our understanding and revealing what is hiding in plain sight, obscured by ideology.

Pygmalion Lens Table: Pygmalion Displacement: When Humanising AI Dehumanises Women, Pg 14

Apex Delusions

We generally assume that humanity – whether via evolutionary process or divine creation – is at the top of a ladder of being. Many of us love our dogs and cats but believe that because we build rockets and computers and they don’t, we occupy a loftier perch (I recall a Chomsky lecture during which he threw cold water on this vainglory by observing that the creation of nuclear weapons suggested our vaunted intelligence ‘may not be a successful adaptation’).

In the Introduction section titled, ‘The man, the myth,’ the authors describe another rung on this mythical ladder:

At the top of the proverbial food chain, a majority presence consists of straight white men, those who created, profit from, and work to maintain the capitalist patriarchy and kyriarchy generally (viz. Schüssler Fiorenza 2001). From this perspective, AI can be seen as aiming to seal all humanity’s best qualities in an eternal form, without the setbacks of a mortal human body. It is up for debate, however, what this idealised human(oid) form should look or behave like. When our creation is designed to mimic or be compatible with us, its creator, it will enact, fortify, or extend our pre-existing social values. Therefore, in a field where the vast majority is straight, cisgender, white, and male (Lecher 2019), AI seems less like a promise for all humanity and more like contempt for or even a threat against marginalized communities.

Pygmalion Displacement: When Humanising AI Dehumanises Women – Pg 3

The AI field, dominated by a small cohort, is shaped not only by the idea of humans as superior to the rest of nature but certain humans being superior to others. The imagined artificial general intelligence (AGI) is not simply a thinking machine, but a god-like, machine version of the type of person seen as being at the apex of humanity.

Further on in the introduction, the authors describe how these notions impact women specifically:

Our focus herein is on women in particular, who dwell within the limits of what is expected, having to adhere to standards of ideal and colonial femininity to be considered adequate and then sexualized and deemed incompetent for conforming to them (Lugones 2007). Attitudes towards women and the feminised, especially in the field of technology, have developed over a timeline of gender bias and systemic oppression and rejection. From myths, to hidden careers and stolen achievements (Allen 2017; Evans 2020), to feminized machines, and finally to current AI applications, this paper aims to shine a light on how we currently develop certain AI technologies, in the hope that such harms can be better recognized and curtailed in the future.

Pygmalion Displacement: When Humanising AI Dehumanises Women – Pg 3

On Twitter, as in our walkabout lives, we see and experience these harms in action as the contributions of women in science and technology (and much else besides) are dismissed or attributed to men. I always imagine an army of Jordan Peterson-esque pontificators but alas these pirates come in all shapes and sizes.

From Fiction to History and Back Again

Brilliantly, the authors create parallel timelines – one fictional, the other real – to illustrate how displacement has worked in cultural production and material outcomes.

In the fictional timeline, which includes stories ranging from ‘The Sandman’ (1816) to 2018’s PS4 and PC sci-fi adventure game, Detroit: Become Human, we are shown how displacement is woven into our cultural fabric.

Consider this passage on the 2013 film, ‘Her’ which depicts a relationship (of sorts) between Theodore, a lonely writer, played by Joaquin Phoenix and an operating system named Samantha, voiced by Scarlett Johansson:

…it is interesting to note that unlike her fictional predecessors, Samantha has no physical form — what makes her appear female is only her name and how she sounds (voiced by Scarlett Johansson), and arguably (that is, from a stereotypical, patriarchal perspective) her cheerful and flirty performance of secretarial, emotional, and sexual labor. In relation to this, Bergen (2016) argues that virtual personal assistants like Siri and Alexa are not perceived as potentially dangerous AI that might turn on us because, in addition to being so integrated into our lives, their embodied form does not evoke unruliness or untrustworthiness: “Unlike Pygmalion’s Galatea or Lang’s Maria, today’s virtual assistants have no body; they consist of calm, rational and cool disembodied voices […] devoid of that leaky, emotive quality that we have come to associate with the feminine body” (p. 101). In such a disembodied state, femininity appears much less duplicitous—however, in Bergen’s analysis, this is deceptive: just as real secretaries and housekeepers are often an invisible presence in the house owing to their femininity (and other marginalized identity markers), people do not take virtual assistants seriously enough to be bothered by their access to private information.

Pygmalion Displacement: When Humanising AI Dehumanises Women – Pg 8

Fictional depictions are juxtaposed with real examples of displacement such as the often told (in computer history circles) but not fully appreciated story of the ELIZA and Pedro speech generation systems:

Non-human speech generation has a long history, harking back to systems such as Pedro the voder (voice operating demonstration) in the 1930s (Eschner 2017). Pedro was operated solely by women, despite the fact the name adopted is stereotypically male. The first modern chatbot, however, is often considered to be ELIZA, created by Joseph Weizenbaum in 1964 to simulate a therapist that resulted in users believing a real person was behind the automated responses(Dillon 2020; Hirshbein 2004). The mechanism behind ELIZA was simple pattern matching, but it managed to fool people enough to be considered to have passed the Turing test. ELIZA was designed to learn from its interactions, (Weizenbaum 1966) named precisely for this reason. In his paper introducing the chatbot, Weizenbaum (1966) invokes the Pygmalion myth: “Like the Eliza of Pygmalion fame, it can be made to appear even more civilized, the relation of appearance to reality, however, remaining in the domain of the playwright.” (p. 36) Yet ELIZA the chatbot had the opposite effect than Weizenbaum intended, further fuelling a narrative of human-inspired machines.

Pygmalion Displacement: When Humanising AI Dehumanises Women – Pg 20

Later in this section, quoting from a work by Sarah Dillon on ‘The Eliza Effect’ we’re told about Weizenbaum’s contextual gendering of ELIZA:

Weizenbaum genders the program as female when it is under the control of the male computer programmer, but it is gendered as male when it interacts with a [female] user. Note in particular that in the example conversation given [in Weizenbaum’s Computer Power and Human Reason, 1976], this is a disempowered female user, at the mercy of her boyfriend’s wishes and her father’s bullying, defined by and in her relationship to the men whom, she declares, ‘are all alike.’ Weizenbaum’s choice of names is therefore adapted and adjusted to ensure that the passive, weaker or more subservient position at any one time is always gendered as female, whether that is the female-gendered computer program controlled by its designers, or the female-gendered human woman controlled by the patriarchal figures in her life.

Pygmalion Displacement: When Humanising AI Dehumanises Women – Pg 21

This passage was particularly interesting to me because I’ve long admired Weizenbaum’s thoughtful dissection of his work. I learned from the critique of computation as an ideology but missed his Pygmalion framing; the Pygmalion Lens enables a new way of seeing assumptions and ideas that are taken for granted like the air we breathe.

There is much more to discuss such as an eye-opening investigation into the over-celebrated Turing Test (today, more marketing gimmick than assessment technique) which began as a theorized method to create a guessing game about gender, a test which (astoundingly) “…required a real woman […] to prove her own humanity in competition with the computer.”

This is a marvellous and important paper which presents more than a theory, it gives us a toolkit and method for changing the way we think about the field of computation (and its loud ‘AI’ partisans) under patriarchal capitalism 

Kinetic Harm

Posted on by
Reply

I write about the information technology industry.

I’ve written about other topics, such as the copaganda of Young Turks’ host Ana Kasparian and Zizek, whose work, to quote John Bellamy Foster, has become “a carnival of irrationalism.” In the main, however, the technology industry generally, and its so-called ‘AI’ sub-category, specifically, are my topics. This isn’t random; I’ve worked in this industry for decades and know its dark heart. Honest tech journalism (rather than the boosterism we mostly get) and scholarly examinations are important but, who better to tell a war story than someone in the trenches?

Because I focus on harm and not the fantasy of progress, this isn’t a pursuit that brings wealth or notoriety. There have been a few podcast appearances (a type of sub-micro celebrity, as fleeting as a lightning flash) and opportunities to be published in respected magazines. That’s nice, as far as it goes. It’s important however, to see clearly and be honest with yourself; it’s a sisyphean task with few rewards; motivations must be found within and from a community of like minded people.

Originally, my motivation was to pierce the curtain. If you’ve seen the 1939 MGM film, ‘Wizard of Oz’ you know my meaning: there’s a moment when the supposed wizard, granter of dreams, is revealed to be a sweaty, nervous man, hidden by a curtain, frantically pulling levers and spinning dials to keep the machinery of delusion functioning. This was my guiding metaphor for the tech industry, which claims its products defy the limits of material reality and surpass human thought.

As you learn more, your understanding should change. Parting the curtain, or, debunking was an acceptable way to start but it’s insufficient; the promotion of so-called ‘AI’ is producing real-world harms. From automated recidivism decision systems to facial recognition based arrests and innumerable other intrusions. A technology sold as bringing about a bright future is being deployed to limit possibilities. Digital computation began as a means of enacting a command and control methodology on the world for various purposes (military applications being among the first) and is, in our age, reaching its apotheosis.

Kinetic Harm

Reporting on these harms, as deadly as they often are, fails to tell the entire story of computation in this era of growing instability. The same technologies and methods used to, for example, automate actuarial decision making in the insurance industry can also be used for other, more directly violent aims. The US military, which is known for applying euphemisms to terrible things like a thin coat of paint over rust, calls warfare – that is, killing – kinetic military action. We can call forms of applied computation deliberately intended to produce death and destruction kinetic harm.

Consider the IDF’s Habsora system, described in the +972 Magazine article, ‘A mass assassination factory’: Inside Israel’s calculated bombing of Gaza’ –

In one case discussed by the sources, the Israeli military command knowingly approved the killing of hundreds of Palestinian civilians in an attempt to assassinate a single top Hamas military commander. “The numbers increased from dozens of civilian deaths [permitted] as collateral damage as part of an attack on a senior official in previous operations, to hundreds of civilian deaths as collateral damage,” said one source.

“Nothing happens by accident,” said another source. “When a 3-year-old girl is killed in a home in Gaza, it’s because someone in the army decided it wasn’t a big deal for her to be killed — that it was a price worth paying in order to hit [another] target. We are not Hamas. These are not random rockets. Everything is intentional. We know exactly how much collateral damage there is in every home.”

According to the investigation, another reason for the large number of targets, and the extensive harm to civilian life in Gaza, is the widespread use of a system called “Habsora” (“The Gospel”), which is largely built on artificial intelligence and can “generate” targets almost automatically at a rate that far exceeds what was previously possible. This AI system, as described by a former intelligence officer, essentially facilitates a “mass assassination factory.”

+972 Magazine – https://www.972mag.com/mass-assassination-factory-israel-calculated-bombing-gaza/

The popular phrase, artificial intelligence, a marketing term, really, since no such thing exists, is used to describe the Habsora system. This creates an exotic distance, as if a glowing black cube floats in space deciding who dies and how many deaths will occur.

The reality is more mundane, more familiar, even banal; the components of this machine are constantly in use around us. Here is a graphic that shows some of the likely elements:

As we use our phones, register our locations, fill in online forms for business and government services, interact on social media and so many other things, we unknowingly create threads and weave patterns, stored in databases. The same type of system that enables a credit card fraud detection algorithm to block your card if in-person store transactions are registered in two, geographically distant locations on the same day can be used to build a map of your activities and relations to find and kill you and those you know and love. This is what the IDF has done with Habsora. The distance separating the intrusive methods of Meta, Google and fellow travelers from this killing machine is not as great as it seems.

Before being driven from their homes by the IDF – homes that were destroyed under the most intensive bombing campaign of this and perhaps even the previous, hyper-violent century, Palestinians in Gaza were subject to a program of surveillance and control which put them completely at the mercy of the Israeli government. All data about their movements and activities passed through electronic infrastructure owned and controlled by Israeli entities. This infrastructure, and the data processing and analysis built upon it, have been assembled into a factory whose product is death – whether targeted or en masse.

The Thin Curtain

Surveillance. Control. Punishment. This is what the age of digital computation has brought on an unprecedented scale. For those of us who live in places where the bombs don’t yet fall, there are things like the following, excerpted from the Forbes article (Feb 23, 2024) ‘Dozens Of KFC, Taco Bell And Dairy Queen Franchises Are Using AI To Track Workers’ –

Like many restaurant owners, Andrew Valkanoff hands out bonuses to employees who’ve done a good job. But at five of his Dairy Queen franchises across North Carolina, those bonuses are determined by AI.

The AI system, called Riley, collects streams of video and audio data to assess workers’ performance, and then assigns bonuses to those who are able to sell more. Valkanoff installed the system, which is developed by Rochester-based surveillance company Hoptix, less than a year ago with the hopes that it would help increase sales at a time when margins were shrinking and food and labor costs were skyrocketing.

Forbes – https://www.forbes.com/sites/rashishrivastava/2024/02/23/dozens-of-kfc-taco-bell-and-dairy-queen-franchises-are-using-ai-to-track-workers/

Inside the zone of comparative safety but, deprivation for many and control imposed on all, there are systems like the IDF’s Habsora in service, employing the same computational techniques, which, instead of directing sniper rifle armed quadcopters and F-16s on deadly errands, deprive people of jobs, medical care and freedom.  Just as a rocket’s payload can be changed from peaceful to fatal ends, the intended outcomes of such systems can be altered to fit the goals of the states that employ them.

The Shadow

As I write this, approximately 1.4 million Palestinians have been violently pushed to Rafah, a city in the southern Gaza strip. There, they are facing starvation and incomprehensible cruelty. Meanwhile, southwest of the ruins of Gaza City, in what has come to be known as the Al Nabulsi massacre, over one hundred Palestians were killed by IDF fire while desperately trying to get flour.  These horrors were accelerated by the use of computationally driven killing systems. In the wake of Habsora’s use in what journalist Antony Loewenstein calls the Palestine Laboratory, we should expect similar techniques to be used elsewhere and to become a standard part of the arsenal of states (yes, even those we call democratic) in their efforts to impose their will on an ever more restless world that struggles for freedom.

References

Artificial intelligence and insurance, part 1: AI’s impact on the insurance value chain

https://www.milliman.com/en/insight/critical-point-50-artificial-intelligence-insurance-value-chain

Kinetic Military Action

https://en.wikipedia.org/wiki/Kinetic_military_action

A mass assassination factory’: Inside Israel’s calculated bombing of Gaza

https://www.972mag.com/mass-assassination-factory-israel-calculated-bombing-gaza

Report: Israel’s Gaza Bombing Campaign is the Most Destructive of this Century

https://english.aawsat.com/features/4760791-report-israels-gaza-bombing-campaign-most-destructive-century

‘Massacre’: Dozens killed by Israeli fire in Gaza while collecting food aid

https://www.aljazeera.com/news/2024/2/29/dozens-killed-injured-by-israeli-fire-in-gaza-while-collecting-food-aid

Dozens Of KFC, Taco Bell And Dairy Queen Franchises Are Using AI To Track Workers

https://www.forbes.com/sites/rashishrivastava/2024/02/23/dozens-of-kfc-taco-bell-and-dairy-queen-franchises-are-using-ai-to-track-workers

The Palestine Laboratory: How Israel Exports the Technology of Occupation Around the World

The Palestine Laboratory

Examples of Other Algorithm Directed Targeting Systems

Project Maven

https://www.engadget.com/the-pentagon-used-project-maven-developed-ai-to-identify-air-strike-targets-103940709.html

Generative AI for Defence (marketing material from C3)

https://c3.ai/generative-ai-for-defense

Command, Control, Kill

Posted on by
Reply

The IDF assault on Nasser hospital in Southern Gaza joined a long and growing list of bloody infamies committed by Israel since Oct 7, 2023. During a Democracy Now interview, broadcast on Feb 15, 2024, Dr. Khaled Al Serr, who was later kidnapped by the IDF, described what he saw:

“Actually, the situation here in the hospital at this moment is in chaos. All of the patients, all the relatives, refugees and also the medical staff are afraid because of what happened. We could not imagine that at any time the Israeli army will bomb the hospital directly, and they will kill patients and medical personnel directly by bombing the hospital building. Yesterday also, Israeli snipers and Israeli quadcopters, which is a drone, carry on it an AR, and with a sniper, they shot all over the building. And they shot my colleague, Dr. Karam. He has a shrapnel inside his head. I can upload for you a CT for him. You can see, alhamdulillah, it was superficial, nothing serious. But a lot of bullets inside their bedroom and the restroom.”

The Israeli military is using quadcopters, armed with sniper rifles, as part of its assassination arsenal. These remote operated drones, which possess limited but still important automatic capabilities (flight stability, targeting persistence) are being used in the genocidal war in Gaza and the war between Russia and Ukraine to name two, prominent examples. They are likely to make an appearance near you in some form, soon enough.

I haven’t seen reporting on the type of quadcopter used but it’s probably the Smash Dragon, a model produced by the Israeli firm Smart Shooter which, on its website, describes its mission:

SMARTSHOOTER develops state-of-the-art Fire Control Systems for small arms that significantly increase weapon accuracy and lethality when engaging static and moving targets, on the ground and in the air, day and night.

Here is a promotional video for the Smash Dragon:

Smart Shooter’s product, and profit source are the application of computation to the tasks of increasing accuracy and automating weapon firing. One of their ‘solutions’ (solving, apparently, the ‘problem’ of people being alive) is a fixed position ‘weapon station’ called the Smash Hopper that enables a distant operator to target-lock the weapon on a person, initiating the firing of a constant stream of bullets. For some reason, the cartoonish word,  ‘smash’ is popular with the Smart Shooter marketing team.

‘AI’, as used under the current global order, serves three primary purposes: control via sorting, anti-labor propaganda and obscuring culpability. Whenever a hospital deploys an algorithmic system, rather than healthcare worker judgment, to decide how long patients stay, sorting is being used as a means of control, for profit. Whenever a tech CEO tells you that ‘AI’ can replace artists, drivers, filmmakers, etc. the idea of artificial intelligence is employed as an anti-labor propaganda tool. And whenever someone tells you that the ‘AI’ has decided, well, anything, they are trying to hide the responsibility of the people behind the scenes, pushing algorithmic systems on the world.

The armed quadcopter brings all of these purposes together, wrapped in a blood stained ribbon. Who lives and who dies is decided via remote control while the fingers pulling the trigger, and the people directing them are hidden from view. These systems are marketed as using ‘AI’ implying machines are making life and death decisions rather than people.

In the introduction to his 2023  book, The Palestine Laboratory, which details Israel’s role in the global arms trade and use of the Palestinians as lethal examples, journalist Anthony Lowenstein describes a weapons demonstration video attended by Andrew Feinstein in 2009:

“Israel is admired as a nation that stands on its own and is unashamed in using extreme force to maintain it. [Andrew Feinstein is] a former South African politician. journalist, and author. He told me about attending the Paris Air Show in 2009, the world’s largest aerospace industry and air show exhibitions. [The Israel-based defense firm Elbit Systems] was showing a promotional video about killer drones, which have been used in Israel’s war against Gaza and over the West Bank.

The footage had been filmed a few months before and showed the reconnaissance of Palestinians in the occupied territories. A target was assassinated. […] Months later, Feinstein investigated the drone strike and discovered that the incident featured in the video had killed a number of innocent Palestinians, including children.  This salient fact wasn’t featured at the Paris Air Show. “This was my introduction to the Israeli arms industry and the way it markets itself.”

The armed quadcopter drone, one of the fruits of an industry built on occupation and death, can be added to the long list of the harms of computation. ‘Keep watching the skies!’ someone said at the end of a 1950s science fiction film whose name escapes me. Never mind though, the advice stands.

References

Democracy Now Interview with Dr. Khaled Al Serr

https://www.democracynow.org/2024/2/15/nasser_hospital_stormed_gaza

Dr. Al Serr kidnapped

The Palestine Laboratory

The Palestine Laboratory

AI Ethics, a More Ruthless Consideration

Posted on by

According to self-satisfied legend, medieval European scholars, perhaps short of things to do compared to we ever-occupied moderns, spent countless hours wondering about topics such as, how many angels could simultaneously occupy the head of a pin; the idea being that, if nothing was impossible for God, surely, violating the observable rules of space and temporality should be cosmic child’s play for the deity…but to what extent?

How many angels, oh lord?

Although it’s debatable whether this question actually kept any monks up at night more than, say, wondering where the best beer was, the core idea, that it’s possible to get lost in a maze of interesting, but ultimately pointless inquiries (a category which, in an ancient Buddhist text is labeled, ‘questions that tend not towards edification’) remains eternally relevant.

At this stage in our history, as we stare, dumbfounded, into the barrels of several weapons of capitalism’s making – climate change being the most devastating – the AI endeavor is the computational equivalent of that apocryphal medieval debating topic; we are discussing the ethics of large language models, focusing, understandably, on biased language and power consumption but missing a more pointed ethical question: should these systems exist at all? A more, shall we say, robust ethics would demand that in the face of our complex of global emergencies, tolerance for the use of computational power for games with language cannot be justified.

OPT-175B – A Lesson: Hardware

The company now known as Meta recently announced its creation of a large language model system called OPT-175B. Helpfully, and unlike the not particularly open OpenAI, the announcement was accompanied by the publication of a detailed technical review, which you can read here.

As the paper’s authors promise in the abstract, the document is quite rich in details which, to those unfamiliar with the industry’s terminology and jargon, will likely be off-putting. That’s okay because I read it for you and can distill the results to four main items:

  1. The system consumes almost a thousand NVIDIA game processing units (992 to be exact, not counting the units that had to be replaced because of failure)
  2. These processing units are quite powerful, which enabled the OPT-175B team to use relatively fewer computational resources than what was installed for GPT-3 another, famous (at least in AI circles) language model system
  3. OPT-175B, which drew its text data from online sources, such as that hive of villainy, Reddit, has a tendency to output racist and misogynist insults
  4. Sure, it uses fewer processors but its carbon footprint is still excessive (again, not counting replacements and supply chain)

Here’s an excerpt from the paper:

“From this implementation, and from using the latest generation of NVIDIA hardware, we are able to develop OPT-175B using only 1/7th the carbon footprint of GPT-3. 

While this is a significant achievement, the energy cost of creating such a model is still nontrivial, and repeated efforts to replicate a model of this size will only amplify the growing compute footprint of these LLMs.” [highlighting emphasis mine]

https://arxiv.org/pdf/2205.01068.pdf

I cooked up a visual to place this in a fuller context:

Here’s a bit more from the paper about hardware:

“We faced a significant number of hardware failures in our compute cluster while training OPT-175B. 

In total, hardware failures contributed to at least 35 manual restarts and the cycling of over 100 hosts over the course of 2 months. 

During manual restarts, the training run was paused, and a series of diagnostics tests were conducted to detect problematic nodes.

Flagged nodes were then cordoned off and training was resumed from the last saved checkpoint. 
Given the difference between the number of hosts cycled out and the number of manual restarts, we estimate 70+ automatic restarts due to hardware failures.”

https://arxiv.org/pdf/2205.01068.pdf

All of which means that, while processing data, there were times, quite a few times, when parts of the system failed, requiring a pause till fixed or routed around (resumed, once the failing elements were replaced).

Let’s pause here to reflect on where we are in the story; a system, whose purpose is to produce plausible strings of text (and, stripped of the obscurants of mathematics, large-scale systems engineering and marketing hype, this is what large language models do) was assembled using a small mountain of computer processors, prone, to a non-trivial extent, to failure.

As pin carrying capacity counting goes, this is rather expensive.

OPT-175B – A Lesson: Bias

Like other LLMs, OPT-175B has a tendency to return hate speech as output. Another excerpt:

“Overall, we see that OPT-175B has a higher toxicity rate than either PaLM or Davinci. We also observe that all 3 models have increased likelihood of generating toxic continuations as the toxicity of the prompt increases, which is consistent with the observations of Chowdhery et al. (2022). As with our experiments in hate speech detection, we suspect the inclusion of unmoderated social media texts in the pre-training corpus raises model familiarity with, and therefore propensity to generate and detect, toxic text.” [bold emphasis mine]

https://arxiv.org/pdf/2205.01068.pdf

Unsurprisingly, there’s been a lot of commentary on Twitter (and no doubt, elsewhere) about this toxicity. Indeed, almost the entire focus of ‘ethical’ efforts has been on somehow engineering this tendency away – or perhaps avoiding it altogether via the use of less volatile datasets (and good luck with that as long as Internet data is in the mix!)

This defines ethics as being the task of improving a system’s outputs – a technical activity – and not a consideration of a system as a whole from an ethical standpoint within political economy. Or to put it another way, the ethical task is narrowed to making sure that if I use a service which, on its backend, depends on a language model for its apparent text capability, it won’t in the midst of telling me about good nearby restaurants, hurl insults like a klan member.

OPT-175B – A Lesson: Carbon

Within the paper itself, there is the foundation of an argument against this entire field, as currently pursued:

“...there exists significant compute and carbon cost to reproduce models of this size. While OPT-175B was developed with an estimated carbon emissions footprint (CO2eq) of 75 tons,10 GPT-3 was estimated to use 500 tons, while Gopher required 380 tons. These estimates are not universally reported, and the accounting methodologies for these calculations are also not standardized. In addition, model training is only one component of the over- all carbon footprint of AI systems; we must also consider experimentation and eventual downstream inference cost, all of which contribute to the growing energy footprint of creating large-scale models.”

A More Urgent Form of Ethics

In the fictional history of the far-future world depicted in the novel ‘Dune’ there was an event, the Butlerian Jihad, which decisively swept thinking machines from galactic civilization. This purge was inspired by the interpretation of devices that mimicked thought or possessed the capacity to think as an abomination against nature.

Today, we do not face the challenge of thinking machines and probably never will. What we do face however, is an urgent need to, at long last, take climate change seriously. How should this reorientation towards soberness alter our understanding of the role of computation?

I think that, in face of an ever-shortening amount of time to address climate change in an organized fashion, the continuation, to say nothing of expansion of this industrial level consumption of resources, computing power, talent and the corresponding carbon footprint is ethically and morally unacceptable.

At this late hour, the ethical position isn’t to call for, or work towards better use of these massive systems; it’s to demand they be halted and the computational capacity re-purposed for more pressing issues.  We can no longer afford to wonder how many angels we can get to dance on pins.

Pointillistic But Useful: A Machine Learning Object Lesson

Posted on by

I devote a lot of time to understanding, critiquing and criticizing the AI Industrial Complex. Although much – perhaps most- of this sector’s output is absurd, or dangerous (AI reading emotions and automated benefits fraud determination being two such examples) there are examples of uses that are neither which we can learn from.

This post briefly reviews one such case.

During dinner with friends a few weeks ago, the topic of AI came up. No, it wasn’t shoehorned into an otherwise tech-free situation; one of the guests works with large-scale engineering systems and had some intriguing things to say about solid, real world, non-harmful uses for algorithmic ‘learning’ methods.

Specifically, he mentioned Siemens’ use of machine vision to automate the inspection of wind turbine blades via a platform called Hermes. This was a project he was significantly involved in and justifiably proud of. It provides an object lesson for the types of applications which can benefit people, rather than making life more difficult through algorithm.

You can view a (fluffy, but still informative) video about the system below:

Hermes System Promotional Video

A Productive Use of Machine Learning

The solution Siemens employed has several features which make it an ideal object lesson:

1.) It applies a ‘learning’ algorithm to a bounded problem

Siemens engineers know what a safely operating blade looks like; this provides a baseline against which variances can be found.

2.) It applies algorithms to a bounded problem area that generates a stream of dynamic, inbound data

The type of problem is within the narrow limits of what an algorithmic system can reasonably and safely handle and benefits from a robust stream of training data that can improve performance

3.) It’s modest in its goal but nonetheless important

Blade inspection is a critical task and very time consuming and tedious. Utilizing automation to increase accuracy and offload repeatable tasks is a perfect scenario.

How Is This Different from AI Hype?

AI hype is used to convince customers – and society as a whole – that algorithmic systems match, or exceed the capabilities of humans and other animals. Attempts to proctor students via machine vision to flag cheating, predict emotions or fully automate driving are examples of overreach (and the use of ‘AI’ as a behavioral control tool). I use ‘overreach‘ because current systems are, to quote Gary Marcus in his paper ‘The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence‘, “pointillistic” – often quite good in narrow or ‘bounded’ situations (such as playing chess) but brittle and untrustworthy when applied to completely unbounded, real world circumstances such as driving, which is a series of ‘edge cases’.

Visualization of Marcus’ Critique of Current AI Systems

The Siemens example provides us with some of the building blocks of a solid doctrine to use when evaluating ‘AI’ systems (and claims about those systems) and a lesson that can be transferred to non-corporate uses.

Attack Mannequins: AI as Propaganda

Posted on by

What follows is a sketch, the foundation of a propaganda model, focused on what I’ll call the ‘AI Industrial Complex‘. By the term AI Industrial Complex, (AIIC) I mean the combination of technological capacity (or the lack thereof) with marketing promotion, media hype and capitalist activity that seeks to diminish the value of human labor and talent. I use this definition to make a distinction between the work of researchers and practical technologists and the efforts of the ownership class to promote an idea: that machine cognition is now, or soon will be, superior to human capabilities. The relentless promotion of this idea should be considered a propaganda campaign.

If There’s No AI, What is Being Promoted?

It’s my position there is no existing technology that can be called ‘artificial intelligence’ (how can we engineer a thing we haven’t yet decisively defined?) and that, at the most sophisticated levels of government and industry, the actually existing limitations of what is essentially pattern matching, empowered by (for now) abundant storage and computational power, are very well understood. The existence of university departments and corporate divisions dedicated to ‘AI’ does not mean AI exists; it’s evidence there’s powerful memetic value attached to using the term, which has been aspirational since it was coined by computer scientist John McCarthy in 1956. Once we filter for hype inspired by Silicon Valley hustling (the endless quest to attract investment capital and gullible customers) we are left with promotion intended to shape common perception about what’s possible with computer power. 

As an example, consider the case of computer scientist Geoffrey Hinton’s 2016 declaration that “we should stop training radiologists now” Since then, extensive research has shown this to have been premature, to say the least (see “Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy“).

It’s tempting to see this as a temporarily embarrassing bit of overreach by an enthusiastic field luminary – yet another example of familiar hype but let’s go deeper and ask questions about the political economy underpinning this messaging excess.

Hinton on Radiology in 2016

Radiologists are expensive and, in the US, very much in demand (indeed, there’s a shortage of qualified people). Labor shortages typically lead to higher wages and better working conditions and form the material conditions that create what some call labor aristocracies. In the past, such shortages were addressed via pushes for training and incentives to workers (such as the lavish perks that were common in the earlier decades of the tech era).

If this situation could be bypassed via the use of automation, that would devalue the skilled labor performed by radiologists, solving the shortage problem while increasing the power of owners over the remaining staff.

The promotion of the idea of automated radiology – regardless of actually existing capabilities – is attractive to the ownership class because it holds the promise of weakening labor’s power and increasing – via workforce cost reduction and greater scalability – profitability. I say promotion, because there is a large gap between what algorithmic systems are marketed as being capable of, and reality. This gap, which, as I stated earlier is well understood by the most sophisticated individuals in government and industry, is unimportant to the larger goal of convincing the general population their work efforts can be replaced by machines. The most important outcome isn’t thinking machines (which seems to be a remote goal if possible at all) but a demoralized population, subjected to a maze of crude automated systems which are described as being better than the people forced to navigate life through these systems.

A Factor Among Factors

Technological systems – and the concepts attached to them – emerge from, and reflect the properties of the societies that create those systems. Using the Hegelian (and later, Marxist) philosophy of internal relations, we can analyze both real algorithmic systems and the concept of ‘AI’ as being a part of the interplay of factors that comprise global capitalist dynamics – both actor and acted upon. From this point of view, the propaganda effort promoting ‘AI’ should not be considered in isolation, but as one aspect of a complex.

Hype vs. Propaganda

What defines hype and what differentiates standard industry hype from a propaganda campaign?

Hype (such as marketing material that makes excessive claims – for example, AI reading emotions) is narrowly designed to attract investment capital and customers. Hype should be considered a species of advertisement. Propaganda has a broader aim, which is described by Jacques Ellul in his work, Propaganda.

Describing one of the four elements of propaganda, and bridging from advertising to propaganda, Ellul writes…

“Public and human relations: These must necessarily be included in propaganda. This statement may shock some readers, but we shall show that these activities are propaganda because they seek to adapt the individual to a society, to a living standard, to an activity. They serve to make him conform, which is the aim of all propaganda. In propaganda we find techniques of psychological influence combined with techniques of organization and the envelopment of people with the intention of sparking action.”

A Propaganda Model: Foundational Concepts

As the model of AI as propaganda is constructed, the works of three thinkers will provide key guidance:

Jacques Ellul: Propaganda

As already noted, Ellul’s key work on propaganda (which, I think, was the first to apply sociology and psychology to the topic) is a critical source of inspiration:

“Propaganda is first and foremost concerned with influencing an individual psychologically by creating convictions and compliance through imperceptible techniques that are effective only by continuous repetition. Propaganda employs encirclement on the individual by trying to surround man by all possible routes, in the realm of feelings as well as ideas, by playing on his will or his needs through his conscious and his unconscious, and by assailing him in both his private and his public life.

The propagandist also acknowledges the most favorable moment to influence man is when an individual is caught up in the masses. Propaganda must be total in that utilizes all forms of media to draw the individual into the net of propaganda. Propaganda is designed to be continuous within the individual’s life by filling the citizen’s entire day. It is based on slow constant impregnation that functions over a long period of time exceeding the individual’s capacities for attention or adaptation and thus his capabilities of resistance”

Full at Wikipedia’s article 

The relentless promotion of the idea that automation is on the verge of replacing human labor can be interpreted as being part of an effort to create a conviction (there is artificial intelligence’, it cannot be stopped) and compliance (resistance to ‘AI’ is retrogressive Luddism).

Noam Chomsky/Edward S. Herman: The Propaganda Model

In their book, ‘Manufacturing Consent’ Chomsky and Herman present a model of propaganda via media:

“The third of Herman and Chomsky’s five filters relates to the sourcing of mass media news: 

The mass media are drawn into a symbiotic relationship with powerful sources of information by economic necessity and reciprocity of interest. Even large media corporations such as the BBC cannot afford to place reporters everywhere. They concentrate their resources where news stories are likely to happen: the White House, the Pentagon, 10 Downing Street and other central news “terminals”. Although British newspapers may occasionally complain about the “spin-doctoring” of New Labour, for example, they are dependent upon the pronouncements of “the Prime Minister’s personal spokesperson” for government news. Business corporations and trade organizations are also trusted sources of stories considered newsworthy. Editors and journalists who offend these powerful news sources, perhaps by questioning the veracity or bias of the furnished material, can be threatened with the denial of access to their media life-blood – fresh news. Thus, the media has become reluctant to run articles that will harm corporate interests that provide them with the resources that they depend upon. 

The dependence of news organizations on press releases from Google and other tech giants that promote the idea of ‘AI’ can be interpreted as being an example of the ‘symbiotic relationship, based on reciprocity of interest’ Chomsky and Herman detail.

Full at Wikipedia’s article

Summary

The concept of “artificial intelligence” is aspirational (like ‘warp drive’) and does not describe any existing or likely to exist computational system. Despite this, the concept is promoted to attract investment capital and customers but also, more critically for my purposes, devalue the power of labor – if not in fact than in perception (which, in turn, becomes fact). For this reason, I assert that ‘AI’, as a concept, is part of a propaganda campaign.

Key Characteristics of AI Propaganda

The promotion of the concept of AI, as a propaganda effort, has several elements:

* Techno-optimism: The creation of thinking machines is promoted as being possible, with little or no acknowledgement of limitations.

* Techno-determinism: The creation of thinking machines is promoted as being inevitable and beyond human intervention, like a force of nature

* An Elite Project: Although individual boosters, grifters, techno enthusiasts and practitioners may contribute within their circles (for ex. social media) to hype, the propaganda campaign is an elite project designed to effect political economy and the balance of power between labor and capital.

* Built on, but not limited to, hype: There is a relationship between hype and propaganda. Hype is of utility to the propaganda campaign but the objective of that campaign is broader and targeted towards changing societal attitudes and norms.

I use the term attack mannequins to describe this complex – lifeless things, presented as being lifelike, used to assault the position and power of ordinary people.

UPDATE: 2 NOVEMBER 2021

In this video, YouTube Essayist Tom Nicholas details the efforts Waymo has made to convince people – via the use of YouTube ‘educators’ – that autonomous vehicles are a perfected technology, superior to human drivers and a solution to traffic safety and congestion issues.

Nicholas makes the point that inasmuch as the Waymo ‘autonomous’ taxi service (supported by a large staff of people behind the scenes) only operates in a subsection of the suburbs of Phoenix, Arizona USA, the PR campaign’s goal can’t be explained as advertising; it’s part of a broad effort to change minds.

In other words, propaganda.