This blog is dedicated to analyzing the field of ‘artificial intelligence’ specifically, and the tech industry generally from a materialist perspective.
By materialist perspective, I mean a point of view informed, in large measure by historical materialism but also, by a deep analysis of the nuts and bolts – the supply chains, extractive industrial activity, data centers and actually existing capabilities undergirding what the tech industry presents as magic.
There is however, another layer operating which is captured by, among other disciplines, psychoanalytic theory.
In this series of video meditations, I consider aspects of the book. This is an ongoing project and I’ll be updating this post as new videos are added.
Last night, before nodding off to sleep, a stray memory flitted across (or through?) my synapses; a posting I made to the Left Business Observer Listserv, titled “Windows Vista as Neoliberal Instrument”. This was, I think, my first attempt to merge my work in information technology with my (always in formation and never complete) Marxian approach to ways of thinking about that industry.
At the time of writing, over a decade ago, February of 2007, the release of Microsoft’s Windows Vista operating system was the source of a lot of debate and frustration. The OS wasn’t performing as hoped and techies were wondering why. It turned out that one of the key reasons was Microsoft’s attempt to enforce copyright via software. This proved to be a rich target for analysis and David Harvey’s ‘A Brief History of Neoliberalism’ provided a powerful analytical framework. Also, this was a total flex.
Introduction
[content originally posted to LBOTalk February, 2007 some new formatting added]
In his most recent book, “A Brief History of Neoliberalism”, David Harvey analyzes the neoliberal turn that first Western, and later, practically every economy on Earth took to varying degrees of depth over the past 30 or so years.
Several key features of neoliberalism are dissected:
1.) neoliberalism as a power restoration technique (i.e., restoring to capitalists the margin of power lost during the postwar years of high growth and detente with labor)
2.) neoliberalism as imperfect tool against stagnation and the problems of overproduction
and
3.) neoliberalism as a method for monetizing practices and spaces previously excluded from market concerns and controls
To properly understand the strategic concessions Microsoft made to the entertainment industry — concessions that led MSFT to deploy a software-based version of the Advanced Access Content System (AACS) in Windows Vista — you need to carefully consider that third aspect of neoliberalism.
What is AACS?
Briefly, the Advanced Access Content System is a platform, created at the behest of the entertainment industry, whose sole purpose is to enforce a (it is vainly hoped) completely uncrackable environment for “premium content” to flow through from player — device or software-based — to a display and/or audio output. Of course, the phrase “premium content” is a term of art inasmuch as the actual content might be anything from a slapdash teen sex comedy to the most subtle examples of musical or filmed art.
The motion picture and recording cartels have long been disturbed by the fact that people could record, remix and redistribute “content” at will. Over the years, many copy protection schemes have been tried; all have failed. Advances in computing power and storage capacity — moving in parallel with advances in cryptology — have finally made the old dream of an automated copyright enforcement system achievable.
Achievable, because under the AACS system, ‘intelligent’ hardware is constantly on the lookout for security breaches (for example, interceptions of the content data stream from player to output) and empowered, so to speak, to take action. What action? Well, action like actively preventing component outs from working if the HD-DVD or Blu ray disk you’re trying to view has been flagged as being compromised (or more specifically, if the cryptological “key” associated with the disk has been compromised, leading to your play privileges being ‘revoked’ by the key issuing authority).
All high definition hardware — players, digital sets, audio units — are designed to enforce this automated copyright infrastructure. Your HD-DVD or Blu Ray player will talk to your high def display over what are called High-Bandwidth Digital Content Protection compliant outputs. Together, they’ll ensure that RIAA and MPAA copyright concerns are being addressed wherever and whenever “premium content” is being viewed.
Rent Seeking via Operating System
Microsoft wanted Vista to be marketable as a media platform (and MSFT also wanted to create the de facto standard for software based AACS implementation) so they crafted a complex encryption/decryption methodology within the operating system that obeys — and then some — AACS rules. Doing so gave them negotiating space with the entertainment industry.
As any user of consumer electronics and Microsoft software knows, shit happens. The copyright enforcement, content monitoring and encryption/decryption technologies in next gen players and Vista are always on. This exacts a performance price from the devices (because our CPUs and memory are good, but not so good that they can effortlessly do both content presentation and advanced cryptological functions without exhibiting some problems at least some of the time) and especially from the software, which is very brittle and prone to malfunction.
But beyond the false piracy alarms, stuttering playbacks and other technical annoyances that are already being seen in the wild, there’s an overriding fact to keep in mind: AACS gives the entertainment industry the ability to treat the products you buy as leased objects, which can be (say, in a case of revocation resolution) the source for ever renewable revenue long after they were originally purchased.
It also creates a method for modularizing in unprecedented ways — and therefore monetizing — functions that were previously considered more or less all of a piece, such as playing and therefore viewing the disks you buy.
In order for this system to work as planned, all devices must comply with the AACS standard. The idea is to close all potential areas of escape. Eventually, perhaps after 5 to 15 years, the full magnitude of the lock-in will be in effect as older DVD and audio players are retired.
It’s been rumored that Hollywood and the RIAA are fully aware AACS is, despite all their efforts, eminently hackable, and that the true target of these new constraints are ordinary people who don’t have easy access to workarounds.
The goal then, is to have a lever that can be pulled at any time to extract more income from “consumers”.
Techno-skepticism is a vital and necessary response to a world awash in self-promoting boosterism and the capitalist utilized ideologies of techno-optimism and techno-determinism.
To define terms, techno-optimism is the belief any proposed technology is possible and good. Optimists look to past examples of things that were once impossible which became possible – such as machine flight – and infer this tendency is universal.
Techno-determinism (which can be considered a species of determinism) builds on tech-optimism’s ideological framework by asserting not just possibility, but inevitability.
For example, a techno-optimist views a development such as ‘robot’ kitchens as being both positive and possibleas presented – determinists assert there’s nothing to stop such a development: it’s inevitable and beyond resistance, like gravity.
Robotic Chef Marketing Video
Skepticism, correctly practiced, isn’t the denial of technological change or the reality of, or potential for, benefits from such change. Skepticism is remembering to ask three questions:
Is it possible as described? A feasibility interrogation
Consider, for example, Amazon’s failed drone delivery service, which Cory Doctorow analyzed here – As Doctorow describes, this idea was inexplicably taken seriously:
“When Amazon announced “Prime Air,” a forthcoming drone delivery service, in 2016, there was a curious willingness on the part of the press – even the tech press – to take the promise of a sky full of delivery drones at face value.
This despite the obvious problems with such a scheme: the consequences of midair collisions, short battery life, overhead congestion, regulatory hurdles and more. Also despite the fact that delivery drones, like jetpacks, are really only practical as sfx in an sf movie.”
At the time this proposed service was announced, I read detailed analyses and excited Tweet threads about the supposed meaning of a bold new age of drone delivery. I noticed however, that simple questions regarding feasibility were rarely asked – optimism and determinism (with a good amount of self-interested boosterism in the mix) prevented a skeptical response
When you read about a technological system, such as delivery via drone, remembering to ask questions about function (the how), benefit (who’s promoting this and why) and feasibility (can this be done at all or as the promoters describe?) is a reliable way to avoid being fooled and knocked from delusion to delusion.
This is unsurprising; both because the idea seems futuristic (being based on a science fiction confection introduced in Neal Stephenson’s dystopian 1992 novel ‘Snow Crash‘) and also, because the tech media space reports anything announced by a so-called FAANG company as if it’s marvelous and inevitable.
Let’s apply a bit of real-ness to this and use a materialist analysis to interrogate the idea of the ‘Metaverse’ (this is similar in theme to my inquiry into Boston Dynamics).
Light Detective Work and Logical Inference
Tech companies create an air of secrecy around projects such as FB’s Metaverse effort for competitive reasons but also, I’d argue, to obscure what is often merely the assembly of already existing elements into platforms. Mariana Mazzucato analyzes this tendency using the iPhone in her book, ‘The Entrepreneurial State‘.
Here’s how the iPhone’s elements are dissected in Mazzucato’s book:
A similar method can be applied to an analysis of FB’s Metaverse.
The Oculus platform and Facebook’s Ray Ban stories glasses provide sufficient information for some light detective work. No matter how secretive a company tries to be, its job postings, properly interpreted and supported by experience, provide a rich source of evidence for what an organization is doing.
Working on the assumption that the Metaverse will primarily consist of repurposed elements (and the fact everything depends on, and leads to data centers), I examined Oculus jobpostings and dissected their contents.
The main technical themes were:
Optics
Haptics
Tracking
Display
Computer vision
User experience
Audio
Perceptual psychology
Research Science
Mechanical Engineering
Electrical Engineering
Software Engineering
Networking
Server operations
Of course, it’s impossible to know the precise details of FB’s system topology without a reference architecture but experience leads me to think we can achieve a solid approximation (and data center dependency is an absolute certainty no matter what else may be going on).
What can we infer from this?
How Sustainable and Realizable Is the Metaverse Concept?
Although the tech press treats every industry pronouncement as an irrefutable prediction there’s precedent of lots of smoke but little to no fire (recall Amazon’s supposedly brilliant drone delivery service). According to some estimates, Facebook has over 2 billion active users. An effort to move all, or even a statistically significant portion of this user base to a platform that generates a virtual reality environment for, and ingests audio/visual data from, hundreds of millions of people means a massive investment in physical infrastructure – computers, network infrastructure, cooling systems and real estate to host this and other relevant equipment (to get a sense of the industrial and extractive elements of what’s called ‘the cloud’ I suggest Nathan Ensmenger’s essay ‘The Cloud is a Factory’).
It also means an increase in demands for data transfer over Internet. It’s easy to project system crashes, bad connections and other problems caused by scalability challenges. It’s fair to ask if, despite the hype, any of this is actually possible as described and if so, how reliable will it be?
Conclusion
There’s abundant evidence Facebook (or whatever it’ll call itself in a week) is a problem. The company’s role in a variety of destructive activities is well documented. For that reason alone, the ‘Metaverse’ push is immediately suspect. I think we can also conclude however, that it might not be achievable as advertised and may turn out to be, like so much else that emerges from Silicon Valley, an elaborate grift, dressed up as a bold vision of the future.
We should recall that in the novel that gave the project its name, the ‘Metaverse’ is the last refuge for people living in a collapsed world. In this case, we might get the collapse without even the warped comforts a virtual world is supposed to offer.
UPDATE (29 Oct)
On 28 October, Facebook announced it was rebranding as ‘Meta’ to reflect its focus on being a ‘metaverse’ company.
The keynote video presented a vision (such as it is) for what the ‘metaverse’ is supposed to be…eventually. Zuckerberg walks within a fully virtual environment, uses a virtual pop-up menu and zooms (virtually) into an environment creatively named “Space Room”.
Rebranding the company formerly known as Facebook as Meta is, in part, surely intended to breathe new life into a moribund platform and distract attention away from the many negative associations Facebook has earned. Even so, we can predict that within the company, there will be efforts to make as much of this notion real as possible – despite the fact promoted elements (such as an environment you can walk through as if it’s real) are thoroughly impossible and likely to remain so for quite some time – indeed, some would require a multitude of breakthroughs in foundational sciences such as physics.
This means that the situation for Meta workers will become more difficult as they’re pushed to do things that simply cannot be achieved.
UPDATE (16 DEC)
On 14 December, Intel’s Senior vice president, General manager of the Accelerated Computing Systems and Graphics Group, Raja Koduri, published this paper which supports my assertion that the ‘Metaverse’ (it pains me to use that term, which describes nothing and is made of hype) will require orders of magnitude more computing capacity than currently available.
Here’s a key quote:
Consider what is required to put two individuals in a social setting in an entirely virtual environment: convincing and detailed avatars with realistic clothing, hair and skin tones – all rendered in real time and based on sensor data capturing real world 3D objects, gestures, audio and much more; data transfer at super high bandwidths and extremely low latencies; and a persistent model of the environment, which may contain both real and simulated elements. Now, imagine solving this problem at scale – for hundreds of millions of users simultaneously – and you will quickly realize that our computing, storage and networking infrastructure today is simply not enough to enable this vision.
We need several orders of magnitude more powerful computing capability, accessible at much lower latencies across a multitude of device form factors. To enable these capabilities at scale, the entire plumbing of the internet will need major upgrades. Intel’s building blocks for metaverses can be summarized into three layers and we have been hard at work in several critical areas.
Of course, this can be interpreted as self-serving for Intel which stands to benefit (to say the least) from a massive investment in new computing gear. That doesn’t negate the insight, which is based on hard material reality.
The spread of AI (algorithmic) harms such as automated recidivism and benefits determination systems has been accelerated by the cloud era which has made the proliferation of algorithmic automation possible; indeed, the companies providing cloud services promote their role as accelerators.
Background
We are witnessing a significant change in the way computing power is used and engineered by public and private organizations. The material basis of this change is the availability of utility services such as on-demand compute, storage and database offered primarily by Amazon (with its Amazon Web Services platform), Microsoft (Azure) and Google (Google Cloud Platform). There are other platforms, such as Alibaba, based in the PRC but those three Silicon Valley giants dominate the space. This has come to be known as ‘public cloud’ to distinguish it as a category from private data centers. The term is misleading; ‘public cloud’ is a privately owned service, sold to customers via the public Internet.
‘Public Cloud’ services make it possible for government agencies and businesses to reduce – or eliminate – the work of hosting and maintaining their own computational infrastructure within expensive data centers. Although the advantages seem obvious (for example, reduced overhead and the ability to focus on the use of computer power for business and government goals rather than the costly, complex, time-consuming and often error-prone task of systems engineering) there are also serious new challenges which are having an impact on US, and global, political economy.
The lowered barrier to entry made possible by ‘public cloud’ algorithmic utility services
The marketing value (supported by AI hype) of creating and promoting a product and/or service as based on ‘AI’ (as labor reducing, or even eliminating, automation)
This combination is producing an explosion of algorithmic platforms which are having a direct, negative impact on the lives of millions – notably the poor and people of color but rapidly spreading to all sectors of the population. My position is that this expansion is materially supported by cloud platforms and a lack of public oversight.
I devote a lot of time to understanding, critiquing and criticizing the AI Industrial Complex. Although much – perhaps most- of this sector’s output is absurd, or dangerous (AI reading emotions and automated benefits fraud determination being two such examples) there are examples of uses that are neither which we can learn from.
This post briefly reviews one such case.
During dinner with friends a few weeks ago, the topic of AI came up. No, it wasn’t shoehorned into an otherwise tech-free situation; one of the guests works with large-scale engineering systems and had some intriguing things to say about solid, real world, non-harmful uses for algorithmic ‘learning’ methods.
Specifically, he mentioned Siemens’ use of machine vision to automate the inspection of wind turbine blades via a platform called Hermes. This was a project he was significantly involved in and justifiably proud of. It provides an object lesson for the types of applications which can benefit people, rather than making life more difficult through algorithm.
You can view a (fluffy, but still informative) video about the system below:
Hermes System Promotional Video
A Productive Use of Machine Learning
The solution Siemens employed has several features which make it an ideal object lesson:
1.) It applies a ‘learning’ algorithm to a bounded problem
Siemens engineers know what a safely operating blade looks like; this provides a baseline against which variances can be found.
2.) It applies algorithms to a bounded problem area that generates a stream of dynamic, inbound data
The type of problem is within the narrow limits of what an algorithmic system can reasonably and safely handle and benefits from a robust stream of training data that can improve performance
3.) It’s modest in its goal but nonetheless important
Blade inspection is a critical task and very time consuming and tedious. Utilizing automation to increase accuracy and offload repeatable tasks is a perfect scenario.
How Is This Different from AI Hype?
AI hype is used to convince customers – and society as a whole – that algorithmic systems match, or exceed the capabilities of humans and other animals. Attempts to proctor students via machine vision to flag cheating, predict emotions or fully automate driving are examples of overreach (and the use of ‘AI’ as a behavioral control tool). I use ‘overreach‘ because current systems are, to quote Gary Marcus in his paper ‘The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence‘, “pointillistic” – often quite good in narrow or ‘bounded’ situations (such as playing chess) but brittle and untrustworthy when applied to completely unbounded, real world circumstances such as driving, which is a series of ‘edge cases’.
Visualization of Marcus’ Critique of Current AI Systems
The Siemens example provides us with some of the building blocks of a solid doctrine to use when evaluating ‘AI’ systems (and claims about those systems) and a lesson that can be transferred to non-corporate uses.
This post presents the notes and arguments I used during my interview with hosts Ned and Ethan for the Day Two Cloud podcast (https://daytwocloud.io/). The subject is the need for a public cloud that’s owned by governments as an alternative to the private Cloud Solution Providers – CSPs.
The primary example of why this is needed is provided by an analysis of Amazon which owns both AWS and Amazon retail. My position is that we have abundant evidence that individuals, plus small and medium sized businesses need:
A computing utility that can be publicly held to account for data mining,
Will not be used for competitive advantage
Is an example of zero carbon compute and expands access.
We have entered a critical phase of our history when computing power is broadly needed for both commercial and non-commercial purposes (climate change is the main driver of this urgency but there are other factors). A computing fabric offered as a common utility would empower more individuals and organizations to truly innovate, creating solutions presently out-of-reach due to the unequal access to computing power.
Inequality
Public cloud and emergent technologies such as serverless and cloud hosted machine learning have, in some sense, ‘democratized’ access to capabilities previously only available to deep-pocketed multinationals (if they were available at all). However, the direction and priorities of cloud solution provider platforms such as AWS, Azure and GCP, although perhaps ‘customer driven’ typically do not reflect the longer term needs of the wider societies these businesses operate in. For example, operating at scale on any of these platforms becomes cost prohibitive for capital deprived people and orgs. This situation is only likely to get worse.
The Myth of Private Innovation as Superior to Government Efforts
In the United States (and to varying extent elsewhere) there’s a belief that governments cannot perform the sort of work necessary to create a robust, well-maintained cloud infrastructure (or much of anything, really). This notion persists, even though all the elements of the cloud platforms we use started as government initiatives (ARPAnet into Internet, microprocessor development, etc.) My argument is that this is a fallacy that must be challenged, and which only serves the hagiography of business leaders such as Jobs and Musk and prevents us from supporting our interests.
Mariana Mazzucato ably dissects this in her book, ‘THE ENTREPRENEURIAL STATE’
As an example, Mazzucato diagrams the government sources of the iPhone’s success:
iPhone sources
Additional evidence of the dependence of supposed ‘innovators’ on government can be found in the histories of Tesla and Oracle.
Monopoly Power
The House Hearings on Monopoly Power in the tech industry concluded that, among assessments of other firms (Facebook, Google, Apple) Amazon used the combination of its market position and the computational power of AWS to muscle even the smallest competitors out of their respective markets. The story of Diapers.com was used as an object lesson. Below, I map out the points from the report using a Wardley map:
AMZ Market Dominance Map
Ethics
Although technologists typically focus on the instrumental aspects of a technology (note, for example the circular debates about Kubernetes vs. Serverless without the context of purpose) there are always ethical considerations. One of the most acute examples of this is the field of AI and the sub-category of machine learning; algorithms with impacts on people’s lives are being brought online without full regard for the consequences.
This leads to a question: can private firms which stand to obtain profit from deploying and promoting the use of powerful technologies (such as Sagemaker on AWS and Azure ML Studio) be trusted to – alone – seriously and consistently over time build with ethics foremost in mind.
A publicly accountable technology stack would truly democratize this by placing oversight into public hands and make oversight a central part of any platform.
Note the still-unfolding story of AI Ethics researcher Dr. Timnit Gebru’s firing from Google, covered in the story below:
Standing with Dr. Timnit Gebru — #ISupportTimnit #BelieveBlackWomen
The circumstances of Dr. Gebru’s firing begs the question: would she have been fired if Google was truly dedicated to AI Ethics and, if there was a publicly accountable platform the Dr. could have worked on that was as dynamic, would there be any debate over the need for ethics (i.e., would there have been a tension between the goal of building an ethical tech and profit)?
Ethan’s Questions and My Answers
Why should there be such a thing?
The primary benefit of public cloud is its ability to provide computing (and related) resources as a utility. To-date, this has been viewed as a benefit to business (governments are also seen as customers – note the JEDI contract but the business advantage is the primary driver).
This shapes the priorities and structure of the public cloud (for example, the current focus on hybrid workloads primarily speaks to the concerns of enterprises).
What’s missing are platforms that meet societal needs for the advantages of a computing utility. Imagine, for example, a team of researchers studying the effects of climate change on their local community. Access to a public service cloud computing platform would accelerate the pace of such vital work.
For whom would this offering exist?
The offering would exist to serve the requirements of individuals, non-profits, small communities and others who have a need for computing power but do not have the deep pockets of corporations and who want to use a platform whose priorities are democratically determined rather than being driven by profit incentives.
What are the typical adoption drivers for organizations who might like this idea?
Lower runtime cost, public accountability, and stability would be among the primary adoption drivers (no doubt there are others not yet thought of). Consider, for example, the needs of a community college or a low-income community that wants to provide computational resources to students and under-served communities, a public utility cloud could serve these needs.
Does the ridiculous pace of so-called innovation at AWS, et. al. make a presumably slower-moving public cloud a non-starter?
The ‘innovation’ we see from the major cloud solution providers is a function of the unified APIs upon which their services are built. The recently announced AWS SageMaker Pipelines for example, is possible because new services such as CI/CD pipelines to a machine learning fabric is a latent capability that only needs to be engineered for realization (this doesn’t minimize the effort required to create new features).
These innovations are created to address the needs of large-scale business (for example, IoT for manufacturing and cloud-scale machine learning for the hydrocarbon industry).
It is debatable however, whether many of these innovations are important from a broader point of view. In the second decade of the 21st century, we can consider computational power to be a firmly established need of a complex society. But a rapid pace of change in private entities that isn’t tied to a larger catalog of needs has no effect on the usefulness of a public computational fabric.
Also, there’s an assumption that a government managed service would be inferior to private services. My argument is that this isn’t based on evidence but on a bias that’s been woven into our discourse which has only served the interests of private power (for example, we praise SpaceX but fail to celebrate NASA’s decades of achievements).
What am I getting in a publicly owned utility cloud that I would not get from a commercial offering?
Public control of priorities
Public control of the pricing models
The broader distribution of computing power to under-served communities and organizations
A counter-weight to private power which should not be able to operate unchecked
How should it be funded?
We can imagine a dual funding model of taxation (a dirty word to many, and yet how do we have roads?) and direct pricing. A community college, to return to that example, would both benefit from the public financing of a platform and pay a nominal and scaled run rate for access to services.
Governance model. You mentioned NIST in the [Twitter] thread, but I can see an argument of interested consumers wanting to have input as well. Lots of angles to come at this from that could quickly get unwieldy.
In this model, we move away from the idea of ‘consumers’ (which barely existed in its present form only a few decades ago) and back to that of citizens with a collective interest in a public good. As with other public goods, such as road systems, the fact of public participation does not automatically mean chaos. There are lessons to be learned from effective school boards and consensus based governance. Consider a scenario: Susan has created a cost benefit analysis showing the value of changing the emphasis of her municipality’s computing grid towards serverless. Although techies often pride themselves on how complex and opaque their areas of expertise are, the truth is that if value cannot be clearly stated, the fault is with us. We should have faith that a plurality of people who see a direct benefit from the investment will act in the public interest.
Where would the metal reside? Would existing commercial interests become providers, i.e. adding capacity as demand grows? And how does that align with HIPAA, GDPR, etc.?
Many cities and towns host appropriate locations that are under-utilized or not utilized at all. There are many empty warehouses, former malls, etc that could serve the purpose. There should be no conflict between the requirements of regulatory requirements such as HIPAA and GDPR, which are built upon the idea of guarding privacy, and the hosting of solutions built on a public computing utility. In fact, one could argue that these regulations are more likely to be strictly adhered to by public servants rather than private orgs which seek ways to use data for advertising purposes.
CSPs might be participating service providers but they would have to strictly adhere to regulatory and contractual obligations that would last for decades (longer than the interest horizon of the average business). This would probably act as a disincentive.
If this would be regional, would there be an interconnection of global public clouds beyond simply the Internet? I raise this as Google Cloud, Azure, and AWS all have their global networks.
We can foresee the creation of a standard kit for compute, database and storage with a zero carbon power infrastructure (imagine an abandoned mall retrofitted as a DC, powered by solar and wind according to a common template). These municipal ‘cubes’ could form the basis of a national and then international super-structure. The National Science Foundation Network provides one model that can be updated.
American vs. European perspectives on the government being in the middle of things.
The primary difference between US and European perspectives is that, in the US, many people believe, often without evidence or with anecdotal evidence such as bad experiences at the DMV, that private orgs are better at this sort of ambitious initiative than governments. Many US-ians genuinely believe that Twitter is a cutting edge tech that could only have been produced by a ‘visionary’ leader. This is a fallacy and one that, while it exists in Europe, is not as deeply entrenched (due to people demanding more of their governments and expecting to receive services in return for taxation).
The fictional world of Star Trek, which depicts fanciful technologies such as warp drive, replicators and transporters, presents a surprisingly more realistic view of the potential uses for, and evolution of, advanced computation than the press releases of Google etc. and supportively breathless media accounts.
I say more realistic, because, with notable exceptions (typically used to prove a larger point or create dramatic tension), computers in Star Trek are understood by in-world characters to be mindless, despite exhibiting capabilities which, by our standards, would be considered astounding achievements and irrefutable signs of intelligence and intent.
Artificial Intelligence, an aspirational term that does not describe any existing technology or collection of technologies, is, as a business endeavor, riddled with hype. Consider the article, ‘A robot wrote this entire article. Are you scared yet, human?’ published in the Guardian, 8 September, 2020. The article, assembled by cherry picking output from GPT-3, was, at the time of its publication, promoted as evidence of GPT-3 being a significant step up the ladder towards what’s sometimes called Artificial General Intelligence or AGI. After pushback and critique, Guardian’s editors added a bit more context, admitting that an AI did not, in fact, write the article: “We cut lines and paragraphs, and rearranged the order of them in some places. Overall, it took less time to edit than many human op-eds.” (the bit of face saving at the end is hilarious).
This hype requires, indeed, demands, a variety of counterpoint arguments. Hopefully this essay and the ones to follow will make a contribution.
In a series of three posts, I’ll present three in-show situations (from both the original and Next Generation series) :
The Original Series Episode “The Ultimate Computer”
The Next Generation Episode “The Measure of a Man”
The Next Generation Episode “Boobytrap”
I’ll use these episodes to illustrate Star Trek’s thematic treatment of computer power – as a tool, not to be confused with the complexity and nuance of living minds. Furthermore, I’ll argue that Star Trek posits that the power of minds comes, perhaps paradoxically, from incompleteness (about which, more later).
This may seem trivial or of only academic interest. My argument is that the presentation of computational systems as possessing intelligence is a propaganda project, intended to demobilize workers and obscure the true sources of harm. Each of us who knows better has a responsibility to shine a light on this propaganda in a variety of ways.
This is a part of that effort.
The Ultimate Computer
Dr. Daystrom explains M5
“The Ultimate Computer” is the twenty-fourth episode of the second season of the American science fiction television series Star Trek. Written by D.C. Fontana (based on a story by Laurence N. Wolfe) and directed by John Meredyth Lucas, it was first broadcast on March 8, 1968.”
In “The Ultimate Computer” the viewer is presented with a clear line of separation between the starship Enterprise’s sophisticated library computer system (known as LCARSin the Next Generation series) – which possesses interactive voice response, large language and text synthesis capacities and extensive command and control capabilities – and a thinking machine, the M5, created by Dr. Richard Daystrom (the scientist who designed standard starship computer systems). The M5, patterned after Daystrom’s mind, is able to reason and indeed, exhibits the ability to think in basic ethical terms during a critical scene, when it’s forced to confront the fact its actions resulted in death. Despite these remarkable capabilities, the machine lacks nuance and could be said to operate on the level of an extraordinarily well-informed child.
For me however, the remarkable thing about this episode is the fact in-world characters such as Spock, Kirk and McCoy collectively express astonishment that the machine is able to think at all.
In their experience, there’s a common understanding of what thinking beings do and what sophisticated computers are capable of. There is, in other words, no confusion between the act of rapid, statistical pattern matching, text parsing and data synthesis via sensors and what they, as people, do from moment to moment.
Consider this scene, when Kirk and Spock debate Dr. Daystrom about just what M5 is:
Spock: (to Daystrom, while examining the M5): I am not familiar with these instruments Dr. You are using an entirely new type of control mechanism. However, it appears to me this unit is drawing more power than before.
Daystrom: Quite right! As the unit is called upon to do more work, it pulls more power to enable it to do what is required of it just as a human body draws more energy to run than to stand still.
Spock: Dr, this unit is not a human body. A computer can process information, but only the information that is fed into it.
Kirk (to Daystrom): Granted, it can work a thousand…a million times faster than the human brain but it can’t make a value judgement, it hasn’t intuition, it can’t think.
Daystrom (smiling like a Cheshire Cat – then, waxing poetic) : Can’t you understand? The Multitronic unit is a revolution in computer science. I designed the duotronic elements you use in your ship right now and I know they are as archaic as dinosaurs compared to the M5…a whole…new approach!
[…]
Later, in a tense scene, after M5 has fired weapons on unprotected starships (misinterpreting an exercise for real combat), wounding and killing many, Daystrom tries to reason with it to stop:
Daystrom reasons with M5
Daystrom (to M5 via audio interface): M5 tie-in
M5 (to Daystrom, via ship audio): M5
Daystrom (stressed, trying to calm his voice): This is…this is Daystrom
M5: Daystrom, acknowledged
Daystrom: M5, do you know me?
M5: Daystrom, Richard, originator of comtronic/duotronic systems born…
Daystrom: Stop. M5, your attack on the starships is wrong. You must break it off.
McCoy (to Kirk): I don’t like the sound of him Jim.
Kirk: You’d better pray the M5 listens to the sound of him.
M5 (still responding to Daystrom): Programming includes protection against attack. Enemy vessels must be neutralized
Daystrom: But these are not enemy vessels! These are federation starships. You’re killing…we’re killing…murdering…human beings, beings of our own kind. You were not…created for that purpose. You’re my greatest creation. The ‘unit to save men’ – you must not destroy men.
M5: This unit must survive.
Daystrom: Survive! Yes! Protect yourself! But, not murder. You must not die, men must not die. To kill, is a breaking of civil and moral laws we’ve lived by for thousands of years. You’ve murdered hundreds of people…we’ve murdered…how can we repay that?
M5: They attacked this unit…
Kirk (whispering to Spock while M5 is still replying to Daystrom): The M5 is not responding to him, it’s talking to him.
Spock: I am most impressed with the technology Captain. Dr. Daystrom has created a mirror image of his own mind.
“It’s talking to him” Kirk observes. For him, and everyone else in this world, a clear distinction is made between programmatic response, and actual conversation. This profound difference is purposely obscured by the current discourse which encourages us to view audio response technologies such as Amazon Alexa, Siri and GPT-3 as being capable of conversation.
M5 in motion
In the end, M5, built to create a new class of autonomous computers, intended to replace crewed space vessels, is shown to be deeply inadequate for the task.
This episode establishes what I’ll describe as the pop sci-fi epistemological framework of Star Trek on the question of what Joseph Weizenbaum defined as “Computer Power and Human Reason” (the difference between judgement and calculation). In Star Trek, Computers, as a rule, are unable to reason and incapable of judgement. Outliers and exceptions, such as M5, illustrate this principle via their existence as outliers (which can’t be productionized).
In the next post, I’ll explore how the question of computer power and human reason is addressed in the ‘Next Generation’ episode, “The Measure of a Man“.
As with death and taxes, you can be certain that whenever a video showing a Boston Dynamics robot is shared on Twitter, there are three reliable formulations:
Typically missing are considerations of BD’s business model; who are the customers and what are these robots actually good for, if anything? I decided to do a bit of research – not very deep to be sure but, enough to go beyond social media flailing about.
The sensible place to start was Boston Dynamics’ website which is, unsurprisingly, polished, showcasing production robots such as ‘Stretch‘, ‘Spot‘, ‘Pick’ and of course, everyone’s favorite dancer/supposed robot overlord, Atlas.
The production robot use-cases – per the website – are warehouse operations (where Pick seems a bit in the way) and hazardous conditions operations (Spot’s supposed value-proposition). I didn’t see mention of Spot’s use by police forces as a remote controlled proxy.
Let’s get to Atlas, which is usually the star attraction and undoubtedly a customer and investment attractor for BD. I found a presentation by Scott Kuindersma, Research Scientist and Atlas Project Lead which provides solid, down-to-earth information about how this robot works.
Kuindersma describes Atlas as a demonstration platform. This is important to note because it separates this machine from production offerings. BD’s non-trivial achievement is building a system that can find its center of mass (a centroidal solution) and perform from a catalog of actions.
Atlas’ actions are defined ‘offline’ (i.e., virtually, via computational modeling) and then applied ‘online’ in the real-world. A system called Model Predictive Control puts Atlas’ library of motions to use while 3D plane fitting algos enable navigation through an environment.
As of June 2020, Kuindersma and his team’s focus was building a library of actions for Atlas that would enable it to perform parkour – or something close to it. This will surely be impressive and no doubt lead to further customers and investment.
Now, let’s ask ourselves, who is helped by BD’s work and what are the most likely use-cases? Videos of the warehouse robots show machines that seem slow and inefficient when compared to people. Boxes are moved from points A to B but lots of people – not shown in the videos – are surely required.
Spot, as we’re seeing, is finding use by police forces, for industrial inspection and by militaries. In a way, Spot can be viewed as an earth-bound drone. We can anticipate seeing it deployed in much the same way other remote operated devices are being used – for surveillance and perhaps distance violence.
Atlas, as a research platform, is obviously meant to both advance the state of the art and provide dream-fuel for those who long to see autonomous machines moving among us. The ‘AI’ dream is certain to be dashed but we can foresee remote operation scenarios that are just as dystopian.
Boston Dynamics efforts do bear watching. Not because Atlas – or some successor platform – will, Hollywood-style, grow weary of us and take over the world. Rather, that, as always, people will be behind the curtain, using the veil of machine distance to obscure culpability.
What follows is a sketch, the foundation of a propaganda model, focused on what I’ll call the ‘AI Industrial Complex‘. By the term AI Industrial Complex, (AIIC) I mean the combination of technological capacity (or the lack thereof) with marketing promotion, media hype and capitalist activity that seeks to diminish the value of human labor and talent. I use this definition to make a distinction between the work of researchers and practical technologists and the efforts of the ownership class to promote an idea: that machine cognition is now, or soon will be, superior to human capabilities. The relentless promotion of this idea should be considered a propaganda campaign.
If There’s No AI, What is Being Promoted?
It’s my position there is no existing technology that can be called ‘artificial intelligence’ (how can we engineer a thing we haven’t yet decisively defined?) and that, at the most sophisticated levels of government and industry, the actually existing limitations of what is essentially pattern matching, empowered by (for now) abundant storage and computational power, are very well understood. The existence of university departments and corporate divisions dedicated to ‘AI’ does not mean AI exists; it’s evidence there’s powerful memetic value attached to using the term, which has been aspirational since it was coined by computer scientist John McCarthy in 1956. Once we filter for hype inspired by Silicon Valley hustling (the endless quest to attract investment capital and gullible customers) we are left with promotion intended to shape common perception about what’s possible with computer power.
It’s tempting to see this as a temporarily embarrassing bit of overreach by an enthusiastic field luminary – yet another example of familiar hype but let’s go deeper and ask questions about the political economy underpinning this messaging excess.
Hinton on Radiology in 2016
Radiologists are expensive and, in the US, very much in demand (indeed, there’s a shortage of qualified people). Labor shortages typically lead to higher wages and better working conditions and form the material conditions that create what some call labor aristocracies. In the past, such shortages were addressed via pushes for training and incentives to workers (such as the lavish perks that were common in the earlier decades of the tech era).
If this situation could be bypassed via the use of automation, that would devalue the skilled labor performed by radiologists, solving the shortage problem while increasing the power of owners over the remaining staff.
The promotion of the idea of automated radiology – regardless of actually existing capabilities – is attractive to the ownership class because it holds the promise of weakening labor’s power and increasing – via workforce cost reduction and greater scalability – profitability. I say promotion, because there is a large gap between what algorithmic systems are marketed as being capable of, and reality. This gap, which, as I stated earlier is well understood by the most sophisticated individuals in government and industry, is unimportant to the larger goal of convincing the general population their work efforts can be replaced by machines. The most important outcome isn’t thinking machines (which seems to be a remote goal if possible at all) but a demoralized population, subjected to a maze of crude automated systems which are described as being better than the people forced to navigate life through these systems.
A Factor Among Factors
Technological systems – and the concepts attached to them – emerge from, and reflect the properties of the societies that create those systems. Using the Hegelian (and later, Marxist) philosophy of internal relations, we can analyze both real algorithmic systems and the concept of ‘AI’ as being a part of the interplay of factors that comprise global capitalist dynamics – both actor and acted upon. From this point of view, the propaganda effort promoting ‘AI’ should not be considered in isolation, but as one aspect of a complex.
Hype vs. Propaganda
What defines hype and what differentiates standard industry hype from a propaganda campaign?
Hype (such as marketing material that makes excessive claims – for example, AI reading emotions) is narrowly designed to attract investment capital and customers. Hype should be considered a species of advertisement. Propaganda has a broader aim, which is described by Jacques Ellul in his work, Propaganda.
Describing one of the four elements of propaganda, and bridging from advertising to propaganda, Ellul writes…
“Public and human relations: These must necessarily be included in propaganda. This statement may shock some readers, but we shall show that these activities are propaganda because they seek to adapt the individual to a society, to a living standard, to an activity. They serve to make him conform, which is the aim of all propaganda. In propaganda we find techniques of psychological influence combined with techniques of organization and the envelopment of people with the intention of sparking action.”
A Propaganda Model: Foundational Concepts
As the model of AI as propaganda is constructed, the works of three thinkers will provide key guidance:
Jacques Ellul: Propaganda
As already noted, Ellul’s key work on propaganda (which, I think, was the first to apply sociology and psychology to the topic) is a critical source of inspiration:
“Propaganda is first and foremost concerned with influencing an individual psychologically by creating convictions and compliance through imperceptible techniques that are effective only by continuous repetition. Propaganda employs encirclement on the individual by trying to surround man by all possible routes, in the realm of feelings as well as ideas, by playing on his will or his needs through his conscious and his unconscious, and by assailing him in both his private and his public life.
The propagandist also acknowledges the most favorable moment to influence man is when an individual is caught up in the masses. Propaganda must be total in that utilizes all forms of media to draw the individual into the net of propaganda. Propaganda is designed to be continuous within the individual’s life by filling the citizen’s entire day. It is based on slow constant impregnation that functions over a long period of time exceeding the individual’s capacities for attention or adaptation and thus his capabilities of resistance”
The relentless promotion of the idea that automation is on the verge of replacing human labor can be interpreted as being part of an effort to create a conviction (there is artificial intelligence’, it cannot be stopped) and compliance (resistance to ‘AI’ is retrogressive Luddism).
Noam Chomsky/Edward S. Herman: The Propaganda Model
In their book, ‘Manufacturing Consent’ Chomsky and Herman present a model of propaganda via media:
“The third of Herman and Chomsky’s five filters relates to the sourcing of mass media news:
The mass media are drawn into a symbiotic relationship with powerful sources of information by economic necessity and reciprocity of interest. Even large media corporations such as the BBC cannot afford to place reporters everywhere. They concentrate their resources where news stories are likely to happen: the White House, the Pentagon, 10 Downing Street and other central news “terminals”. Although British newspapers may occasionally complain about the “spin-doctoring” of New Labour, for example, they are dependent upon the pronouncements of “the Prime Minister’s personal spokesperson” for government news. Business corporations and trade organizations are also trusted sources of stories considered newsworthy. Editors and journalists who offend these powerful news sources, perhaps by questioning the veracity or bias of the furnished material, can be threatened with the denial of access to their media life-blood – fresh news. Thus, the media has become reluctant to run articles that will harm corporate interests that provide them with the resources that they depend upon.
The dependence of news organizations on press releases from Google and other tech giants that promote the idea of ‘AI’ can be interpreted as being an example of the ‘symbiotic relationship, based on reciprocity of interest’ Chomsky and Herman detail.
The concept of “artificial intelligence” is aspirational (like ‘warp drive’) and does not describe any existing or likely to exist computational system. Despite this, the concept is promoted to attract investment capital and customers but also, more critically for my purposes, devalue the power of labor – if not in fact than in perception (which, in turn, becomes fact). For this reason, I assert that ‘AI’, as a concept, is part of a propaganda campaign.
Key Characteristics of AI Propaganda
The promotion of the concept of AI, as a propaganda effort, has several elements:
* Techno-optimism: The creation of thinking machines is promoted as being possible, with little or no acknowledgement of limitations.
* Techno-determinism: The creation of thinking machines is promoted as being inevitable and beyond human intervention, like a force of nature
* An Elite Project: Although individual boosters, grifters, techno enthusiasts and practitioners may contribute within their circles (for ex. social media) to hype, the propaganda campaign is an elite project designed to effect political economy and the balance of power between labor and capital.
* Built on, but not limited to, hype: There is a relationship between hype and propaganda. Hype is of utility to the propaganda campaign but the objective of that campaign is broader and targeted towards changing societal attitudes and norms.
I use the term attack mannequins to describe this complex – lifeless things, presented as being lifelike, used to assault the position and power of ordinary people.
UPDATE: 2 NOVEMBER 2021
In this video, YouTube Essayist Tom Nicholas details the efforts Waymo has made to convince people – via the use of YouTube ‘educators’ – that autonomous vehicles are a perfected technology, superior to human drivers and a solution to traffic safety and congestion issues.
Nicholas makes the point that inasmuch as the Waymo ‘autonomous’ taxi service (supported by a large staff of people behind the scenes) only operates in a subsection of the suburbs of Phoenix, Arizona USA, the PR campaign’s goal can’t be explained as advertising; it’s part of a broad effort to change minds.
Computational Impacts 