AT THE END of ´The Wizard of OZ´, Metro-Goldwyn-Mayer´s 1939-released, surrealist musical fantasy, our heroine Dorothy and her loyal comrades complete a long, arduous (but song filled) journey, finally reaching the fabled city of OZ. In OZ, according to a tunefully stated legend, there’s a wizard who possesses the power to grant any wish, no matter how outlandish. Dorothy, marooned in OZ, only wishes to return home and for her friends to receive their various hearts´ desire.
As they cautiously approach the Wizard’s chamber, Dorothy and her friends are met with a display of light, flame and sound; ¨who dares!?¨ a deafening voice demands. It’s quite a show of apparent fury but illusion crumbles when it’s revealed (by Dorothy´s dog, Toto) that behind it all is a rather ordinary man, hidden on the other side of a velvet curtain, frantically pulling levers and spinning dials to keep the machinery powering the illusion going while shouting, “pay no attention to that man behind the curtain!“
Behind the appearance of magic, there was a noisy industrial process, belching smoke. Instead of following the Wizard’s advice to pay no attention, let’s pay very close attention indeed to what lies behind appearances.
THERE’S AN INESCAPABLE MATERIALITY behind what’s called ‘AI’ deliberately obscured under a mountain of hype, flashy images and claims of impending ‘artificial general intelligence’ or ‘AGI’ as it’s known in sales brochures disguised as scientific papers.
At the heart of the success of techniques such as large language models, starting in the latter 2010s, is the graphics processing unit or GPU (in this essay about Meta´s OPT-175B, I provide an example of how GPUs are used). These devices use a parallel architecture, which enables greater performance than the general purpose processors used for your laptop; this vastly greater capability is the reason GPUs are commonly used for demanding applications such as games and now, the hyper-scale pattern matching behind so-called ´AI´ systems.
All of the celebrated feats of ‘AI’ – platforms such as Dall-E, GPT-3 and so on, are completely dependent on the use of some form of GPU, most likely provided by NVIDIA, the leading company in this space. OpenAI, a Microsoft partner, uses that company’s Azure cloud but within those ´cloud´ data centers, there are thousands upon thousands of GPUs, consuming power and requiring near constant monitoring to replace failed units.
GPUs are constructed as the result of a long and complex supply chain involving resource extraction, manufacturing, shipping and distribution; even a sales team. ‘AI’ luminaries and their camp followers, the army of bloggers, podcasters and researchers who promote the field, routinely and self-indulgently debate a variety of esoteric topics (if you follow the ´AI´ topic on Twitter, for example, odds are you have observed and perhaps participated in these discussions about vague topics such as, ´the nature of intelligence´) but it’s GPUs and their dependencies all the way down
GPU raw and processed material inputs are aluminum, copper, clad laminates, glass, fibers, thermal silica gel, tantalum and tungsten. Every time an industry partisan tries to ‘AI’-splain the field, declaring it to be a form of magic, ignore their over-determination and confusion of feedback loops with cognition and think of those raw materials, ripped from the ground.
The ‘AI’ industrial complex is beset by two self-serving fantasies:
1.) We are building intelligence
2.) The supply chain feeding the industry is infinite and can ‘scale is all you need’ its way forever to a brave new world.
For now, this industry has been able to keep the levers and dials moving, but the amount of effort required will only grow as the uses to which this technology is put expand (Amazon alone seems determined to find as many ways to consume computational infrastructure as possible with a devil take the hindmost disregard for consequences), the need for processors grows and global supply chains are stressed by factors such as climate change, and geopolitical fragmentation.
The Wizards, out of tricks, curtains pulled, will be revealed as the ordinary (mostly) men they are. What comes next, will be up to us.
Some Key References:
Computational Impacts 