The Summariser

Original text:

Artificial intelligence in its current form is based on the wholesale appropriation of existing culture, and the notion that it is actually intelligent could be actively dangerous.

In January 2021, the artificial intelligence research laboratory OpenAI released a piece of software called Dall-E. The software allowed users to enter a simple description of an image they had in their mind, and, after a brief pause, the software would produce an almost uncannily good interpretation of their suggestion. Typing in, for example, “a pig with wings flying over the moon, illustrated by Antoine de Saint-Exupéry” resulted, after a minute or two of processing, in something reminiscent of the watercolour brushes of the creator of The Little Prince.

The internet went wild. Social media was flooded with all sorts of bizarre and wondrous creations. And a few months later it happened again, this time with language, and a product called ChatGPT, also produced by OpenAI. Ask ChatGPT to produce a summary of the Book of Job in the style of the poet Allen Ginsberg and it would come up with a reasonable attempt in a few seconds.

The name Dall-E combines the robot protagonist of Disney’s Wall-E with the Spanish surrealist artist Salvador Dalí. On the one hand, you have the figure of a plucky, autonomous and adorable little machine sweeping up the debris of a collapsed human civilisation, and on the other a man whose most repeated bon mots include, “Those who do not want to imitate anything, produce nothing,” and “What is important is to spread confusion, not eliminate it.” Both make admirable namesakes for the broad swathe of tools that have come to be known as AI image generators.

AI image generation relies on the assembly and analysis of millions of tagged images; that is, images that come with some kind of description of their content already attached. These images and descriptions are then processed through neural networks that learn to associate particular and deeply nuanced qualities of the image – shapes, colours, compositions – with certain words and phrases. These qualities are then layered to produce new arrangements of shape, colour and composition. But where did all those original images come from? These images are “public” images in the broadest sense: any image ever published on the internet may be gathered up.

AI image and text generation is pure primitive accumulation: expropriation of labour from the many for the enrichment and advancement of a few Silicon Valley technology companies and their billionaire owners. They have enclosed our imaginations in much the same manner as landlords and robber barons enclosed once-common lands. They promised that in doing so they would open new realms of human experience, give us access to all human knowledge, and create new kinds of human connection. Instead, they are selling us back our dreams repackaged as the products of machines, with the only promise being that they’ll make even more money advertising on the back of them.

The weirdness of AI image generation exists in the output as well as the input. One user tried typing in nonsense phrases and was confused and somewhat discomforted to discover that Dall-E mini seemed to have a very good idea of what a “Crungus” was: an otherwise unknown phrase that consistently produced images of a snarling, naked, ogre-like figure. Crungus was sufficiently clear within the program’s imagination that he could be manipulated easily: other users quickly offered up images of ancient Crungus tapestries, Roman-style Crungus mosaics, oil paintings of Crungus, photos of Crungus hugging various celebrities, and this being the internet, “sexy” Crungus.

So, who or what is Crungus? Twitter users were quick to describe him as “the first AI cryptid”, a creature like Bigfoot who exists, in this case, within the underexplored terrain of the AI’s imagination. The Crungus is a dream emerging from the AI’s model of the world, composited from billions of references that have escaped their origins and coalesced into a mythological figure untethered from human experience. Which is fine, even amazing – but it does make one ask, whose dreams are being drawn upon here? What composite of human culture, what perspective on it, produced this nightmare?

A similar experience occurred to another digital artist experimenting with negative prompts, a technique for generating what the system considers to be the polar opposite of what is described. When the artist entered “Brando::-1”, the system returned something that looked a bit like a logo for a video game company called DIGITA PNTICS. That this may, across the multiple dimensions of the system’s vision of the world, be the opposite of Marlon Brando seems reasonable enough. But when they checked to see if it went the other way, by typing in “DIGITA PNTICS skyline logo::-1”, something much stranger happened: all of the images depicted a creepy-looking woman with sunken eyes and reddened cheeks, who the artist christened Loab. Once discovered, Loab seemed unusually and disturbingly persistent. Feeding the image back into the program, combined with ever more divergent text prompts, kept bringing Loab back, in increasingly nightmarish forms, in which blood, gore and violence predominated.

There’s one explanation for Loab, and possibly Crungus: although it’s very, very hard to imagine the way the machine’s imagination works, it is possible to envisage it. It is going to have areas full of information and areas lacking many features at all. Those areas of high information correspond to networks of associations that the system “knows” a lot about. One can imagine the regions related to human faces, cars and cats, for example, being pretty dense, given the distribution of images one finds on a survey of the whole internet. It is these regions that an AI image generator will draw on most heavily when creating its pictures. But there are other places, less visited, that come into play when negative prompting – or indeed, nonsense phrases – are deployed. In order to satisfy such queries, the machine must draw on more esoteric, less certain connections, and perhaps even infer from the totality of what it does know what its opposite may be. Here, in the hinterlands, Loab and Crungus are to be found.

That’s a satisfying theory, but it does raise certain uncomfortable questions about why Crungus and Loab look like they do; why they tip towards horror and violence, why they hint at nightmares. Perhaps this is just a sign that these systems are very good indeed at aping human consciousness, all the way down to the horror that lurks in the depths of existence: our fears of filth, death and corruption. And if so, we need to acknowledge that these will be persistent components of the machines we build in our own image.

While claims about AI’s “creativity” might be overblown – there is no true originality in image generation, only very skilled imitation and pastiche – that doesn’t mean it isn’t capable of taking over many common “artistic” tasks long considered the preserve of skilled workers, from illustrators and graphic designers to musicians, videographers and, indeed, writers. This is a huge shift. AI is now engaging with the underlying experience of feeling, emotion and mood, and this will allow it to shape and influence the world at ever deeper and more persuasive levels.

ChatGPT was introduced in November 2022 by OpenAI, and further shifted our understanding of how AI and human creativity might interact. Structured as a chatbot – a program that mimics human conversation – ChatGPT is capable of a lot more than conversation. When properly entreated, it is capable of writing working computer code, solving mathematical problems and mimicking common writing tasks, from book reviews to academic papers, wedding speeches and legal contracts.

It was immediately obvious how the program could be a boon to those who find, say, writing emails or essays difficult, but also how, as with image generators, it could be used to replace those who make a living from those tasks. Many schools and universities have already implemented policies that ban the use of ChatGPT amid fears that students will use it to write their essays, while the academic journal Nature has had to publish policies explaining why the program cannot be listed as an author of research papers.

If it would be inappropriate to replace our communications wholesale with ChatGPT, then one clear trend is for it to become a kind of wise assistant, guiding us through the morass of available knowledge towards the information we seek. Microsoft has been an early mover in this direction, reconfiguring its often-disparaged search engine Bing as a ChatGPT-powered chatbot, and massively boosting its popularity by doing so. But despite the online (and journalistic) rush to consult ChatGPT on almost every conceivable problem, its relationship to knowledge itself is somewhat shaky.

I recently asked ChatGPT to suggest some books to read based on a new area of interest: multi-species democracy, the idea of including non-human creatures in political decision-making processes. And ChatGPT obliged. It gave me a list of several books that explored this novel area of interest in depth, and described in persuasive human language why I should read them. This was brilliant! Except, it turned out that only one of the four books listed actually existed, and several of the concepts ChatGPT thought I should explore further were lifted wholesale from right-wing propaganda: it explained, for example, that the “wise use” movement promoted animal rights, when in fact it is a libertarian, anti-environment concept promoting the expansion of property rights.

Now, this didn’t happen because ChatGPT is inherently right-wing. It’s because it’s inherently stupid. It has read most of the internet, and it knows what human language is supposed to sound like, but it has no relation to reality whatsoever. It is very good at producing what sounds like sense, and best of all at producing cliché and banality, which has composed the majority of its diet, but it remains incapable of relating meaningfully to the world as it actually is.

The belief in this kind of AI as actually knowledgeable or meaningful is actively dangerous. It risks poisoning the well of collective thought, and of our ability to think at all. If, as is being proposed by technology companies, the results of ChatGPT queries will be provided as answers to those seeking knowledge online, and if, as has been proposed by some commentators, ChatGPT is used in the classroom as a teaching aide, then its hallucinations will enter the permanent record, effectively coming between us and more legitimate, testable sources of information, until the line between the two is so blurred as to be invisible. Moreover, there has never been a time when our ability as individuals to research and critically evaluate knowledge on our own behalf has been more necessary, not least because of the damage that technology companies have already done to the ways in which information is disseminated. To place all of our trust in the dreams of badly programmed machines would be to abandon such critical thinking altogether.

If these current incarnations of “artificial” “intelligence” are so dreary, what are the alternatives? Can we imagine powerful information sorting and communicating technologies that don’t exploit, misuse, mislead and supplant us? Yes, we can – once we step outside the corporate power networks that have come to define the current wave of AI.

In fact, there are already examples of AI being used to benefit specific communities by bypassing the entrenched power of corporations. Indigenous languages are under threat around the world. The UN estimates that one disappears every two weeks, and with that disappearance goes generations of knowledge and experience. This problem, the result of colonialism and racist assimilation policies over centuries, is compounded by the rising dominance of machine-learning language models, which ensure that popular languages increase their power, while lesser-known ones are drained of exposure and expertise.

In Aotearoa New Zealand, a small non-profit radio station called Te Hiku Media, which broadcasts in the Māori language, decided to address this disparity between the representation of different languages in technology. Its massive archive of more than 20 years of broadcasts was being digitised but needed to be transcribed to be of use to language researchers and the Māori community. In response, the radio station decided to train its own speech recognition model, so that it would be able to “listen” to its archive and produce transcriptions.

Over the next few years, Te Hiku Media, using open-source technologies as well as systems it developed in house, achieved the almost impossible: a highly accurate speech recognition system for the Māori language, which was built and owned by its own language community. This was more than a software effort. The station contacted every Māori community group it could and asked them to record themselves speaking pre-written statements in order to provide a corpus of annotated speech. There was a cash prize for whoever submitted the most sentences but the organisers found that the greatest motivation for contributors was the shared vision of revitalising the language while keeping it in the community’s ownership.

Te Hiku Media’s achievement established the principle of data sovereignty around indigenous languages, and by extension, other forms of indigenous knowledge. When international for-profit companies started approaching Māori speakers to help build their own models, Te Hiku Media campaigned against these efforts, arguing, “They suppressed our languages and physically beat it out of our grandparents, and now they want to sell our language back to us as a service.”

The lesson of the current wave of “artificial” “intelligence”, I feel, is that intelligence is a poor thing when it is imagined by corporations. If your view of the world is one in which profit maximisation is the king of virtues, and all things shall be held to the standard of shareholder value, then of course your artistic, imaginative, aesthetic and emotional expressions will be woefully impoverished. We deserve better from the tools we use, the media we consume and the communities we live within, and we will only get what we deserve when we are capable of participating in them fully. And don’t be intimidated by them either – they’re really not that complicated. As the science-fiction legend Ursula K Le Guin wrote: “Technology is what we can learn to do.”

Adapted from this original article, published in The Guardian.

Reading notes:

• In January 2021, OpenAI released Dall-E: software capable of creating images from textual descriptions. The software demonstrates interpretative ability, such as producing images that remind one of Antoine de Saint-Exupéry’s style from a simple prompt.
• A few months after Dall-E, OpenAI introduced ChatGPT, another breakthrough product, which focuses on language. This product captured public attention through its ability to generate text based on specific prompts, including summaries in the style of famous writers.
• The name Dall-E is a blend of Disney’s Wall-E and Salvador Dalí, reflecting the fusion of a machine and a surrealist artist known for encouraging confusion.
• AI image generation works by analysing millions of tagged images to learn associations between words and visual qualities. It relies on public internet images, which raises questions about the origins and ownership of these images.
• Silicon Valley companies’ development and use of AI technologies are critiqued as a form of primitive accumulation, exploiting collective labour for private gain while promising but failing to deliver meaningful enhancements to the human experience.
• AI image generation’s capacity to interpret and visualise even nonsensical inputs is illustrated by the phenomenon of “Crungus,” an imagined figure that the AI consistently depicted in various artistic styles.
• The creations of Crungus and Loab demonstrate the AI’s ability to generate mythological figures from the collective cultural database, prompting reflections on the origin and ownership of these AI-generated dreams.
• The discovery of Loab through negative prompting highlights the AI’s capacity to navigate less understood, esoteric associations, suggesting that AI can infer from vast data pools to visualise concepts opposite to given prompts.
• The existence of figures like Crungus and Loab raises questions about the nature of AI’s “imagination” and its tendency towards creating images of horror and violence, suggesting a reflection of deep-seated human fears.
• Despite scepticism about AI’s “creativity,” its ability to perform tasks traditionally reserved for skilled human artists indicates a significant shift in how creative work is understood and executed.
• The introduction of ChatGPT and its implications for professional writing, including academic integrity and the potential displacement of human workers, reflects broader concerns about AI’s role in creative industries.
• An example of ChatGPT providing inaccurate information illustrates the limitations and potential dangers of relying on AI for knowledge, highlighting the importance of critical evaluation and verification.
• The proliferation of AI-generated content threatens to blur the line between legitimate information and AI fabrications, undermining critical thinking skills.
• Initiatives like Te Hiku Media’s development of a speech recognition system for the Māori language demonstrate the potential for AI to serve specific communities rather than corporate interests, emphasising the principle of data sovereignty.
• The critique of corporate-driven AI development concludes with a call for technologies that empower and enrich human communities.

Summary of James Bridle’s article “The Stupidity of AI”:

In the article entitled “The Stupidity of AI,” published in The Guardian on March 16, 2023, the author, James Bridle, criticises corporate-driven AI development and argues that it exploits collective labour for the enrichment of a few, undermining human creativity and the potential for technology to enhance lives. Bridle begins by introducing Dall-E and ChatGPT, two AI products from OpenAI that captivated the public with their ability to generate images and text from simple prompts. These tools, according to Bridle, may seem like technological marvels, but they also raise profound questions about the nature of creativity, ownership, and the origins of the images and narratives they produce.

Bridle takes issue with the model of “primitive accumulation” underlying AI development, where Silicon Valley companies amass vast datasets from the public domain, effectively privatising collective human creativity and experience for commercial gain. He highlights AI’s bizarre and often unsettling outputs, such as the fictional entities “Crungus” and “Loab,” to underscore the unpredictability and deep cultural entanglements of AI-generated content. These examples question the source and ownership of the “dreams” AI draws upon, suggesting a disconcerting detachment from human values and experiences.

The article further critiques AI’s foray into artistic and professional domains traditionally reserved for skilled human labour. It suggests that while AI may perform these tasks, it lacks genuine creativity and understanding. Bridle recounts personal experiences with ChatGPT, pointing out its tendency to produce plausible yet factually incorrect or misleading information. This, he argues, represents a significant risk, as reliance on AI for knowledge could erode critical thinking skills and blur the line between verified information and AI-generated fabrications.

Bridle concludes by advocating for a reimagining of AI development outside the confines of corporate interests, highlighting initiatives like Te Hiku Media’s Māori language speech recognition system as exemplars of community-oriented, empowering technology. He calls for technologies that enrich human communities and knowledge, thus urging a shift towards AI that serves the collective good rather than corporate profits.