Literary Attractor States

Share
Literary Attractor States

Another week, another scandal: last month, a short story bearing many of the hallmarks of AI writing won the Caine Prize, a literary award for the best short story by an African writer in the English language. The story was quickly flagged as being AI. This is just one of several AI literary scandals this year. The Caine joins the Commonwealth Short Story Prize, Prix du roman Fnac, Almadía-Ventosa Arrufat International Short Novel Prize, and the (admittedly less-prestigious) Harper’s Bazaar Short Story Competition in awarding AI-generated writing. But now the Caine committee evidently decided to up the ante. The committee released a statement defending their decision, and their statement has been flagged as being AI as well.

It seems clear to me that the literary establishment doesn’t know what to do about AI writing. The flood of slop enveloping magazines and award juries is causing traditional artisanal processes of judging literature to break down, and new methods are yet to take their place. People in the literary establishment are understandably embarrassed and hostile, and while I believe the hostility is fruitless, the embarrassment can be productive.

What I mean is that the literary establishment needs to engage with AI seriously. Refusing to engage with AI at all is counterproductive. It may be that literary judges lack sufficient familiarity with language model output to pick up on the tics and tells. If you don’t use the tools, or ever bother to learn how they work, you won’t build up the prerequisite “immunity” necessary to identify their output. Multiple studies (here and here) have found that familiarity with AI helps you better identify AI output than literary expertise. Likewise, I believe that accusing Pangram, probably the most well-known AI detector, of being uniquely hostile to African or non-western literature is profoundly misguided (and very likely untrue). It’s a form of shooting the messenger, and will only serve to undermine this artform’s survival.

I am not a member of the literary establishment. I have neither won nor judged a major literary award. But I do believe I have something to add to this discussion. I have an MFA, and I have built and trained AI models. I am currently engaged in a research project to expand the pipeline of language model persona training using historic literature. I have spent a considerable amount of time teaching both humans and machines to be writers, as it were. So, when I look at this problem, it is as if I am looking at it through a pair of glasses with lenses of two different colors: one lens colored by my training as a writer, the other colored by my work as a model designer.

I am, regrettably, monolingual. I won’t write about the Almadía-Ventosa Arrufat scandal, though it has been thoroughly covered in a great (Spanish-language) post by @brunella on her Substack Que Verguenza. Nor will I attempt to write about the Prix du roman Fnac. But I can put on my dual-colored glasses and look at the three prize-winning short stories and the Caine Prize statement. I am interested in the qualitative features of AI writing and the reasons that it exhibits those features. My hope is to help others interested in literature engage seriously with AI writing as well.

Something Like Catching Light Without Asking Permission

Before we get into the nitty-gritty mechanics of why machines write the way they do, let’s start with some examples. "The Serpent in the Grove" by Jamir Nazir (winner of the Commonwealth) and “Back and Forth” by Kavyta Kay (winner of Harper's Bazaar) share a tortured prose style, a kind of dime-store Arundhati Roy, and many of the same descriptive sentences. Here nonphysical entities act “without asking permission”:

The Serpent in the Grove: “Hard living lays itself on a man like wet sacking; it never asks permission.”
Back and Forth: “Time folded, as it always did, without asking permission”

Another noteworthy similarity: fabric “catching light.” It’s not totally unusual to describe an object as “catching light,” but it is somewhat odd to describe textiles in that way, especially textiles (like towels) that aren’t especially reflective: 

The Serpent in the Grove: “He thinks of a towel catching light, a jasmine moved from a mouth to a door, letters spelling a name until a name meant breath.” 
The Serpent in the Grove: “She had the kind of walking that made benches become men. Hair tumbling wild, a dress that caught and released light, laughter with iron under it.”
Back and Forth: “Behind her, the apple-tree stood in the dusk. Green silk catching what remained of the light—gold thread, small flames against the grey.”

The prose rhythm of the stories also has the fingerprints of AI. One common tell of AI writing is a triple-adjective structure, for example:

The Serpent in the Grove: “Outside, little Puttie – three years old, sun-dark, bright-eyed – chased a yard fowl through the dust, his laughter like water over pebbles.”

“The Blue Fighter” by Simbarashe Steyn Kundizeza, which won the Caine prize, lacks the tortured prose of the other two short stories, but it exhibits some of the rhythms of AI prose. Here, once again, we see the three-adjective structure:

The Blue Fighter: “We sat in silence for a while, watching him move. Restless, beautiful. Alone.”

You can see the same cadence in the Caine Prize’s statement, which, again, was flagged by Pangram as being AI: 

Another famously-common element of AI writing is the “Not X, but Y” structure. It occurs in several permutations—”not an X, just a Y,” “Y, not an X,” “Not X, not Y, but Z”— but once you recognize it, it is unmistakable. Here is the Caine Prize’s statement, with the first line being our example:

And several examples of the "not x, but y" structure from the stories:

Back and Forth: “The tears came quietly, the way winter rain comes down in London: persistent, unhurried. Not dramatic sobbing, just a steady leak, her body admitting what her mind wouldn’t.”
Back and Forth: “She understood now – not all at once, not cleanly. Just enough to stop pretending she didn’t.”
The Serpent in the Grove: “They say the grove still hums at noon. Not the bees’ neat industry or the clean rasp of cutlass on vine, but a belly sound – as if the earth swallows a shout and holds it there.” 

When I read “The Blue Fighter,” the “not X, but Y” construction occurred four times in a story that runs a little over 3,500 words:

“Not with anger, but with the quiet resignation of someone who had known all along this moment would come.”
“Not broken, just learning how to live differently.”
“Not a question, but an accusation.”
“Father's head snapped up. "They're not coffins. They're investments."’

Finally, the three stories share a tendency toward weird vagaries. To wit:

Back and Forth: Deepa would realise this later, when clarity returned – if it ever did. For now, the knowing belonged to something older than her, something rooted and patient.
The Serpent in the Grove: He looked at the plank mouth and the cutlass leaning casual, as if it had legs. Something coiled inside her chest.
The Serpent in the Grove: He saw all of it in a knife-second. Something moved in him.
The Blue Fighter: The sharp edges of her cheekbones softened as something like purpose filled the hollows beneath them.
The Blue Fighter: Father's shoulders slumped. He looked at the tanks, then at us, as if seeing the distance between these worlds for the first time. 

You may think I'm being uncharitable with the passages above, but in fact, while the stories are eager to talk about how the characters experience "something like" something else, they are allergic to naming what the thing is. The knowing belonged to something older than her, but we never learn what. Father looks as if seeing the distance between worlds for the first time, but we don't know if he is actually seeing the distance between his worlds for the first time. Throughout the stories, there are many, many instances of this literary-sounding equivocation.

The occurrence of these features alone does not prove AI writing. Every writer in history has used these structures, or written a clunky metaphor or a vague description of somebody's interior state. I know that I have put my belief in AI detectors here, and maybe Pangram will be revealed as a scam and I'll be revealed as a schmuck. But putting the validity of automated AI detectors aside (and there is good reason to believe the validity of Pangram in particular) the recurrence of these stylistic features both within and across stories is interesting to me. I doubt the authors of any of these four pieces are one-shot prompting their stories. There is probably quite a bit of back and forth going on as they feed the model ideas, ask for changes, and maybe make edits of their own. I, however, trust Pangram's assertion that they are written by machines, and the fact is that the four pieces are similar in ways that are consistent with machine writing.

Why does AI write this way?

Let’s start with the clunky metaphors. When we use a metaphor, we want to say that one thing is like another thing. Their marriage was a ticking time bomb. The baby is a giggling ball of dough. Metaphors are fun to write, and student writers will enthusiastically mix them together: the baby was a giggling ball of dough that blew their marriage to smithereens. It’s an amusing image, but it doesn’t make any sense! Dough doesn’t make anything explode. We construct metaphors from observations about the world—the doughiness of babies or the volatility of a bad marriage. While mixed metaphors are not terrible writing per se, they do make it difficult for your readers to understand what you’re really saying. If the purpose of writing is to communicate the way you observe the world, mixed metaphors make your observations inscrutable.

AI, though, does not write or “think” in this way. It has no experience of the world and has never observed anything. Instead, it constructs sentences through probabilities: given the prior context X, what is a probable token Y that follows? AI models embed a great deal of information about the relationships between tokens, but fundamentally, “probability of token Y given prior context X” is how they operate. Since they have no perceptual experience, their metaphors are not rooted in observations about the world, but what is locally probable based on what they have gleaned from their training data. Take this sentence from “The Serpent in the Grove”:

“In the hot hush, the grove held its breath and released it – small and entire, like a last stitch drawn through a wound that had finally decided to close.”

The grove is breathing, but the breath is like a suture on a wound? It’s a mixed metaphor, and a confusing one. Likewise, take an example we have already seen from the same story: “Hard living lays itself on a man like wet sacking; it never asks permission.” The sentence's clumsy phrasing conjures the bizarre image of wet sacking asking permission before it lays itself on a man. “The Serpent in the Grove” and “Back and Forth,” in particular, are both littered with confusing mixed metaphors, because the metaphors do not arise out of observations about the world. Instead, a human is prompting a machine to write in a literary manner, and the machine constructs that writing by next-token probability. It means that each part of the metaphor may be locally valid, but there is no global constraint. This process produces plausible-sounding literary moves that do not cohere when taken together.

The structured sentences we see across all four pieces are the result of a similar mechanism. Reading trillions and trillions of tokens during pretraining, the models have learned that descriptors sometimes occur in triplicate—Hobbes’ “nasty, brutish, and short”, Frost’s “Lovely, dark and deep”, Franklin’s “healthy, wealthy, and wise.” Likewise, the models have learned that when a sentence begins with “It’s not X…” it often ends with “it’s Y.” If you were to prompt a base model to write a short story, these constructions would appear at a roughly similar frequency as they do in the pretraining corpus.

However, a base model’s training corpora includes a long tail of writing, from singular masterpieces of world literature to Reddit how-to posts to hate-filled screeds. During the post-training process, publicly-available commercial models are trained to behave as “helpful digital assistants." There are many methods to do this, but the final step is often reinforcement learning, which rewards the model for good output and penalizes the model for undesirable output. This has the effect of reducing the probability of outputs that do not reflect that persona and do not help the user. This shift in the total probability mass reliably reduces the diversity of model output.

For something like coding, this is very useful. For literature, the best of which lives in the long tail of the strange and beautiful, the effect is homogeneity. Commercial models prompted to tell a story will reach for the same constructions, the same metaphors, and the same cliches, because post-training has concentrated their output probability within a narrow distribution. A language model won't necessarily produce these constructions. It depends on the pretraining corpus and post-training decisions. Some fine-tuned models can be very good at imitating the writing of famous authors, but fine-tuning can be expensive and quite technical. The publicly-available commercial models are easy to use and, even better, free. This has created a set of literary attractor states that have become the hallmark of machine writing.

That’s why we see “it’s not X, it’s Y” occurring four times in one story: a narrowed output diversity means that certain constructions occur at a much higher frequency than they do in “natural” human writing. If a construction appears in the output, it in fact makes additional appearances of that construction more likely. Furthermore, the model won’t look at a given construction and say, “Hmm, I used that two pages ago. Better phrase it differently here”—the way a human writer would. One piece of advice sometimes given to young writers is to keep a mental model of a reader in their heads while reviewing their own work. This teaches them to think, "How will another person reading this prose react to what they encounter?" AI models undergo a different process. Instead, they have learned in post-training to optimize for a narrow set of behaviors. If a construction has been rewarded in post-training, it will reach for that construction again and again. When it is modeling the preference of the reader, it is modeling the grader in its reinforcement learning environment—which is frequently another model.

The vagaries are also examples of the models learning surface-level literary maneuvers without committing the subtext needed to round them out. They know that phrases like "something moved in him" appear in literary writing, without grasping how that phrase fits in to the larger project of describing a character's internal state to the reader. Instead, they will write "something moved in him" to emulate good literature. Exactly what moved—whether it was his conscience or a gas bubble—remains an enigma. This is because models are great hedgers. They are penalized in their reinforcement learning environments for being wrong, and so they learn to hedge. For something as subjective as literary fiction, being unspecific is less risky than attempting to be specific and getting it wrong. When the model attempts to bring some sort of symbolism to its images, it is spelled out in thuddingly literal terms:

The Blue Fighter: Instead, I stared at the fish, their silver bodies glinting dully in the candlelight as they circled their shrinking world. The dying fish had become symbols of our decline…

This is writing that tells but doesn’t show. Models, remember, are trained to be helpful assistants. It spells out the symbolism because it doesn't want the reader to become confused. It thinks it's helping!

The lack of subtextual facility may also explain the tone of the three stories. Each of them is dead-serious. There is very little humor or lightheartedness, or even tonal shift. The authors of these three stories may have asked for serious literature, which pushed the model output, once again, into a narrow band of its distribution.

Not that a short story has to be funny, but the seriousness, or rather, lack of self-awareness, is notable especially in “The Blue Fighter.” The story is about a civil engineer who plunges his family into poverty after quitting his job to become a builder of exotic home aquariums (though, somewhat confusingly, he seems to have a job again about halfway through the story, albeit one that doesn't pay him regularly). The story is told at an absolute deadpan, and the father’s motivations are never explained. Looking through my literary lens here, I wonder: why is the narrator given no space to reflect on the absurdity of the father’s new profession? Alternatively, why is no space given to comment on the father’s evident lack of self-reflection?

The irony of all of this is that “The Blue Fighter” was inspired by an unpublished poem, a fragment of which appears at the beginning of the story:

“my father was a man constantly building fish tanks.
he failed often. leaving behind
wet floors and shards of glass. Once,
he bought a carnivorous fish because it was beautiful.
the following day it had eaten all the other fish
in the tank.
my mother said nothing of her husband’s love for fish.”

The poem is evocative. The father is clearly trying to construct some kind of a life or household, but failing and leaving the dangerous detritus of failed relationships in his wake. It is a poem with subtext, but the story trades subtext for literalism. It reminds me of the viral AI video of Rumpelstiltskin. In the fairy tale, Rumpelstiltskin spins straw into gold using a spinning wheel. However, the video model that was prompted to create the animation doesn't know that context without explicit instruction, and so it renders Rumpelstiltskin himself as spinning. In "The Blue Fighter," we have a man obsessed with building fish tanks for unspecific reasons. The story tells us that the dying fish are symbols of the family's decline. It tells us that "he has always been a dreamer." We never learn why the father has jeopardized his family in this way, and we never learn what those fish tanks might mean to him. Beyond being "a dreamer," his motivations are never teased out.

“The Serpent in the Grove” and “Back and Forth” exhibit a similar self-seriousness, with very little self-reflection on the part of the characters. Almost like they are not guided by interiority, but a sort of probabilistic understanding of what literary writing looks like. Hmm…

What Is Writing?

Sometimes, as an artist, you encounter people who are eager to use your skills to realize their own ambitions. I have worked as a playwright, so the conversation usually goes something like this: they say, “Hey, I want to partner with you. I supply the ideas, and you write them down to make them into a play!” I would always politely turn down these “offers,” because the counterparty is bringing nothing to the table. The work of the artist is in the writing of the piece, determining the mechanics of how a character’s actions drive the plot forward, and not in the high-level view of what a play is “about.” Even providing a beat-by-beat outline of the story isn't the same as writing the story itself. There are innumerable decisions to be made about how characters go from plot point A to plot point B, and the one making those decisions is actually deciding what the play is about.

The same goes for prose writing. Arguments are built sentence-by-sentence, and in that construction you discover and refine what you are actually saying. I frequently use coding agents like Claude Code. I know that when I ask Claude Code to build a script for me, that doesn't make me a software engineer. I may have a high-level idea of what I want the script to do, but Claude Code is making all of the important decisions of how it is actually implemented. Likewise, when you ask ChatGPT or Claude to write a story for you, you are not making the important decisions—you are leaving them up to the model. The model, in turn, reaches for the same narrow maneuvers that it has learned to favor in post-training. Whatever you want to say remains inchoate and unsaid, subsumed into the generalities of the model. It doesn't know what's going on inside of you, and when you use it to write, you don't either. It can certainly feel like writing, but when you have the final product, you end up with the same arguments, the same prose, the same hollow writing as everybody else.