How I Make Things

A response to the Pangram rollout, plus a few changes to the newsletter

A few days ago, Substack introduced its Pangram AI Detector integration to thunderous applause widespread anger and confusion. They’ve walked back a few details of this implementation based on feedback, but in order to give context to later sections of this post, these were my chief complaints.

It was applied too broadly, too suddenly. Normally, when a customer signs up for Pangram, they do so alongside an internal policy governing its use. A school using it as a plagiarism detector will have guidance on when to run it and how to handle potential false positives. An academic journal might include additional peer review steps when the AI detector triggers an alert. News or review aggregators might not care at all.

Each customer’s needs will vary, and those needs will be discussed before they sign a contract and start running Pangram on everything. Applying one policy to every single Substack page overnight caught a lot of people flat-footed, including me.

There was (initially) no opt-out procedure. In order to disable AI detection, you had to run AI detection on your page. I understand Substack’s stance with this, but it had the same “you’ll eat shit and you’ll like it” mentality that seems to govern the modern tech world, especially the AI space. This rhymed too closely with the ubiquitous data scrapers that already run rampant across the Internet, and the Substack team should have expected the PTSD response that followed.

This has since been rolled back, which I think is a net good. You can now disable AI detection before hitting publish, which gives creators more time to review Pangram’s Data Privacy policy and ask questions about the agreement between Substack and Pangram.

Personally, I think Pangram derives a lot of market value from not using scanned data for training purposes, and breaching that trust would cause them to lose customers elsewhere (you lied to Substack, how do we know you’re not lying to us?). They also have very good reasons to prevent people from contaminating their proprietary training sets. Call me naive, but I think Pangram probably isn’t using Substack to scrape data, and is actually just using Substack as an easy revenue stream, but time will tell.

In the meantime, guinea pig signups are on a volunteer basis.

It’s mostly accurate, and “mostly accurate” isn’t good enough. It has higher false positives for second-language English speakers, for highly polished academic texts, for poetry, and for seemingly random paragraphs that happen to include a slightly higher percentage of AI-isms than the average bear.

Pangram knows this. They publish statistics on their own false positive rate, which changes depending on the type of text being scanned. They offer guidelines for its use, and even recommend not trusting its output on certain types of documents.

But the overwhelming majority of people who use the tool aren’t going to read any of that shit. They’re just going to click the button and say “Ah-ha!”

How I Plan on Using Pangram

Short answer:

I won’t.

Long answer:

I’d like to give you a brief glimpse into what goes on behind the scenes when I write. Let’s use a hypothetical scene where a character is nervous about walking down a dimly lit hallway in a hotel. I might start with something like this:

Ted glanced at his room key. Room 419, all the way at the end. Past the broken down ice machine, past the flickering hallway light, past the blotchy stain on the carpet that probably wasn’t blood, but who could say for sure?

Personally, I think that’s a pretty good starting point. The second sentence feels a bit awkward. I can’t decide if it would be better longer, or if I should replace the comma with a colon, or if I should just delete the second clause entirely. (Room 419. Past the broken…)

More importantly, it doesn’t introduce enough tension. The “probably wasn’t blood” line is good, but it’s too cutesy, there aren’t any real knives hidden in it. Bad things are coming, and I want the reader to feel nervous about them. Let’s add a few more descriptions so that the paragraph reads slower.

Ted checked his room key. Room 419. The sign told him it sat at the end of the hall, past the broken down ice machine, past the flickering hallway light, past the blotchy stain that seemed to weigh down the carpet like a sin. Doors stood to either side like sentinels in faded armor. Tarnished handles, dented kickplates, cobwebs in every corner. They noted his passage as he shuffled between them.

Better? I can’t decide.

I’ve added words, but I’ve also added a whole host of cliches. This might work for a pulpy noir, but that isn’t the atmosphere I’m going for. I don’t want people distracted by visual details that add flavor but no stakes. Let’s try again.

Ted stared down the length of the hallway, feeling like an intruder. The last hotel he’d stayed at was grubbier than this one. Busted lights, thin walls, worn carpet, all the usual sins. But it hadn’t made him feel like this.

It was something about the doors, he thought. Most hotel doors were just another part of the wall. They were there to keep people out, but once the locks and bolts and chains had all settled into place, they went to sleep. No one on either side of the door had to think about anyone on the other side. Everyone was safe from one another.

These doors turned to look at him as soon as he stepped off the elevator. He felt a need to justify his presence here, to show his papers to the men with the guns. He fished out his room key and cursed as it twisted between his fingers and fell to the carpet. As he bent to pick it up, he stopped, suddenly embarrassed, because he knew someone had seen him drop it.

Behind him, the elevator closed its doors, retreated to the safety of the lobby.

I like this a lot better. I’ve ditched a few of the visual descriptions to focus on what’s important: the feelings of isolation, exposure, and menace, now represented by the doors. It no longer matters whether the hallway is bright or dark. Ted has got to get the fuck out of there. Even the elevator knows better.

Back to the point at hand, what would Pangram tell me about this scene that I don’t already know? I know it was written by a human. More importantly, I know that it avoids obvious cliches, it builds tension, and it puts the reader firmly into Ted’s perspective. These are the hallmarks of half-decent writing, and they are the only standard I need to concern myself with.

Gen AI may eventually steal some of my tricks, like replacing a conjunction with a comma between two clauses that share a subject (the elevator closed its doors, retreated). If it does, my style would likely get flagged as partially written by AI.

Fine. I like it, I’m not changing it, and I’d rather not know about it.

I also won’t be using it to examine other writers. I can already tell who’s using AI and who isn’t. It isn’t as hard as people seem to think.

What I Plan on Doing Instead

Pangram’s entire business model rests on one simple idea. When it comes to certain goods, people are willing to pay a little extra for a certificate of authenticity. This was true for baseball cards, and it seems to hold true for novels, academic essays, movie reviews, and a few other forms of text.

To this, I would add two considerations:

1. I’m better at establishing trust with my audience than Pangram is.

2. If I’m the one adding value to the product, I should be the one getting paid for it.

Going forward, I’ll be adding a new section to the newsletter called Under the Influence. This will be a mix of biographical details, writing advice, and a behind the scenes look at my writing process from the original inspiration to the final draft. These posts are intended to give you a closer look at the human intelligence that goes into writing my stories, and should hopefully clear up any doubt about how the work gets done.

I’m still deciding whether these will be included with the stories themselves, written as standalone posts, or a mix of both. What I do know is that the first three will be free to read. After that, they’ll go behind a paywall. The stories will continue to be free, as they always have been.

(For anyone who thinks this is a cash grab, I donate my Substack proceeds to charities to promote literacy, and I actually lose money on paid subscribers).

I’ll also be disabling the ability to scan my posts with Pangram. I’ll earn your trust or I won’t. I’m not outsourcing that process to an AI tool.

---