LLMs for technical editing: The good, the bad, and the ugly

The experiment techstackups.com/articles/llms-for-tech...-editing-the-good-the-bad-and-the-ugly

With the existence of Opus 4.8 and the limited re-release of Fable to the global public, you may be thinking that it’s possible to completely replace your writers and editors with AI.

It’s certainly possible, but it would also be the most inefficient, self-sabotaging decision you could make if you want people to actually care about your content and connect with your brand.

That said, I’ll admit that I have a bias – I'm an editor who finds the corporate obsession with AI counterproductive.

Nevertheless, some people are still convinced that AI can replace editors, and I'm going to show you why that isn't true. In the interests of remaining objective, I've used AI to edit an already published article seeded with errors.

By the end of this experiment, we'll be able to tell where Claude falls between two extremes: Can it replace editors entirely, or is it just fancy (and sometimes incorrect) autocomplete?

I seeded our published AX article with 23 errors of varying severity:

Error type

Count

Example

Homophones & near-homophones

4

"you can er on the side of longer docs"

Grammar

4

"How to testing your AX" (a heading)

Consistency

4

"optimise" in an otherwise US-English article

Punctuation

3

A deleted period creating a run-on

Logic

3

The article's own framework defined backwards

Typos

2

"a devv prompts the agent"

Doubled word

1

"short and and snappy"

Verbatim duplication

1

An entire paragraph pasted twice, back to back

Structure

1

A transition paragraph moved two sections too early

Then I asked both Opus 4.8 and Fable to evaluate the error-ridden article, using prompts fromour editing prompt library.

NB: I asked Claude to identify the errors first before suggesting fixes.

(I'd also recommend reading this article to see how Opus 4.8 fared on editing the clean version of the AX article.)

The Good: Holding your piece together techstackups.com/articles/llms-for-tech...-editing-the-good-the-bad-and-the-ugly

Okay, as much as I hate to say it, Claude did really well with flagging structural and logica…