I keep asking AI to stop putting dashes in the middle of sentences. It agrees. It explains the preference back to me. Sometimes it apologises with the solemnity of a company announcing a data breach.

Then the dash comes back.

I wanted a writing preference respected. I appear to have acquired a punctuation subscription.

That small irritation is worth examining because the error is so easy to see. You can check a dash in a second. Checking a convincing reference, a security claim or a carefully worded promise takes rather more effort.

The repeat offender

The following is an illustration of the pattern, rather than a transcript or a measured test:

An illustrative exchange. One very small acceptance test.
The instruction
Use commas and full stops. Avoid dashes between clauses.
The reassurance
Understood. I will follow that style.
The next draft
The plan is simple — start with the basics.

Preference acknowledged. Output still needs editing.

There is nothing grammatically criminal about an em dash. Human writers use them, sometimes beautifully. A dash is also not an AI detector. Nobody should be convicted of using a chatbot because their keyboard has punctuation on it.

My objection is to the repeated construction and the way it makes otherwise different pieces sound alike. More importantly, once I have asked for a particular style, explaining that request back to me is only half the job.

The other half is doing it.

Understanding the request is only the beginning

A model can produce a convincing explanation of a rule without applying it consistently throughout its answer. From the outside, those can feel like the same capability. In practice, they need separate checks.

There is a plausible explanation for the recurring style: familiar writing patterns can compete with a local instruction. But we cannot inspect the cause of a particular dash by looking at the finished sentence. Training patterns, prompt wording and the surrounding conversation are possible influences, not a diagnosis of this instance.

Calling the model stubborn is convenient shorthand. It does not establish that the system has decided my editorial preferences are beneath it.

Likewise, an apology is another piece of generated text. However sincere it sounds, it is not evidence that the next response has passed a style check.

The apology has excellent manners. It still needs an editor.

What the research actually tested

There is useful research behind the broader concern. It needs reading with its small print attached.

Negation can be difficult, but the pink elephant is not a diagnosis

Jang, Ye and Seo tested negated prompts across nine tasks using models including GPT-3 and OPT. Their 2022 paper found that larger models in the tested families often performed worse on the negated versions. Some tasks asked for an incorrect answer rather than a correct one. This was not a punctuation study. [1]

It gives us a reason to test negative instructions carefully. It does not prove that every current model will follow “use commas” more reliably than “avoid dashes”. Trying a clear positive example is sensible; treating it as a guaranteed fix would be getting ahead of the evidence.

A conversation can complicate the job

In LLMs Get Lost In Multi-Turn Conversation, Laban and colleagues evaluated 15 models across six generation tasks. They compared complete instructions given at once with simulated conversations that revealed requirements progressively. The paper reports an average performance drop of 39% in the latter setting. [2]

That is not a 39% chance of forgetting your style guide. It is a result from a specific experimental setup. The authors describe premature assumptions and difficulty recovering from earlier answers. Their practical recommendation to consolidate requirements before retrying is relevant here, without turning the study into a universal law about chat length.

Some instructions can be checked without asking the model

The IFEval benchmark takes a refreshingly practical approach: choose instructions whose results can be checked. Its original paper describes 25 instruction types and 541 prompts, including word limits and required or forbidden words. It also discusses edge cases in automated checking. [3]

That is useful thinking for a punctuation preference. We do not need a philosophical discussion about whether the model has absorbed the rule. We can search the finished text.

The dash is cheap to catch. Other errors are not.

A style violation does not prove that a model invents facts, and these failures need not share one cause. The connection is practical: a fluent answer still needs checking against its actual requirements.

Two examples show why that check matters beyond a house style.

A citation with all the trimmings

In Mata v. Avianca, the court addressed fabricated authorities supplied through ChatGPT and submitted by lawyers. The June 2023 sanctions order imposed a US$5,000 penalty jointly on two lawyers and their firm. It also records the failure to verify the material and the subsequent defence of the false cases. [4]

The lesson is more useful than “AI makes things up”. A reference can have the right shape and still refer to nothing. Checking that it looks like a citation is a formatting review. Opening the real authority and checking what it says is verification.

Code that looks ready to ship

Pearce and colleagues generated 1,689 programs across 89 security scenarios using an early version of GitHub Copilot. Approximately 40% were vulnerable in that evaluation. The work dates to 2021 and was accepted for the 2022 IEEE Symposium on Security and Privacy. It is not a current defect rate for coding assistants. [5]

Its relevance is the distinction between producing code and establishing that the code is safe. A successful demonstration is encouraging. It does not replace security review.

Give the easy checks to software

If I care about a particular character, a search is a better use of time than asking for a more heartfelt promise.

Be precise about the rule first. An em dash is U+2014. An en dash is U+2013. The ordinary hyphen in a command, a filename or a compound word is a different character. Removing all of them indiscriminately can damage text that was correct.

For a house style that excludes en and em dashes in prose, this small Python check flags them in an exported plain text draft:

from pathlib import Path

text = Path("draft.txt").read_text(encoding="utf-8")
characters = {"\u2013": "en dash", "\u2014": "em dash"}

for number, line in enumerate(text.splitlines(), start=1):
    found = [name for char, name in characters.items() if char in line]
    if found:
        print(f"Line {number}: {', '.join(found)}")

This is a flag for review, not an automatic rewrite. Use the rendered prose or a plain text export. Searching raw HTML can miss a dash encoded as an entity. Quotations, code and deliberate examples may also need exceptions, including the example near the top of this article.

For more consequential work, choose checks that match the risk:

What an automated check can establish, and what still needs judgment
RequirementUseful checkStill needs review
Follow the house styleFlag unwanted characters and phrases.Does the writing sound natural?
Use real sourcesResolve the citation or document identifier.Does the source support this exact claim?
Produce working codeRun relevant tests and security tools.Are the design, permissions and failure modes sound?
Respect approval boundariesRequire approval before a consequential action.Is the proposed action appropriate?

No single check earns the whole document a clean bill of health. A working link can lead to an irrelevant paper. A style checker can approve a beautifully punctuated mistake.

Keep a human in the reading chair

“Human review” has to mean more than someone being nearby when the document leaves the building.

For a blog post, that might mean reading it aloud, opening the sources and checking the claims. For security guidance or professional advice, it means a reviewer with the relevant expertise and enough time to challenge the output.

Here is the routine I would actually use:

  1. Write a short brief. State the audience, purpose and style. Include one example of the voice you want.
  2. Consolidate before another attempt. If the conversation has become a trail of corrections, put the current requirements together and request a fresh draft. Check the result again.
  3. Inspect the output against the brief. The reassuring introduction is not the deliverable. The document underneath it is.
  4. Open the evidence. Check numbers, quotations and claims at their sources. A second AI opinion can help locate a problem; it does not replace the source.
  5. Use mechanical checks where they help. Character searches, link checks and appropriate tests catch repeatable mistakes without another negotiation.
  6. Make someone responsible for the final version. That person needs the authority to reject it, not just a box to tick.

People make mistakes too. The point is to combine checks that catch different things, then give the final reader a real job to do.

What this article does, and does not, establish

This is a research commentary prompted by a recurring editing frustration. The punctuation example is illustrative. We have not run a controlled comparison of models, counted a failure rate or established the internal cause of the behaviour.

The cited studies examine different tasks and model generations. Their percentages belong to those experiments, not to today's AI output as a whole. The court case demonstrates a particular verification failure, not its prevalence.

Those boundaries leave a fairly straightforward conclusion. AI can save time and produce useful work. The saving is real only after you account for the checking and editing it still needs.

Let it write the draft. Read the draft.

And if it apologises for the dash, check the apology too.

Sources and further reading

  1. Jang, Ye and Seo. Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts. Preprint, September 2022. Original research, not a punctuation benchmark.
  2. Laban, Hayashi, Zhou and Neville. LLMs Get Lost In Multi-Turn Conversation. Preprint, May 2025. Simulated conversations with progressively specified requirements.
  3. Zhou and colleagues. Instruction-Following Evaluation for Large Language Models. Preprint, November 2023. Verifiable instruction checks and their limitations.
  4. Mata v. Avianca, Inc., opinion and sanctions order. US District Court, Southern District of New York, 22 June 2023. Court decision reproduced by FindLaw.
  5. Pearce and colleagues. Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions. December 2021 revision; accepted for IEEE S&P 2022. Historical security evaluation.

Prepared 17 October 2026. Based on the author's supplied observation and a review of the cited research and court decision. No new model benchmark was conducted for this article.

Related research: I Asked ChatGPT to Stop Making AI Slop. It Couldn't. explores a similar gap between stated creative direction and generated images.