How to review an AI-written spec:a checklist before the agent builds.

Why AI specs need a different kind of review

A person drafting a spec tends to leave gaps where they are unsure. A model writes fluent text for everything, including the parts it has guessed. So the usual signs of a weak spec, vague wording and obvious holes, are missing, and the problems are hidden in confident sentences. Review for the things a good author would have asked about.

The checklist

1. Scope: does it solve what you asked?

   Compare the first paragraph with your request. Look for solved-a-different-problem and for features nobody asked for.

2. Non-goals: what will it not do?

   A spec with no non-goals has no edge. Add the extras you do not want.

3. Assumptions: what did it guess?

   Look for "presumably", "likely" and anything stated as fact about code or data that it may not have read. Ask it to cite the files.

4. Edge cases: what happens when it goes wrong?

   Empty input, very large input, no permission, no network, two users at once, the same action twice.

5. Data: what is stored, changed or deleted?

   Fields, formats, who can read them, and what happens to existing records.

6. Migrations: how does the old data move?

   A schema change needs an order of steps, a plan for a failure half way, and a way to run it twice safely.

7. Interfaces: what else is touched?

   APIs, screens, jobs, other teams' code. Anything the spec changes but does not list is a surprise later.

8. Security and privacy: who can do what?

   New permissions, new data leaving the system, secrets in logs.

9. Tests: how will you know it works?

   Each requirement should map to a check. If a requirement cannot be tested, rewrite it.

10. Rollout and rollback: how do you undo it?

    A flag, a staged release, a way back. "We will fix forward" is not a plan.

11. Open questions: who decides them?

    An honest spec lists what it does not know. If there are none, be suspicious.

A comment that works

Vague feedback gets a vague rewrite. A good review comment points at one passage, says what is wrong and what you want instead:

> The export runs when the user taps Export.

Missing: what happens with a 2 GB file? Add a limit (R4) and an
error message, and say whether the export can be cancelled.

Keep the comments separate from the spec text so the file stays clean, and batch them: read the whole spec once, then comment, so the agent rewrites once rather than ten times.

The review loop

1. Read it end to end first

   Before commenting, make sure you understand the whole plan.

2. Comment on the exact passages

   One issue per comment.

3. Let the agent revise

   It fixes each comment and produces a new version.

4. Check what changed, not the whole document

   Read the differences. A fix can break something that was fine.

5. Approve only when the checklist is clean

   Then let it build, against the numbered requirements.

The same checklist works when one agent reviews another's spec. Ask the reviewer to answer each item with a quote from the spec.

Doing it in DotMD

You can do all of this on the page. In DotMD you select the passage and write a comment on it; the comments are kept in DotMD, not in the file. The agent fixes them and sends the document again, and reviewing the changes shows the old text in red and the new in green so you can accept them one by one. The setup is described in review AI specs. Reading is free; see pricing for reviews.

Common mistakes

• Reviewing the style, not the substance. A neat spec can still be wrong.
• Approving because it is long. Length is not coverage.
• Skipping the non-goals and rollback, the two sections models most often leave thin.
• Fixing the code instead of the spec. If the plan is wrong, correct it first.
• No second look. After the agent revises, check the diff.

Questions about reviewing AI-written specs

More questions