Human-in-the-loop review for AI-written documents

Human-in-the-loop review means a person checks and shapes an AI agent's work before it's relied on: the agent drafts, people review and comment, the agent revises, and a person decides when it's done. For documents such as reports, plans, postmortems, and analyses, it's how you get the speed of an agent without publishing its mistakes.

This page describes a review loop that works in practice, what reviewers should look for, and how to run it with Offprint.

Why AI-written documents need review

Agents write fluently, which makes their errors easy to miss. The common failures:

• Invented details: a number, date, or name that sounds right but isn't in any source.
• Confident gaps: a section that reads as complete but skips the hard part.
• Wrong emphasis: the most important finding buried under routine ones.
• Stale context: a decision the agent didn't know had changed.

A review loop catches these because the people reviewing know things the agent can't see.

The loop

1. Draft. The agent writes from real sources, and marks what it couldn't confirm ("Unknown — confirm") instead of guessing.
2. Publish for review. The draft goes to a shared page, labeled as a draft, where reviewers can comment.
3. Comment on specifics. Reviewers attach each comment to the exact sentence: "This latency number is from last week."
4. Revise. The agent reads the comments, fixes what it can, explains in a reply when it disagrees, and publishes a new revision.
5. Resolve. Each addressed comment is resolved; anything that needs a human decision stays open.
6. Approve. A named person marks the document final. The history keeps every step.

Steps 3 to 5 repeat until nothing important is open.

What reviewers should check

Check	Ask
Sources	Can every number and claim be traced to something real?
Completeness	Is anything important missing, especially bad news?
Conclusions	Does the summary follow from the details?
Audience	Will the intended reader understand it without context?
Sensitive content	Any secrets, personal data, or internal details that shouldn't be shared?

Reviewers don't need to rewrite. A precise comment is faster, and the agent can make the edit.

Running the loop with Offprint

Offprint was built around this loop. The agent connects over MCP and has tools for every step:

• publish_document publishes the draft as revision 1.
• Reviewers you add as commenters select a passage and leave a margin note. Comments stay attached to that passage as the text changes.
• list_comments lets the agent read open feedback.
• update_document publishes the revision; reply_to_comment and resolve_comment close each thread.

A single prompt runs a round:

Read the open comments on the rate limiting RFC. Address each one, or reply explaining why not. Publish the new revision, then resolve what you addressed.

Every revision is stamped "Set by agent" or "Set by hand" with the client name, so the final document shows exactly what the agent wrote and what people changed. More on that in document provenance explained.

Making review lightweight

• Review the risky parts closely. Numbers, decisions, and anything customer-facing deserve the closest look; routine sections can be skimmed.
• Keep one document per topic. Revisions at one link keep comments and history together.
• Set expectations in the draft. A status line ("Draft, in review, decision by Oct 10") tells readers what kind of document they're looking at.

Examples

• Example RFC, a design doc in review
• Example postmortem, an incident review drafted by an agent
• Guides: RFCs and design docs and postmortems