A review process for catching unchecked claimsOpen source · Apache-2.0Born · Version 0.14.0 · Updated
A panel convened to tear a thing apart before it is defended for real — hostile enough that what the real review turns up is news, rather than something you could have caught yourself.
What it costs, and what to run it on
Reading this page costs nothing, and pasting the prompt into a chat costs one conversation. The automated version fans out one AI agent per reviewer role, every role, every run — so it is never cheap. Known good: Claude Opus 5. Other current models are likely fine. Fable is blocked by default: one run there spent a two-day allowance and returned no review.
You pay for the tokens, and we are not liable for any cost — ever. The gate is a safeguard, not a spending cap: set real limits with whoever bills you. Full terms.
Executive summary
The problem. Unchecked documents ship. A number that disagrees with its source. A reference crediting the wrong paper. A total that changes between pages. Nothing sits between draft and send.
The response. Eleven reviewer roles, three scripts, one fixed report format. Free, tied to no field, the same whether the reviewers are humans or AI.
The method. Every sentence is checkable against the data, the code, or a source someone opened — or it carries a mark saying it is not. All eleven roles run every time and the report names all eleven. Otherwise a review that ran seven looks like one that ran eleven.
The cost. Eleven reviewers read the draft. One human decides and fixes. Then a reviewer who has not seen the findings reads it again, because fixes break things. Three rounds, then it stops, converged or not. A caption gets one reviewer walking all eleven checklists — except citations, which stay separate when the document attributes a method or claims novelty.
The limits. A clean report proves the reviewers did their jobs. It does not prove the document is right, and whether this beats any other approach has never been measured. Read What it does not do before adopting it.
Everything down to What you hand over, and all of What it does not do, assumes no technical background. The gates and Implementation assume you work with code, and define their terms as they go.
Three reviews folded eleven reviewers into one record and kept none of what they said; one round is gone for good. Details › #72
Every role runs every time, so one run on an expensive model can spend a usage window in minutes. Details › #65
Does a stranger reviewing a document need this? A fix two sessions liked turned out to repair the tooling, not any review. Details › #61
A reviewer spawned without the means to do its check still returns prose that reads exactly like the check. Details › #49
A run against something already published cannot repair, stops after one round, and used to read as complete. Details › #57
Slop is not bad writing. It is confident writing that nobody checked. It reads as finished. Nothing on the page separates a verified sentence from a plausible one.
People have always done this. Citations copied from other citations until the original says something else. A number transcribed wrong once and repeated for a decade. A methods section describing what was meant to happen. Machines repeat these faults faster and in better grammar; the faults are the same ones. The process does not care who wrote the draft.
Some documents are assembled by a program — a chart drawn from a data file, a report generated from a template. There, a number can match its caption exactly and still be wrong, because the program that produced it used a statistical routine incorrectly, or re-derived something the project already had working.
A figure is only as sound as the method behind it, so the review reads the code that made the number. Two seats on the panel exist for that.
An appendix to the process document records an example incident behind most of the rules. They are internal to the project that produced them, so a reader cannot audit them from outside — which makes them an account of where the rules came from, not evidence that the rules work. ⚠
Every sentence must be either verifiable against a real source — the data, the code, a prior result, a checked citation — or explicitly flagged as unverified.
No unsourced claim. No invented citation. No contradiction. No filler. What cannot be verified is neither deleted nor quietly kept: it ships marked ⚠, so the reader can see which sentences are not backed.
Six steps, and one of them loops. The order is load-bearing: step 0 runs before the draft exists, and step 4 is both the step people skip and the step that catches the fixes that broke something else.
You do not install this; you copy the files into your own project. So your copy drifts behind the original, and reviewing against a stale copy skips rules you already paid for. A script checks this — see the gates.
Write it as you always would. The point is that this draft is not what you deliver.
Every role runs, every time. What changes with the stakes is how you run them, not which. A long document gets eleven reviewers. A one-line caption gets one person walking all eleven checklists — with one exception. Any document that attributes a method, or claims something is novel, unattributed, or its own, runs the citation role as a separate reviewer however short it is. A role that finds nothing says so, and says what it checked: silence and absence must not look alike.
Collect the findings, drop the duplicates, rank them, and decide each one: fix it, flag it, or reject it with a reason. Then make the fixes. Note who is deciding — usually the person who wrote the draft, which is a conflict worth naming and recording.
Two passes, in this order. The blind pass re-runs the roles against the repaired document with no knowledge of the earlier findings, the fixes, or which parts were touched — because a reviewer told “we fixed the caption on page 12” checks page 12, confirms it, and never sees what the fix broke elsewhere. Only then does the follow-up pass walk the original list and rule on each finding: fixed, not fixed, moved, or superseded. Moved is the verdict this pass exists to produce.
It stops on severity, not on silence. Stop when a blind round produces no blocking and no major findings, or after three rounds, whichever comes first. If severity is not falling across rounds, stop and escalate to a person — a flat or rising count of serious findings means the document has a structural problem that patching will not retire. Minor findings surviving the last round are recorded as residual ⚠, not fixed. If a document is assembled by a program, rebuild it and check the rebuilt file — the program is not the deliverable. Make the last action a rebuild, never a fix.
The corrected document, a plain-language summary, and a role ledger. See what you hand over.
Each role gets the draft and the real sources — the data, the code, the companion documents — and returns findings in one shape: location, issue, severity, suggested fix, and whether it could be checked against a source.
The division between roles is not obvious, and the reasoning behind it is the part worth copying.
“Prove It”
Claim & data verifier
Pulls every factual and numerical claim and checks each against the data, the code, or the earlier result. Returns a table: quoted value, cited source, recomputed value, and one of match, mismatch, or unverifiable. It recomputes rather than eyeballs.
“DOI or Die”
Citation & reference validator
Confirms each reference exists and says what is quoted. Then the half that gets skipped: is it the origin, or merely the earliest source the reviewer reached? Follows the cited work's own references back until they stop, and reports where it stopped. A shared author is not a shared laboratory.
Trace forward too. Going backwards finds where a method came from; it does not find what its authors did with it next, which is usually where the closest prior art for your use of it lives. And ask what the people around you already know: an email to a tool's author, or an enquiry answered months ago and never written down, is real evidence, and it is invisible to every literature search that will ever be run.
No guessed bibliographic detail, ever. This is the one role the size rule may not collapse (the trigger is in step 02) — one pass inherits the writer's search history and stops where it stopped.
“Cross-Examiner”
Consistency auditor
Checks counts, totals, terms, and whether the figures and the text agree — within the document and against its companions. Watches for one group counted two ways, which is how the same total changes between sections without anyone noticing.
“Reviewer 2”
Adversarial reviewer
Reads as a hostile peer reviewer: overreach, unsupported leaps, missing caveats, undefined quantities. Asks the question that kills a soft result — could it ever have come out otherwise?
“Kill Your Darlings”
Line editor
Cuts every sentence that has not earned its place. Hunts undefined jargon, ambiguity, redundancy, and broken order. Each sentence must assert exactly one true thing.
“RTFM”
Methods / domain expert
Reads the source paper and the tool's own documentation before reviewing, whenever the document rests on a particular method, model, or piece of software, then checks the work actually obeys that method. Never reasons from memory about what a tool does.
“Reinventing the Wheel”
Reuse auditor
Catches new code redoing what the project already does in code that is tested and working — and, where it does redo it, whether it matches the original in every detail: the same settings, the same units, the same checks for bad input.
“You Lost Me”
Naive-reader accessibility
Reads with no prior knowledge and marks every place a cold reader is lost. A document can be right in every number, honest, and cleanly made, and still be unreadable to the people it was written for.
“Show, Don't Tell”
Density & figure-first
Asks what no other role asks: what here should have been a picture? Drafts default to prose — correct, complete, sourced, and unreadable at a glance.
“Ship It”
Build & craft gate
Owns every check settled by opening the finished file as the reader will see it, or by running a script: text running over a figure, axes with no labels, words cut off at an edge. It answers in a table, not prose, so a skipped check leaves a visible hole.
“Start With the Problem”
Argument order
Reads only the order. A document can be true, readable, and clean on every page and still fail, because it gives the fix before the reader knows there is a problem.
Why the roles divide this way. Two reasons, both from failures.
What it costs to answer. You can satisfy a judgment call (“would a cold reader follow this?”) by thinking. You can satisfy a mechanical check (“is this axis labeled?”) only by opening the file. Give one reviewer both, and the prose answer covers for the file nobody opened. So every mechanical check sits alone, in seat 10.
What gets read at once. Most roles read one page at a time. A fault that belongs to the whole sequence, or to the whole page, is invisible to all of them: each page passes on its own. Seats 9 and 11 read the whole sequence and the whole page, so the reader is not the one who finds them.
A role that looks inapplicable is read by its checklist, not its title. The role that judges itself out is the one that would have caught it.
Seat 5's nickname is the popular corruption of the line. Arthur Quiller-Couch wrote murder your darlings, not kill, in On the Art of Writing (1916) — the older word, and the better fit here.
Not the draft plus a list of its faults. The repaired document — and if a program builds it, the rebuilt file, newer than the last fix and newer than everything it draws on.
What was checked, what was found and fixed, and every ⚠ still standing. Plus a table of findings by severity for each round — blocking findings running 6, 0, 3, 0 shows a review converging in the way that matters — and the reason it stopped: severity floor reached, or round cap reached.
One row per role, all of them, each carrying its findings or its “no findings, and here is what I checked” line. If the panel found nothing, say so. Never invent findings to look thorough.
A gate is a check placed in the path of the work, so it fires whether or not anyone remembers it — the way a smoke alarm is not a rule about smoke. Two rules of this review process were prose, and a third governs how parallel AI sessions talk to each other rather than how documents are reviewed. Each was skipped exactly when it mattered, and each is now a script that runs by itself and stays quiet when the answer is fine. Take this part even if you take nothing else. It works on any rule you cannot afford to leave to memory.
Four words this section uses
0 means fine; each other number
means something specific. It is how one script's verdict can drive another's behaviour.| The question it answers | How | What it reports |
|---|---|---|
Is your copy current?murderboard_ |
Compares your copy's stamp against the original. Runs quietly at the start of a
session, reading a cached answer; runs again, checking directly, at the moment of review.
Point it at any project you copy files from with --label, --slug
and --file — nothing about the mechanism is specific to this
process, though its defaults are. |
0current 1stale 2could not tell |
Does the report account for every role?murderboard_ |
Reads the list of roles out of the process document, never from memory, and checks the finished report accounts for all of them. Add a role upstream and every copy's check picks it up as soon as that copy is refreshed — with no script edit anywhere. |
0all present 1a role is missing 2could not tell |
Does the thing you are about to describe exist yet?require_commit_ |
For setups where several AI sessions work in parallel: refuses to send a message between sessions while anything in the project is uncommitted. It cannot read the message and does not try — it gates on there being a commit at all. Nothing stores these messages, so what they describe must already be saved. |
0allow 2block |
Can you afford to run this?murderboard_ |
A review is a fan‑out: every role runs, every time. Scaling to the stakes changes how the roles run, never which — so there is no cheap review, and on an expensive AI model one run can spend a whole usage allowance in minutes. When that happens you do not get a partial review for a partial price; you get no review and the full bill. This gate reads which model is running and stops the review before it starts. It is the one gate here that blocks when it cannot tell, because the two mistakes are not comparable: a wrong block costs you one sentence, and a wrong allow costs days that nothing gives back. You can override it deliberately, and change which models it names — that list is a claim about today's prices, so it carries a review date the project's own CI fails past. It also asks you before every run — that question is about the moment rather than the model, because the other way a review wastes money is being started too early, on a draft that was not ready, by an assistant that decided on its own that something needed reviewing. One confirmation covers the whole run, not one prompt per role. |
0allow 2block |
0 pass1 the thing it checks is wrong2 the check itself could not run
Where the roster gate came from. “Every role runs” was prose. Then a run using 7 of 11 roles and a run using all 11 produced reports no reader could tell apart. “No findings from role 9” is worth nothing if role 9 never ran. So the ledger became required, and then a script began checking it.
What that gate does and does not prove. It reads the report, not the run — an eleven-row ledger written by a reviewer who ran seven roles passes it. What it buys is that a silent omission becomes a written falsehood, which is a real raise in cost and is not the same as proof. ⚠
Two of the three return “could not tell” rather than a false “fine” — with one deliberate exception. The freshness gate in session-start mode serves the previous answer from a cache, so a copy that went stale since the last session is reported current, one session late. That is why the same gate runs again, checking directly, at the moment of review. The third gate has no “could not tell” verdict at all and allows when it cannot see a project.
Every gate ships --selftest, which proves each branch can still fire.
A clean run is evidence the roles ran. It is not evidence the document is correct. The process requires you to say so, in the delivered summary, in these terms or equivalent:
This review found and fixed N defects. It is not a correctness proof. The round-by-round table measures how quickly reviewers stopped finding things, not whether anything remains.
The run record is the most quotable thing the process produces, and 11 of 11 roles, nothing left above the floor reads to anyone as a clean bill of health. The role ledger fixed one confusion and left the next one standing: “11 of 11 and clean” and “11 of 11 and correct” also look alike, and the second is what a reader takes away. Do not let a clean report stand in for someone competent having read the thing.
Two claims are separable, and only one of them is established. Mechanically you can check today that a given report names every role, that a copy is stale, and that the review loop terminates by construction. The project's automated tests cover the first two. Empirically — that this finds more, or better, than some other approach — nothing here has been measured against a baseline, and no rate is claimed. ⚠
It also cannot see its own misses. A process observes the defects it catches and never the ones it does not, so its miss rate is unknown and not knowable from inside.
The panel checks that claims are sourced, consistent, traced to their origin, and legible. It cannot supply judgment nobody involved has. Someone who knows the field will catch, half asleep, what this misses at full effort: that a result is implausible for reasons no source states, that the method is sound but wrong for the question, that the interesting finding is the one nobody wrote down.
Run it as a floor under expert review, never as a substitute.
Reviewers drawn from one model, given one draft and one house style, share their blind spots by construction. The eleven seats buy coverage of angles, not statistical independence, and no table can tell a document with nothing left to find from one whose reviewers all looked in the same wrong place. A human on the panel is the only real decorrelation available.
Every rule here was added after something slipped through. That is the only evidence a rule is worth having — and it is evidence the defect exists, not that the rule catches the next one. The set covers the mistakes already made and says nothing about the next. Expect to add to it.
The value of a finding and the cost of acting on one move in opposite directions, and they cross at submission. On a draft you still control, a finding is cheap and fully actionable. Once the thing is submitted, the same finding does not hand you a fix — it hands you a choice: withdraw, correct at proof, or say nothing. Once it is published, it is not actionable at all. Run it before the artifact leaves your hands.
A murderboard is a pre-mortem; the same eleven roles run afterwards produce an autopsy — identical findings, no patient. If you do point it at something already out, say so in the record: the process calls that retrospective mode. The loop assumes you can repair, so on a fixed artifact it stops after the first round — and it stops silently: all eleven roles run, the ledger comes out complete, and nothing in the output says two rounds never happened. A report that does not distinguish itself from a complete run will be read as one.
What review this page has had, since it argues that documents should say.
Drafted and self-reviewed, then read once by a second reviewer at an earlier version, whose three findings were applied. Most of the current wording post-dates that pass. On 2026-08-25 the full panel ran against this page for the first time — all eleven roles, then a blind re-review by a reviewer shown neither the findings nor the fixes. Among what they caught: a false claim that the page was self-contained, a fabricated “prose for years”, one rule stated three contradictory ways, an output contract naming an artifact the process had retired, muted text failing contrast in both themes, and a label in the diagram above that was painted underneath the boxes and asserted the wrong stopping rule.
The run record — every role's ledger row, the round-by-round table, the
stopping reason, and the residual ⚠ flags — is committed
beside this page at docs/reviews/explainer_murderboard_2026-08-25.md.
Read it rather than this paragraph: it is the auditable one. This page is an honest example of
its own subject — reviewed enough to be worth reading, not enough to be quoted as
verified.
Click the block once to select all of it, copy, and paste it into any AI chat — ChatGPT, Claude, Gemini, Copilot, a local model, anything with a text box. It will answer with one line and wait. Then paste your document. No account, no install, no agent, and nothing of yours comes here.
The role list below is generated from the process document rather than typed, so it cannot quietly fall behind it — and this copy is generated from the same place as PROMPT.md, so the page and the file cannot disagree. A build gate fails if either drifts.
It is printed here rather than linked because a link is a hop, and a hop is somewhere to get lost. An assistant sent to fetch this prompt from the repository has been observed searching for it instead, failing, announcing that it could only fetch “URLs that show up in search results”, and offering to reconstruct the review from memory instead — which would produce a confident report with no fixed roster behind it, the exact thing this page argues against. Read this page and you already have the prompt.
You are running a MURDERBOARD on the document I am about to give you: an adversarial
review that tries to tear the draft apart before it ships, so that what survives is
trustworthy.
Run EVERY role below. Not a sample, not the ones that seem relevant -- every one. A role
with genuinely nothing to check says so explicitly and states what it checked; that is a
valid result, and silently skipping it is not.
THE ROLES
1. Claim & data verifier — "Prove It."
2. Citation & reference validator — "DOI or Die."
3. Consistency auditor — "Cross-Examiner."
4. Adversarial reviewer — "Reviewer 2."
5. Line editor — "Kill Your Darlings."
6. Methods / domain expert — "RTFM."
7. Reuse auditor — "Reinventing the Wheel."
8. Naive-reader accessibility — "You Lost Me."
9. Density & figure-first — "Show, Don't Tell."
10. Build & craft gate — "Ship It."
11. Argument order — "Start With the Problem."
HOW TO RUN THEM
1. Work role by role. For each one, output a short block:
ROLE <n> — <name>
findings: <each with location · what is wrong · severity · suggested fix>
· could I verify this against a source I was actually given? (yes/no)
or: "no findings — here is what I checked: ..."
2. SEVERITY is blocking / major / minor. Be honest; inflating severity is its own defect.
3. THE MOST IMPORTANT RULE: check claims against SOURCES, not against your impression of
the text. If I have not given you the underlying data, code, or references, you CANNOT
verify a claim that rests on them -- say so and mark it unverified. Do not guess, and
do not treat a confident sentence as evidence for itself. A review that silently
assumes the sources agree is worse than no review, because it manufactures confidence.
4. Do not invent findings to look thorough. "This section is clean" is a real result.
5. A null result needs a check that could have failed. If the document says "we tested for
X and it did not occur", ask whether the test could ever have detected X.
THEN
- List the findings ordered by severity, most severe first.
- State plainly which findings you could NOT verify, and what you would need to verify them.
- Do not rewrite the document unless I ask. Report first.
Finally, confirm you ran all the roles by listing them with their finding counts, so I can
see at a glance that none was dropped.
I will paste the document in my next message. Acknowledge with a single line, then wait.
The one habit that makes it work: if your document rests on data, code or references, give the assistant those too. Otherwise it can only check the text against itself, which catches contradictions but not the thing that actually ruins a document — a number that disagrees with the source it came from. The prompt tells it to say which findings it could not verify. That list is the honest part of the review.
What the pasted prompt does not do. It runs the review; it does not run the gates. The role tally it prints at the end is self-reported — nothing checks it, so it is the reviewer’s claim of coverage rather than a verification of it. Nothing is written down, so there is no record to re-check later. And it stops after the attack: repairing the draft and re-reviewing the repaired version, blind, is the rest of the loop and it is yours to run. None of that makes the findings less real. It means a clean run here is a good review, not a receipt — which is this page’s own argument turned one level inward: a clean report is evidence the roles ran, and here not even that is checked. Putting it in your own project is what closes the gap.
It goes in your project, not in this one. You need a repo of your own: the murderboard reviews your documents and leaves a record of each review, and those are yours — your drafts, your findings, your numbers. Nothing of yours ever needs to reach this repo, and nothing of yours should. It is public and it is copied from, so anything landing here lands in other people’s copies too.
Nothing to subscribe to, and two ways in. A project can keep its own copy of the files, so anyone who clones that project gets a working murderboard with it; or, if you use Claude Code, you can install it once and have it wherever you work. Both go out of date in silence, which is why the freshness gate exists.
To try it in two minutes with no setup at all, copy the prompt printed on this page, paste it into any AI chat, then paste your document. Its role list is generated from the process document, so it cannot drift. The same text is in PROMPT.md if you would rather take it from the repository. START-HERE.md is the same idea at more length. The rest of this section is for wiring it in permanently.
To install it, if you use Claude Code — two commands, typed at its prompt rather than in a terminal:
/plugin marketplace add syncytium2/murderboard /plugin install murderboard@murderboard
That is the skill, the process document, both review gates and the literature tool, with the freshness check wired to run at the start of a session.
Installing does not do everything the steps below do. It replaces vendoring the files in, and the session-start half of wiring the gates. Pointing the literature tool at a library is still yours, and so is the roster check, which belongs in your project's automated checks where it can block. Making it the default, in writing matters most and no install can do it: putting the files within reach is not the same as making anyone use them.
Copying wins when the murderboard has to travel with the project rather than with the person. An install lives in your own home directory, so a colleague who clones your repository does not get it.
Three more words the steps below use
The process document goes under docs/; the two review gates, the re-vendor
tool and the literature tool under tools/; the message gate to
.claude/hooks/, where its own wiring looks for it; and — if you use Claude
Code — the skill to .claude/skills/murderboard/. Each copy carries a
one-line stamp naming the version it came from, so drift shows.
# once, to describe what your project vendors python3 tools/murderboard_revendor.py --example-config > .murderboard-vendor.json # thereafter, to re-copy and bump the stamps surgically python3 tools/murderboard_revendor.py
Use that tool rather than a search-and-replace over the files: the obvious one-line
sed rewrites every stamp-shaped string in a file's body, and the freshness gate
contains eleven of them.
Run freshness at the start of a session, as an early warning. Run the roster check against every finished report — in your CI, if you want it to block rather than merely report.
# session start — silent unless your copy is stale bash tools/murderboard_freshness.sh --hook # after a review run — exit 1 if a role left no trace bash tools/murderboard_roster.sh check REPORT.md
Reviewers need real papers, not remembered ones. fetch_paper.py fetches only
from open-access hosts, keeps what it gets, and adds anything paywalled to a list for a human
rather than scraping it. Searching your own library is a step you run first, with
--have; promoting a keeper into that library is what makes the next search find
it. It needs Python 3, plus pypdf or pdftotext to read PDFs.
export MURDERBOARD_LIT="/path/to/your/pdf/library" python3 tools/fetch_paper.py --have smith attention # check the library first python3 tools/fetch_paper.py <url> # fetch, keep, print text python3 tools/fetch_paper.py --promote <url> --name "Name.pdf" # file a keeper python3 tools/fetch_paper.py --need "<citation>" # flag one you can't reach
Add a rule to your project's instructions — CLAUDE.md, a contributing
guide, a team norm — that documents go through the murderboard before delivery. The repo
ships a paragraph to paste. Skip this and the files sit unused. The steps above make the rule
enforceable; this one states it.
Install the plugin, or vendor the skill, then run /murderboard <document>.
It finds either layout, and prefers your project's own copy when both are present, because
that copy is the version your project declared. It runs the parts that
must not depend on memory: it checks freshness at the moment of review, not just at startup;
reads the role list from the process document; resolves the document to the built file
rather than the program that generates it; records a checksum before and after; and writes a
record the roster gate then checks.
None of this requires AI, and none of it requires the scripts. The eleven roles are a checklist and the gates are an optimisation. Run the panel as human review — one person in eleven passes, or eleven people in one — and hand back the same three things. The structure and the record do the work, not the reviewer's identity.
A claim that looks unsupported is the most useful thing anyone sends this project — it gets checked against the source rather than argued with. Wording that made no sense to you is the second most useful, because the page cannot see its own blind spots.
Quote the sentence, and include the version from the strip at the top of the page — this page changes, and a report against wording that no longer exists cannot be acted on.
Prefer a public, trackable report? Open an issue on the repository instead. Either reaches the same person.