The method

The one claim

The score exists to be acted on, not adjudicated. Every finding names the edit to make. That is the product.

What sets the score

One thing, measured hard: a library of 794 phrases and shapes that machine writing reaches for and people almost never do. Every one of them was tested against 10,008 real human business emails and 2,500 real human posts, and only kept if under 0.25% of real people write it.

The rule kills things that feel like slop. "In terms of" writes at 1.2% of real emails. "That said" at 1.08%. "Please find attached" at 0.54%. "At your earliest convenience" at 0.31%. All four are out, because a tool that flags what people actually write is a tool that accuses people.

Then the harder test. Each phrase being rare on its own says nothing about 794 of them running together, so we measured how many real humans trip at least one.

So the bands are not a taste call and not a percentile you have to trust. They are that table. When the result says "four stock expressions, only 0.11% of real human samples contain this many," every number in the sentence can be checked against the corpus it names.

Two other counts sit beside it, both measured the same way. Em dashes: zero of 4,077 real business emails contain one, against 9.1% of the machine written sample. Worth knowing that this corpus is plain text mail, so some of that zero is the format rather than the writer. Length: the median real business email is 90 words, and 90% are under 305.

The corpus

A percentile needs something real to be a percentile of. Each venue ranks against human writing from that venue, published before generative writing tools were widely used, so no sample can quietly be machine-written. Two venues have no corpus of their own yet. They borrow the closest one and say so on the result screen.

Academic and news were added after they broke it. The doc curve is SEC press releases, and against that curve 100% of real academic and news writing came back flagged. A student pasting a term paper was being ranked against corporate boilerplate and told they were contaminated. Both new corpora come from RAID's human rows, split in half by a stable hash of the row id so the fitted half and the held-out half can never leak into each other.

Human samples are capped at 60 per sender so a few heavy writers cannot own the curve. Quoted replies and headers are stripped, and anything under 120 words is dropped. The machine corpus is 174 samples across 13 sets, written to ordinary work briefs: decline a grant, dispute a denied claim, write an about page. The models were never asked to sound human or to beat a detector. Asking for that produces writing nobody would send, and it makes the measurement meaningless.

The corpus itself is not published. Publishing it hands every model vendor a tuning target, and the numbers stop meaning anything inside a release cycle. The method goes public instead. That is the part worth checking.

Six features, and why they no longer score

These were the backbone until 2026-08-26. They are now observations that never set the band. The sections below are kept because the reasoning is worth reading and because the findings they produce are still shown. What changed: measured against eleven other models they caught 20% of machine writing while flagging 28% of real people, and in a review of fourteen real drafts they put a good human board memo and an AI cover letter at the same percentile. A signal that cannot separate the two populations cannot be allowed to accuse anyone.

The first version of this scorer ran on word lists. It caught 3% of the 174 machine samples. A modern model doing real work does not write "delve." It writes competent prose. What it cannot help doing is writing shorter, more even sentences, with more commas, more colons, a wider vocabulary and no contractions. Those are structural, and they survive a light edit.

Separation is how far apart the two populations sit, measured as the share of combined interquartile range that does not overlap. Weight is derived from separation, clamped so nothing is free and no single feature can carry a score. None of it was set by taste. Regenerate the whole profile with node fit.mjs > profile.mjs.

The word lists still run. They sit on top as a lexical and register layer, and they earn their place on phrases nothing human writes. They are no longer the backbone.

Read the next two sections before trusting any of this. Tested outside the corpus these features were chosen on, the six of them stop separating machine writing from human writing, and in one measurement they separate it backwards.

Measured and rejected

Where the line sits

Band edges are an operating point read off the measured curve, not a taste call. Every threshold below is a real trade: catch more machines, flag more people. The shipped edge is the highlighted row.

The five bands are READS HUMAN, TRACE, ELEVATED, CONTAMINATED and TOXIC, with edges at p60, p85, p95 and p99. Moving ELEVATED from p75 to p85 gave up four points of recall to stop calling one real writer in four contaminated, which is the failure that gets a tool like this thrown out of a building.

Held-out validation

The curves are fitted on the human corpus, so scoring that same corpus proves nothing: it comes out uniform by construction. This splits the humans in half, fits on one half, and reports what the other half gets told. Run it with node validate.mjs.

A percentile scale puts 15% of humans above p85 by definition. Landing on 15.0% is the result that says the curve generalizes rather than memorizing the half it saw. Much higher would mean the thing is fitted to its own corpus and should not ship.

Tested against other models

Every number above this line was measured on machine writing from a single model family: ours. RAID (Dugan et al., ACL 2024) is a public benchmark of 2.27M generations across eleven models with matched human rows, none of it used to fit anything here. We pulled 600 machine samples and 50 human samples from the two RAID domains that map onto a venue we claim, news and academic abstracts, and scored them with the shipped scorer.

The first column is what happens when the curve does not match the writing. Flagging every real person in the sample is the worst outcome this tool has, and giving those two registers their own human corpora fixed it.

The second column is the honest one. With matched corpora the scorer catches 20% of machine writing and flags 28% of real people. Real people are flagged more often than machines are caught, so in academic and news writing these features do not separate the two populations in either direction. The result screen says so in plain words on those venues rather than letting a number imply otherwise.

The reason is the corpus, not the text. The doc curve is fitted on SEC 8-K press releases, whose median sentence runs 31 words with a vocabulary spread of 72. Human academic and news prose runs 21.5-word sentences at a spread of 83, which our curve reads as machine-like. The same 50 human samples score a median p97 against doc, p76 against social and p53 against email. The benchmark chosen, not the writing, is producing the answer.

Refitting the doc curve on RAID's own human writing inverts the signal outright. Machine text lands at a median of p16 and human text at p56, and all eleven generators sit below the humans. What the six features actually measure is a register: dense, even, formal, contraction-free prose. That register describes our own corpus. It also describes an arXiv abstract and a wire story, which is why real writers in those domains score as slop.

Reproduce it with python fetch_ai_bench.py then node bench.mjs. RAID is free and needs no key.

Email is not in this test. RAID has no email domain, so the venue this tool is built around is also the one with no independent evidence behind it. Every email number here comes from writing by a single model family scored against Enron. That gap is real and it is not closed.

Scoring against yourself

Every failure on this page has the same shape: the curve did not match the writer. A shared corpus is a population of strangers, and when you sit far from that population the number is about the corpus rather than about you. A personal baseline removes the mismatch instead of managing it. You supply your own writing, we fit a curve from it, and a draft is ranked against how you normally write.

It is the same percentile machinery with one author instead of thousands. No model runs, nothing is stored, and the profile stays in your browser.

How much writing it takes is measured, not guessed. The Enron corpus is organized per sender, which is the exact shape of a personal corpus, so we refit 40 real senders at several sample counts and compared each personal curve against that sender's full curve.

Read the last row first. At ten samples a personal curve is worse than using a stranger's writing, so a profile that thin is noise wearing your name and the tool refuses to build one. Thirty is the first point where your own writing clearly beats a stranger's. Fifty is where the band label steadies. The error is still falling at sixty, so sixty is where the measurement stopped, not where it converges.

Reproduce it with node min_samples.mjs.

What this does not do