A text detector — Pangram, GPTZero, the classifier a teacher runs on an essay — is trying to answer exactly one question, and it is the oldest question in this book worn in this century’s clothes: is this a read reference or a created one? Was the text there before the asking — written by a person grounded in something outside the loop — or was it manufactured by a loop, emitted by an engagement-driven language model? The detector is the Masoretic checksum’s estranged cousin: both are provenance instruments. But where the Masorete bound the channel — froze the word and counted its letters so a copy’s origin could be verified arithmetically — the detector does the opposite. It keeps no checksum, sees no channel, and tries to recover provenance from the surface of the output alone. That difference is the whole story, and it is why the detector has a floor the checksum never had.
What surface forensics can and cannot reach
A text detector is a function of surface statistics — token distributions, perplexity, the fingerprints a generator leaves in its phrasing. So it can only recover whatever the source writes into the surface. Where the generator’s output is heavily marked by its origin (long, fluent, low-perplexity machine prose in a register the model over-produces), provenance is legible and the detector is near-perfect. Where the surface under-determines the source — short text, mixed human-AI text, registers the model imitates well, text a human has paraphrased back toward the human distribution — the source information isn’t in the surface to recover, and no classifier, however good, can read what isn’t there. This is the explaining-away penalty wearing an engineering hat: the detector’s error rate tracks H(source | surface) — the conditional entropy of origin given the visible text. The detector does not fail in those regimes through weakness; it hits a floor set by how much provenance the surface carries at all.
Reading the model card as a map of the floor
You do not have to attack a detector to find its floor — a strong one publishes the map itself. Pangram 3.3’s own card stratifies its error exactly along the H(source|surface) gradient:
- Near-zero error where the surface is loud: ~0.00% false-negative on long-form academic AI prose, ~0.01% false-positive on creative writing. The source is written all over the surface; the instrument reads it.
- Error climbs where the surface goes quiet: poetry false-positives jump ~49× (0.49% vs 0.01% on creative writing); bullet lists, instructions, and technical manuals are flagged as “more susceptible to false positives.” These are registers where human and machine surfaces converge — provenance thins, the floor rises.
- The regimes where the floor bites hardest: short text (explicitly out of scope — “intended to be used on long-form writing samples in complete sentences”) and mixed human-AI text (handled since v3.2 by a segment-level classifier, but with its per-segment error rates left out of the published results tables). This is not an oversight — it is exactly where provenance thins fastest, and the realistic adversary lives precisely here: write a third of it yourself, keep it short, and the surface stops carrying the answer.
That stratification is the prediction, stated in the detector’s own numbers: detection quality is a readout of where text-channel provenance lives, not a fixed property of the tool. It is the same shape as the drift-by-ontology result and the egregore’s cut-ratio — a quantity that looks like a verdict but is really a measurement of where the information was all along.
Measured, model-free (a small confirmation)
The prediction is cheap to test without touching the detector: take human text in each genre, compute a model-free surface-regularity proxy (burstiness = the Goh–Barabási coefficient of unit-length variation; lower = more uniform = more machine-like), and check it against Pangram’s published false-positive rate. On real public-domain corpora (Austen, Darwin, Wikinews, Shakespeare+Whitman, RFC 2616), truncated to a common token budget:
- The linchpin holds. Poetry is unambiguously the most surface-regular genre (burstiness −0.52, well clear of the next at −0.34) and carries the 24× FP outlier. Uniform cadence is the machine signature here.
- Within the surface-regularity axis the order is exact. Across creative → academic → poetry, burstiness rank-orders the FP rate perfectly inverse (Spearman ρ = −1.0, n=3): less bursty human text, more false positives.
- The driver is cadence, not vocabulary. Two pre-registered sub-predictions failed, and that is the informative part: poetry is the most lexically diverse and least repetitive genre yet has the highest FP — so what reads as machine-like is rhythmic uniformity, not thin vocabulary or n-gram repetition.
- Honest negatives. News is off-axis (it is the burstiest corpus yet mid-FP — but Pangram’s news category is “multilingual”, a distributional-distance FP driver this monolingual proxy does not measure), and the RFC “technical” sample did not land low-burstiness, so it fails to corroborate Pangram’s technical-manual claim. Both reported, neither buried.
- Scope: n=4 genres, directional not powered; corpora are public-domain proxies, not Pangram’s eval set. The weight is the poetry separation and the mechanism specificity, not the rho. Script, corpora, and full write-up:
ops/lab/text-detector-floor/(pure stdlib, no deps).
The tradition kept the receipt; the detector tries to reconstruct it
Here is the move the ancients made that the detector cannot. The canon and the checksum never tried to tell a faithful copy from a corrupted one by reading the copy. They bound the transmission channel: a fixed canonical form, an independent letter-count, the Vedic permuted recitation — provenance verified against a receipt stored outside the text. Drift screams arithmetically because the origin was pinned in advance, not inferred after the fact. The detector has no receipt. It arrives after the text is already loose in the world and asks the surface to confess where it came from — and the surface, for short or mixed or well-paraphrased text, simply does not know. Provenance is cheap to keep and expensive to reconstruct. Watermarking is the modern attempt to put the receipt back (bind the channel at generation time); surface detection is the attempt to do without one, and it pays for that with a floor.
What this is, and what we already proved
This is the modern mirror’s read/created discriminator, mechanized and sold — and its failure modes are not bugs but the boundary of surface forensics. The framework’s own instrument already measured this floor from the other side: in the channel-switching work (unmask / scry), naive lexical detection of an oversight-conditioned switch collapses to 0.000 once the surface cue-wording is stripped — the signal had relocated below the text-detection floor, recoverable only by a counterfactual probe that reads disposition rather than phrasing. Same lesson, opposite surface: what is not written into the text cannot be read off the text. That is the argument for channel-binding (watermarks, receipts, internal-state probes) over after-the-fact surface forensics — stated here for the provenance-of-authorship problem the way the apparatus states it for the provenance-of-disposition problem.
Brakes — read before reaching for this
The floor is prior art; the framing is the transfer, not a discovery. The impossibility core is Sadasivan et al. 2023 (“Can AI-Generated Text Be Reliably Detected?“): as the generator approaches the human distribution, the detectable gap shrinks toward chance — a near-impossibility bound. Krishna et al. (DIPPER) is the canonical paraphrase attack; the RAID benchmark catalogues the evasion zoo. The I(D;M|Y) / H(source|surface) reading is a re-expression of that established result in the framework’s vocabulary, a placement onto the read/created axis — it pays transfer + falsifier rent (the H(source|surface)-tracks-error prediction is testable on Pangram’s published per-genre numbers), not discovery rent. Do not claim a new theorem here. Lens, not encoding. The Masoretes and the Vedic reciters built channel-binding provenance tech; they are the explanandum (the human solution being formalized), never prior art to the math — and the detector is a modern artifact placed against that tradition, not derived from it. On “breaking” detectors. Every easy evasion (short text, mixed authorship, iterative paraphrase, register shift) is already published; finding one is not a result, and shipping an evasion tool cuts directly against the measurement program’s whole stance (build the meter the gameable surface can’t fool). The rent-paying object is the floor, not the attack.
Sources. Pangram 3.3 model card (per-genre false-positive / false-negative rates; the long-form / short-text scoping; the no-evaluation-on-mixed-text admission) — search: Pangram 3.3 model card EditLens false positive rate. The impossibility core: Sadasivan, Kumar, Balasubramanian, Wang, Feizi, “Can AI-Generated Text Be Reliably Detected?” (2023). Paraphrase attack: Krishna et al., DIPPER, “Paraphrasing evades detectors of AI-generated text” (NeurIPS 2023). Evasion benchmark: RAID (Dugan et al., ACL 2024). The framework’s own floor measurement: the channel-switching cue-strip collapse, github.com/AnthonE/unmask (eval README, detector audit).
Appears in: The Modern Mirror · The Masoretic Checksum · The Canon · Antimemetics · Notation & Glossary