The State of AI Text Detection — May 2026

If you want the honest answer for AI-text detection in May 2026, it's this: there is no single detector you can honestly market as a reliable universal "AI lie detector" for arbitrary text.

That's not pessimism. It's what the public research keeps finding, in study after study, including a 2025 FTC enforcement action against an AI-detection vendor for unsupported accuracy claims. This post is our working map of what actually works, what's still broken, and what we ship in /check/text because of it.

Why the case against generic detection is strong

The biggest problem is generalization. A detector that looks superb on one dataset can collapse on another, even when both are "AI vs human."

A 2026 explainability-focused study found that models reach leaderboard-level in-domain performance but degrade sharply cross-dataset and cross-generator. SHAP analysis showed many models were leaning on unstable cues like paragraph count and GZIP compression ratio — dataset artifacts, not durable signals of machine authorship. False-negative rates on unseen frontier models like GPT-5.2 and Gemini 3 Pro varied wildly between configurations, sometimes by an order of magnitude.

The second problem is metric gaming. High AUROC can hide unacceptable false positives. The 2025 NAACL paper A Practical Examination of AI-Generated Text Detectors for Large Language Models argues for TPR at a fixed low FPR, not just AUROC. Its results showed that in some settings, TPR@1% FPR fell to 0% for detectors with decent AUROC.

The third problem is that human-AI collaboration is not the same problem as pure AI authorship. Most products force a binary, but the realistic task is tri-class: human, AI, mixed. GPTZero's 2026 paper explicitly argues for ternary framing. Authorship-analysis research found that authorship embeddings outperform generic detectors when text is co-written.

The fourth problem is false positives on tidy human writing. The 2026 Spotlights and Blindspots evaluation found high variance based on dataset and metric and poor performance on novel human-written texts in high-risk domains.

There's also a fairness concern around non-native English writing. OpenAI cited disproportionate impact on non-native speakers as one reason for hesitating to ship text watermarking. Pangram's technical report disputes the bias claim, but independent cross-vendor fairness audits remain thin.

What zero-shot statistical detectors can and can't do

The classic zero-shot family: DetectGPT (arXiv:2301.11305), Fast-DetectGPT (arXiv:2310.05130), Binoculars, GLTR (arXiv:1906.04043), plus simpler perplexity / burstiness / rank-ratio methods. In 2026 they still matter — but mostly as auxiliary evidence, not as primary truth engines.

The strongest recent negative evidence is from DetectRL. Under that benchmark's realistic setup:

• DetectGPT fell to 22.15 AUROC on academic writing and 12.21 AUROC on news
• Binoculars scored only 55.15 AUROC on Claude-generated text
• Adversarial perturbation pushed zero-shot detectors to an average 34.32 AUROC
• Mixed human/AI text dropped average performance to ~52.51 AUROC — barely above coin-flip

The 2025 NAACL paper found that on GPT-4o question answering, Binoculars hit only 0.05 TPR@1% FPR with 0.6533 AUROC. Fast-DetectGPT hit 0.11 TPR@1% FPR with 0.6981 AUROC.

The 2026 Detecting the Machine benchmark makes an important refinement: raw perplexity-style unsupervised detection is weak as a standalone rule, but sentence-level perplexity coefficient of variation is valuable inside a broader stylometric feature set. That's a useful product insight — surface burstiness and token-surprise maps as evidence, but treat them as features in an ensemble.

For paraphrased or humanized text, the outlook is worse. A 2025 paper on 19 AI humanizer and paraphrasing tools concluded that many existing detectors fail. 2026 attack papers like MASH and HIP show iterative paraphrasing and style transfer remain potent detector-evasion strategies.

Watermarking and provenance

Watermarking is the only text-detection family that can plausibly beat post-hoc inference — but only when the generator cooperates and the text hasn't been heavily transformed.

SynthID-Text (Google DeepMind, Nature, 2024) is the only clearly deployed large-scale text watermarking system in production. Rolled out across Gemini app and web, open-sourced via developer docs, with SynthID Detector launched for early testers in 2025. Adoption numbers Google publishes for cross-modal SynthID are impressive (100B+ images/videos, 60,000 years of audio, 50M Gemini verifications), but I couldn't find comparable text-specific figures.

The main weakness is robustness under meaningful rewrites. Independent 2025 work found paraphrasing, copy-paste modification, and back-translation can significantly degrade SynthID-Text detectability. A proposed extension called SynGuard reported an average 11.1% F1 improvement in recovery under attacks.

OpenAI is the opposite story. OpenAI said in 2024 it had built a text watermarking method with high accuracy but had not launched it because it was less robust to translation and rewording, could be trivially circumvented in some cases, and might have socially uneven effects.

The Aaronson scheme remains influential in research but hasn't become widely documented in mainstream commercial generation.

For your product: if provenance exists, privilege it above inferred statistical authorship. And if provenance doesn't exist, the UI should say "no provenance found", not "this is human."

Stylometry and authorship attribution

If zero-shot heuristics are too brittle and watermarks are too sparse, what's left? Stylometric or hybrid supervised detection, especially when interpretable.

The 2026 Detecting the Machine benchmark is important for product design. A stylometric hybrid XGBoost with features like sentence-level perplexity CV, connector density, and AI-phrase density achieved 0.9996 AUROC in-distribution and ~0.904 AUROC cross-domain. Better than naive classical baselines, close to fine-tuned transformers, and remaining interpretable.

But stylometry isn't a universal silver bullet either. The 2026 explainability paper showed feature-based systems hitting F1 0.9734 on benchmark data while failing under domain or generator shift. SHAP revealed some apparent gains came from exploiting formatting quirks. The right product move is stylometry in an ensemble with calibration and visible caveats.

Authorship analysis becomes much more interesting with a claimed human author and a reference corpus from that author. Research on human-AI collaborative writing found that authorship embeddings outperform earlier generic AI detectors in co-writing settings. The 2025 ACL paper The Two Paradigms of LLM Detection: Authorship Attribution vs. Authorship Verification pushes this distinction.

Strategic insight for the roadmap: generic AI detection and authorship verification are different products. A journalist verifying whether a press release "sounds like AI" has one problem. A newsroom or law firm asking whether a document is consistent with this known writer's old work has a much better-posed problem.

Vendor claims vs. independent audits

GPTZero says it detects ChatGPT, GPT-5, Claude, and Gemini, advertising 99% accuracy. Its 2026 paper reports very strong internal results — 99.9 AUC across several domains, 93.5% recall on a paraphrase/bypass dataset — but the paper itself acknowledges the lack of standardized public benchmarks and the risk of cherry-picking.

Pangram publicly claims 99.98% accuracy. Its 2024 technical report is vendor-authored, but Pangram has the strongest recent independent validation: a 2025 UChicago audit that compared Pangram, OriginalityAI, GPTZero, and a RoBERTa baseline. Pangram was the only detector meeting a stringent FPR ≤ 0.005 policy cap without sacrificing detection, remained robust on StealthGPT-humanized text, and was cheaper per correct detection than OriginalityAI and GPTZero. Among visible vendor claims, Pangram is the one most clearly supported by recent independent audit.

Originality.ai emphasizes third-party-verified accuracy and detection of paraphrased text including recent models like Llama 4 and Claude Opus 4.7. UChicago places it in a real but secondary tier — better than GPTZero on AUROC, much stronger than open-source RoBERTa, but worse than Pangram under strict low-FPR caps.

Copyleaks markets ">99% accuracy" from internal English testing plus reference to third-party studies. No comparable neutral academic audit in 2025–2026 that included Copyleaks.

Turnitin emphasizes "less than 1% false positive rate" and is notably more conservative about instructor judgment than many marketing pages. No recent peer-reviewed independent benchmark against the full 2026 frontier-model set.

Winston AI publishes responsible-use educational material. No comparable peer-reviewed independent benchmark.

The independent headline: commercial tools can outperform open-source baselines by a lot, but the ranking among commercial tools is still context-dependent, and only a small subset has been stress-tested by neutral parties.

What you can defensibly promise

The safest marketing copy is not "we can tell if text is AI." It's closer to:

"We estimate AI-likelihood, identify possible mixed authorship, verify provenance where available, and show the evidence behind every result. Best on medium-to-long prose. Less certain on short, translated, or heavily rewritten text."

That's defensible because it matches the literature's strengths and failure modes. It doesn't promise universal correctness. It centers evidence. And it aligns with the FTC's clear hostility to unsupported accuracy claims (the 2025 Workado action is the precedent everyone should know).

The constraints to state visibly:

• Best-fit scope: medium and long English prose, limited post-editing, enough text for stable signals
• Known weak spots: short text, translation, strong paraphrasing, AI humanizers, mixed authorship, highly formulaic technical writing
• Interpretation rule: scores are triage and review support, not sole basis for discipline, takedown, or fraud adjudication

What we ship

If we had to pick two-to-three detectors to ensemble, we wouldn't pick three zero-shot detectors. We'd pick three different kinds of evidence:

1. A supervised encoder + stylometric hybrid as the core. Fine-tuned encoder for raw discrimination, paired with an explainable stylometric model (XGBoost over sentence-level perplexity CV, connector density, AI-phrase density, compression statistics, paragraph structure, sentence-variation signals).

2. One zero-shot statistical detector as auxiliary feature. Binoculars is the most defensible single pick from the classical family, with Fast-DetectGPT as runner-up. Auxiliary, not front-page number.

3. Provenance verification whenever available, overriding inference. Watermark and metadata checks when a provider cooperates, plus signed logging or chain-of-custody metadata.

Optional fourth layer for the forensic brand: author-baseline verification. Given prior human-authored samples, compute whether new text is consistent with that author's historical style. Defensible in journalism, legal workflows, and internal investigations.

What we ship next

Top move: three-way text verdict with evidence cards. Ship Human / Mixed / AI-assisted as the top result with confidence bands and sample-quality flags. Show sentence-level contribution heatmaps, a stylometric evidence panel, and a provenance panel with hard distinction between "provenance found," "no provenance found," and "not enough text." Gate on minimum length. Visibly lower confidence for short samples.

Next: a humanizer / paraphrase stress-test harness in every report. Make laundering part of the product. For each analysis, run a paraphrase robustness pass and show whether the verdict is stable under light perturbation. Label it "robustness under rewrite," not "evades AI detection."

Third: an author-baseline mode for high-trust customers. Journalists, legal teams, publishers, eDiscovery — let customers upload verified writing samples from a claimed author and score new text for consistency with the known writer's historical style. Moves you from the crowded "generic AI detector" lane into the more defensible forensic authorship lane.

Open questions

The biggest unknown: there's still no strong neutral public benchmark cleanly auditing all major commercial detectors against the newest frontier models (GPT-5, Claude Opus 4.7, Gemini 3, Llama 4) across mixed authorship, paraphrase, translation, and short-text conditions.

Second: text-watermark adoption volumes specifically. Google has shipped SynthID-Text, but I didn't find public text-specific usage numbers comparable to its image/video/audio figures.

Third: fairness under real deployment, especially for non-native English, accessibility writing assistance, and domain-specific professional prose.

The strategic open question: how much of this space will be won by post-hoc inference vs. provenance infrastructure. The policy direction in Europe and the provenance direction from Google and OpenAI suggest the long-term equilibrium is hybrid: watermarking and signed metadata where possible, cautious statistical inference everywhere else. If that's right, the best long-term bet is not a bigger black-box score. It's the evidence layer that can combine both.

That's the /check/text we're building.