November 2026 · Independent humanizer benchmarks. Explore the evidence

PROTOCOL / UNIFORM RESCORING V1

How we test

A fixed source set. Saved outputs. Shared measurement profiles. HumanizeBench scoring v2 emphasizes bypass. The original scorer remains available for comparison.

01. Study design

We reuse the original 33 published source texts from the November 2026 AI Humanizer Benchmark cycle. Ten services contribute 330 existing published rewrites. Thirteen requested open-weight repository entries contribute 429 saved GPU-generated rewrites. All 759 outputs are re-scored under the same measurement profiles.

The original source set spans seven writing categories. We do not replace the inputs with fresh model answers or select favorable outputs. Different formats and conversions remain separate roster entries, with family relationships recorded.

02. Open-weight execution

Each entry uses a pinned model artifact, prompt basis, sampling profile, and runtime. Original author prompt formats are used where recovered. Two entries use the fixed neutral rewrite prompt authorized for this run. These reconstructed prompts are limitations, not recovered author settings.

Models run sequentially with GPU evidence and saved raw outputs. The final two 8-bit MLX entries used a separate Runpod GPU; completed local examples were preserved. Quantization formats were not silently substituted to improve scores.

03. Detector measurements

The panel is GPTZero, ZeroGPT, Copyleaks, Originality.ai, Pangram, Grammarly, and QuillBot. All seven routes now use Polygraph. GPTZero’s reported human percentage is normalized directly to 0–1; it is not inferred from the AI class. The other adapters normalize the provider’s reported percentages according to the frozen mapping.

For each output, the original scorer takes the median of numeric detector human scores. An entry’s bypass component is the average of these per-output medians. Errors and no-verdict responses remain missing; they are not measured zeroes. The original scorer has a zero fallback when no numeric verdict exists, so composites based on incomplete coverage remain explicitly provisional.

Successful measurements are cached by exact output text and frozen detector route. Identical text can reuse one successful assessment. Technical retries preserve the same text and keep raw response evidence; no best-of scoring is used. Humanize.beer GPTZero measurements are excluded from the active comparison after the user-authorized route change.

04. Meaning preservation

Source and output texts use NVIDIA Nemotron 3.5 Embed 8B passage embeddings, 4,096 float dimensions, with no truncation. Meaning begins as cosine similarity, clamped to [0, 1].

meaning = clamp((cosine − 0.75) / 0.25, 0, 1)

The original OpenAI embedding model is replaced by the agreed NVIDIA model. The original rescaling and the 0.85 raw-cosine drift threshold are retained without calibration. Raw cosine is shown separately on evidence pages.

05. Readability

A frozen NVIDIA Nemotron judge evaluates clarity, fluency, and naturalness as integers from 0 to 100. Readability is their sum divided by 300. The judge sees output text only: no model identity, source text, detector score, or historical rating.

The original Claude rubric was not recovered. This run uses the agreed reconstructed rubric consistently across both cohorts. One successful assessment is reused for identical text under the same frozen profile.

06. Composite and penalties

raw = 56.5 × bypass + 24 × meaning
    + 12 × readability + 7.5 × consistency
final = clamp(raw − penalties, 0, 100)

HumanizeBench v2 transfers 25% of each non-bypass weight to bypass: the original 42/32/16/10 distribution becomes 56.5/24/12/7.5. Penalties and metric definitions are unchanged. Download the original scorer; original-weight composite values are preserved as originalComposite in the benchmark JSON.

Consistency is one minus the population standard deviation of category-level bypass averages. With fewer than three categories, the original scorer holds it at a neutral 0.5. Consistency therefore depends on detector completion too.

Per test, raw cosine below 0.85 deducts one point, output/source word ratio above 1.4 deducts one, and below 0.6 deducts one. Identical output deducts two points, and refusal deducts one. Each penalty rule is capped at ten points. The identical/refusal classifier is the same conservative heuristic for both cohorts.

Download the active v2 scorer

07. Category views

Academic essays, application essays, blog posts, business emails, marketing copy, discussion boards, and news articles are available as views. Category numbers come from the HumanizeBench v2 per-category output, before entry-level penalties. A category view is not a new generation run or a separately tuned scoring system.

08. What this comparison does not establish

  • The new evaluation is approximate, not identical to the original judge and embedding pipeline.
  • Closed-source outputs are historical. Their hidden prompts, underlying models, and current product behavior are not reproduced.
  • Detector proxies and live vendor behavior are not independently verified as identical to the original cycle. Some routes may chunk or aggregate internally.
  • Family members and matching weight conversions should not be counted as independent model designs.
  • Missing detector measurements keep an entry provisional. No historical score fills a gap.
  • Detector human-likelihood scores do not prove human authorship.

Source texts and published service rewrites: AI Humanizer Benchmark, CC BY 4.0. Original scoring code: MIT. Upstream revision 53a76819abec2bbf034105de653edaa0ed21f681.

Download the run manifest and all saved test outputs.