System Analysis · 36/36 Tests Passing

How Baloney
Detects AI Content

A multi-signal ensemble system tested against 205 labeled samples across text, image, and video. Every number on this page comes from our production test suite.

93.2%
Precision
When we flag AI, we're right
97.0%
Specificity
Human content stays unflagged
1.197
Cohen's d
Large effect size separation
36/36
Tests Passing
10 test suites, 205 samples

What We're Best At

  • 93.2%When Baloney flags content as AI, it's correct 93% of the time. Only 3% false positive rate.
  • 97.0%Human-written content is correctly identified 97% of the time. Your real posts won't get flagged.
  • 1.197Cohen's d effect size — the statistical separation between AI and human signals is large and robust.

How It Works

  • 3-4Independent detection signals per modality — primary commercial APIs plus local analysis.
  • CascadeSynthID watermarks checked first, then commercial APIs (Pangram for text, SightEngine for images).
  • AdaptiveWeights redistribute when APIs are down. The system never fully fails — it degrades gracefully.

Detection Pipeline

Primary commercial APIs with watermark detection and local analysis signals

Text Detection (3 Signals)

Pangram API
SOTA commercial detector (arXiv:2402.14873)
99.85%
38% weight
SynthID TextWATERMARK
Google Gemini watermark detection
Override
7% weight
Statistical (12 feat)
Burstiness, entropy, readability, hedging
67.3%
18% weight

Image Detection (4 Signals)

SightEngine
ARIA benchmark #1, 120+ AI generators
98.3%
32% weight
SynthID ImageWATERMARK
Google Imagen watermark detection
Override
10% weight
Frequency/DCT
Multi-scale variance + spectral analysis
83.3%
18% weight
Metadata/EXIF
Camera provenance and format signals
~75%
13% weight

Video Detection

Extracts up to 8 frames, runs image ensemble per-frame, then applies temporal consistency analysis. Standard deviation < 0.08 with high avg score triggers temporal AI bonus. Covers deepfakes, AI-generated clips, and spliced content.

Signal Separation

Measured across 205 labeled samples — 105 AI + 100 human

Feature
AI Mean
Human Mean
Delta
Signal (composite)
0.5362
0.3413
+0.1949
Burstiness
0.5718
0.7805
-0.2087
Readability
0.9478
0.8074
+0.1404
Avg Sentence Len
20.79
16.04
+4.75
Avg Word Len
5.84
4.73
+1.11
Perplexity Norm
0.7397
0.9265
-0.1868

Ensemble Weight Tuning

We tested 5 weight configurations to find the optimal balance between precision and recall

Weight Configurations

Read-heavy69.8% acc / 61.3% F1
Burst-heavy66.3% acc / 54.3% F1
Current67.3% acc / 55% F1
Equal weights55.6% acc / 26% F1

Verdict Threshold Tuning

Lenient (0.70/0.50/0.30)
Accuracy
69.3%
FP Rate
8%
FN Rate
52.4%
Current (0.75/0.55/0.35)
Accuracy
67.3%
FP Rate
3%
FN Rate
61%
Strict (0.80/0.60/0.40)
Accuracy
65.4%
FP Rate
1%
FN Rate
66.7%

Our current thresholds prioritize low false positives (3%) — we'd rather miss some AI than wrongly flag a human.

Image Frequency Analysis

83.3%
Local Method F+G Accuracy
Without any commercial APIs
0.999
Smooth vs Noisy Separation
AI smooth: 0.999, Random noise: 0.281
2.7x
AI vs Human Freq Ratio
AI avg: 0.733, Human avg: 0.274

Cascade Architecture

1
SynthID WatermarkEarly exit

Detects Google Gemini/Imagen watermarks. If found, returns 0.95-0.97 probability immediately.

2
Commercial APIPrimary

Pangram (text, 99.85%) or SightEngine (image, 98.3%). High accuracy, rate-limited.


Extension Features

Works Everywhere

Scans images and text on any website — Instagram, X, Reddit, news sites, and more.

Auto-Scanning

Automatically detects AI content as you scroll. No clicks needed.

Content Filtering

Blur or hide AI-generated content. Choose Scan, Blur, or Block modes.

Real-Time Stats

Track scans, AI exposure, and per-page breakdowns in the popup.

Instant Results

Color-coded dots and underlines appear in real time as you browse.

Deep Analysis

Click any detection to see per-method score breakdown in the sidepanel.

Install in 3 Steps

1

Download

Download the Baloney extension folder from our GitHub repository.

2

Unpack

Unzip the folder to a location on your computer you'll remember.

3

Load Extension

Open chrome://extensions, enable Developer Mode, click 'Load unpacked', and select the folder.

View Full Evaluation
Try the Detector