Companies claiming their AI-detection tools could separate human from machine writing with near-perfect accuracy are now in the FTC’s crosshairs.

The commission recently finalized an order against Workado, LLC (which marketed its detector via the website formerly known as Content at Scale, now rebranded as Brandwell), citing claims that these tools were trained broadly when in fact they were mostly trained on academic writing. That’s according to the investigation detailed in this report.

The FTC flagged a boldly claimed “98 % accuracy” rate as unsupported. In the order, Workado must pull those claims, send notices to prior users, maintain compliance logs and keep evidence for future ad claims.

The message: if you say your AI can detect AI like the human eye spots a counterfeit bill, you better have the receipts.

The timing is telling. With AI-generated text flooding classrooms, newsrooms and corporate comms, the pressure on detection tools is intense — but so is the temptation to oversell.

A separate analysis from Reddit users pointed out that “accuracy” is a slippery metric in detection tools, especially when the base rate (how often something is AI-generated) varies wildly.

Another angle: this is part of a broader regulatory trend. For instance, the FTC’s previous enforcement blitz targeted other “AI promises” involving tools that promised automated legal services or “AI-powered” storefronts but couldn’t substantiate the claims. In other words: the AI label isn’t a free pass.

What I find personally a bit unnerving is the ripple effect on institutions relying on these detectors — schools, publishers, even governments.

If the detectors are overstating their accuracy or being used without transparency, then false positives become a big deal.

An investigation into university AI-detection systems found that students were wrongly flagged, sometimes without human review. So this FTC action could be a welcome wake-up call.

Here are a few things to keep an eye on: 1) Will other companies get similar orders? 2) Will buyers of detection tools demand more transparency — like detailed training data, error rates and test splits? 3) Will institutions re-evaluate how they use these tools, maybe making them one layer in a process rather than the final word?

My take: AI content-detectors have a role — especially as AI-generated content becomes more pervasive — but they’re far from a silver bullet. Treating them as such is a recipe for trouble.

If you’re relying on one, ask hard questions: What was it trained on? What is its false positive rate? Who audited it?

Because if the FTC says you can’t simply slap “98 % accurate” on the box without proof, then you should probably demand that proof too.

Origianl Creator: Mark Borg
Original Link: https://ai2people.com/ai-content-detectors-get-the-hammer-federal-trade-commission-cracks-down-on-misleading-claims/
Originally Posted: Mon, 03 Nov 2025 12:24:30 +0000