Anthropic's Clio analyzes AI use without spying on chats

What are millions of people actually doing with AI chatbots all day? Surprisingly, even the companies making these models don't really know, mostly because reading private conversations is a massive privacy violation.

Anthropic pulled back the curtain on Clio, an automated internal analysis system now named Anthropic Insights, along with findings from an analysis of 1 million conversations on claude.ai.

Teaching Claude to summarize Claude

Traditional trust and safety monitoring usually relies on top-down filters. You write rules for what bad behavior looks like, and you scan for it. Clio flips this by working from the bottom up, grouping similar conversations into high-level topic clusters much like Google Trends.

To keep user data strictly private, human analysts at Anthropic never see the raw text. Instead, Claude does all the heavy lifting.

The system extracts metadata like conversation turn counts and language, groups similar chats into semantic clusters, and generates an anonymized summary for each topic while stripping away any identifying details.

There is also a strict minimum threshold of unique users. If a topic is only discussed by a couple of people, it never surfaces to human analysts at all.

So what did the 1 million conversation sample reveal? Coding is king on Claude. Web and mobile application development represented over 10% of all chats, covering everything from debugging code to explaining Git mechanics.

Education and learning took more than 7% of conversations, while business strategy and operations accounted for nearly 6%. The long tail was wildly creative, spanning dream interpretation, disaster planning, Dungeons & Dragons campaigns, and users relentlessly asking Claude to count the r's in strawberry.

Catching botnets and fixing false alarms

Beyond curiosity, Clio serves as a detection engine for platform abuse.

In late September, Anthropic used Clio to bust a coordinated network of automated accounts churning out SEO spam. While no individual prompt violated rules on its own, aggregating them revealed an automated operation that violated terms of service.

Clio also helped fix frustrating false positives. Previously, job seekers uploading resumes were flagged simply because their contact information was present. Developers asking networking and web scraping questions were mislabeled as hackers, and tabletop gamers discussing combat stats were flagged for violence. Clio helped the team pinpoint and correct these errors.

Why it matters

For anyone concerned about their privacy when using AI, Anthropic's approach sets a reassuring benchmark. Proving that an AI company can study product usage and catch bad actors without human employees peeking at your chat logs is a huge win for user trust.

The findings also validate where Claude's real strength lies. Over 10% of traffic is pure software development. If you are only using Claude for light web browsing or quick text edits, you are missing out on what power users clearly rely on it for: heavy debugging and business workflow management.

For the AI industry, Clio marks a shift in how platforms must defend against malicious use. Threat actors no longer rely on single toxic prompts; they operate automated swarms generating subtle spam or reselling unauthorized access. Spotting these attacks requires macroscopic pattern matching, not just static keyword filters.

That said, we think the threshold-based clustering approach has a natural blind spot. If sophisticated attackers deliberately keep their operations small, fragmented, and below cluster thresholds, automated tools might overlook them entirely. The battle between distributed abuse and privacy-preserving surveillance is only just beginning.