ElevenLabs clamps down after trolls weaponize voice clones
ElevenLabs just announced it is tightening safety controls after 4chan trolls weaponized its beta tool to generate fake audio of celebrities like Emma Watson spewing hate speech. While the startup says it can trace generated clips back to accounts, it is now weighing payment verification, ID checks, or manual approvals to stop the abuse.
All it takes is sixty seconds of clean audio to make anyone say literally anything. Unfortunately, the internet trolls figured that out immediately.
Voice generation startup ElevenLabs found this out the hard way over the weekend, when its newly launched beta was hijacked by 4chan users to flood the web with abusive celebrity deepfakes.
A wild weekend on 4chan
ElevenLabs, founded by ex-Google machine learning engineer Piotr Dabkowski and ex-Palantir strategist Mati Staniszewski, recently announced a $2 million pre-seed round led by Credo.
The pitch was simple: high-quality speech synthesis and custom voice cloning for newsletters, audiobooks, and videos. To clone a voice, a user only needed to upload a clean sample longer than a minute.
By Sunday, clips generated using the tool were circulating widely on 4chan, accompanied by direct links to the ElevenLabs beta platform.
The generated audio ranged from trolling to overt hate speech. In one clip, a voice mimicking actress Emma Watson read excerpts from Mein Kampf. In another, a synthetic Ben Shapiro made racist remarks targeting Alexandria Ocasio-Cortez.
Other clips featured the voice of Rick Sanchez from Rick and Morty screaming violent domestic threats, mocking the actual real-world legal troubles of the show's creator.
Scrambling for safeguards
On Monday, ElevenLabs publicly acknowledged the chaos on Twitter, admitting it saw an increasing number of voice cloning misuse cases.
The startup noted that its backend can trace any generated clip back to a specific registered user, but that does little to stop an anonymous audio file once it hits the open internet.
Now, the company is weighing several emergency friction points. Potential new safeguards include requiring valid payment info, mandating full ID verification before unlocking the voice cloning tool, or manually reviewing every single clone request by hand.
Why it matters
We saw this exact movie play out with deepfake video years ago: novel technology debuts, bad actors exploit it for non-consensual content, and developers scramble to put guardrails on an open gate.
Audio is arguably more dangerous because the barrier to entry is practically zero. A one-minute sample is effortlessly scraped from podcasts, TikToks, or YouTube videos. Anyone with a microphone is now vulnerable to having their voice repurposed for defamation, harassment, or financial scams.
ElevenLabs considering paywalls and manual verification feels like a harsh overcorrection, but it highlights a harsh truth: frictionless AI onboarding and sensitive generative tech cannot coexist without serious abuse.
We think voice cloning startups are going to face immediate regulatory and legal heat if they don't introduce cryptographic provenance or strict biometric consent from day one. Trolls will always test the fences first, and ElevenLabs just proved the fences weren't built high enough.
Sources (1)
- vice.com(opens in a new tab)Originalvice.com