Geoffrey Hinton isn't convinced good AI can beat bad AI

The tech industry loves comforting narratives, and its current favorite goes like this: as long as we build enough good AI, it will easily keep the bad AI in check.

AI pioneer Geoffrey Hinton isn't buying it.

Speaking at the Collision conference in Toronto this week, the former Google researcher argued that there is zero guarantee benevolent systems will come out on top. Having left Google earlier this spring specifically so he could speak freely, Hinton used the stage to challenge the unchecked acceleration of generative tech.

Killer robots, wealth gaps, and plumbing

Hinton pointed out that technology is only as reliable as the humans directing it, and malevolent applications could easily gain the upper hand. He noted that stopping the military-industrial complex from deploying battle robots will be extraordinarily difficult, simply because armies and defense contractors might love wars where casualties are replaceable hardware.

On the economic front, Hinton acknowledged that large language models like GPT-4 will drive massive productivity gains. But he warned that those gains risk being hoarded by the ruling class, directly making the rich richer and the poor poorer.

When asked about automation and mass job displacement, Hinton did not mince words. He suggested that some form of socialism might be required to tackle resulting inequalities, and jokingly—yet earnestly—advised people to look into resilient physical trades, like plumbing.

He also pushed back against critics who dismiss catastrophic AI risks as science fiction. If an intelligence surpasses humans, Hinton warned, our chances of remaining in charge are slim. His fear is that society will only act to rein in autonomous killer machines after seeing their horrors firsthand.

A tale of two stages

The contrast between Hinton and industry executives at the same conference was stark.

Google DeepMind business chief Colin Murdoch framed the past six to twelve months as a lightbulb moment for humanity. He highlighted DeepMind’s work mapping every known protein, fighting antibiotic-resistant bacteria, and accelerating malaria vaccines, insisting regulation should never choke off breakthroughs.

Roblox chief scientist Morgan McGuire struck a similar tone, celebrating how generative tools and open models like StarCoder bridge the gap for new game creators in the metaverse.

Hinton, for his part, admits he still loves the science. Had he stayed at Google, he says he would likely be building multimodal systems inspired by how children learn. But he argued that admiring early signs of reasoning in models like GPT-4 should not blind anyone to the severe lack of guardrails.

Why it matters

We think Hinton’s warning strikes at the heart of Silicon Valley’s favorite self-defense mechanism: the idea that every problem created by AI can simply be solved by buying more AI.

In the real world, defense is almost always more expensive and brittle than offense. Relying on good algorithms to police bad ones ignores power incentives. Defense contractors, bad actors, and deep-pocketed monopolies will naturally pursue advantages that safety-conscious models are constrained from matching.

For builders and professionals, the takeaway is clear: lean into generative tools today for immediate personal efficiency, but do not mistake corporate optimism for societal safety. When the creator of modern neural networks tells you that safety cannot be solved by code alone, the industry needs to stop treating regulation as an afterthought.