AI coding tools make developers write buggier code, study finds

AI is supposed to make developers 10x engineers, but it might actually be turning them into 10x bug creators.

A team of researchers affiliated with Stanford released a study examining how code-generating AI tools affect software security. The subject of their test was OpenAI's Codex, the underlying model that powers GitHub Copilot. The verdict: developers who leaned on the AI produced significantly more security flaws than those who did not.

More bugs, higher confidence

The researchers set up an experiment with 47 developers, ranging from undergraduate computer science students to seasoned industry veterans with decades of experience. The participants were asked to tackle security-sensitive coding problems across Python, JavaScript, and C.

The findings revealed a striking psychological trap. Developers working with Codex were far more likely to produce incorrect and insecure solutions compared to the control group. Even worse, those same developers were far more likely to believe their insecure code was rock-solid.

Lead author Neil Perry, a PhD candidate at Stanford, emphasized that these systems are in no way a replacement for human engineers. He warned that using AI outside your core expertise is particularly risky, and even when using it to speed up familiar tasks, developers must double-check every output and consider how it fits into the broader application.

Co-author Megha Srivastava added that the tools often default to standard library settings that are ripe for exploitation. Because many general developers lack specialized security backgrounds, those subtle traps slip past unnoticed.

From copyright slip-ups to default traps

The researchers were careful not to dismiss code generation entirely. Srivastava noted that AI is genuinely useful for low-risk work, such as exploratory research, and suggested companies could fine-tune internal models on proprietary, vetted codebases to enforce better security practices.

Still, code vulnerabilities are only half the headache. Codex was trained on billions of lines of public code, some of which carry restrictive licenses. Users have famously prompted Copilot into regurgitating proprietary code from Quake or full exercises from programming textbooks.

GitHub rolled out a filter in June to check roughly 150 surrounding characters against public repos, but it remains leaky. Texas A&M professor Tim Davis pointed out that even with the filter turned on, Copilot still spat out large chunks of his copyrighted code, complete with original attributions and licenses.

Why it matters

We think this study exposes the most dangerous pitfall of the generative AI boom: the illusion of competence. When an assistant outputs clean, syntactically perfect code in seconds, developers naturally let their guard down. Conflating working code with secure code is a disaster waiting to happen in production environments.

For individual engineers and engineering leads, the takeaway is clear: treat AI as a junior intern writing rough drafts, not an architect. If you use it to move faster, you need to budget that saved time back into code reviews, automated security scanning, and sanity checks — especially on anything touching cryptography or user input.

For vendors like GitHub and OpenAI, the pressure is on. Slapping a rudimentary duplication filter on the output is no longer enough. The next battleground for coding assistants won't just be speed or parameter count; it will be whether these models can actively enforce safe coding defaults before dangerous code ever reaches the terminal.