OpenAI fixes AI coding benchmarks with SWE-bench Verified
OpenAI just launched SWE-bench Verified, a human-validated subset of the popular software engineering benchmark built with its original authors. Working with 93 professional Python developers, the team filtered out ambiguous issue descriptions and unfair unit tests to curate 500 reliable tasks. Why it matters: Coding benchmarks often penalize correct code with broken unit tests, and this cleaner test set gives the industry a far more accurate gauge of real-world agent capabilities.