OpenAI SWE-Lancer tests LLMs against $1M in freelance gigs
OpenAI just launched SWE-Lancer, a benchmark testing whether frontier LLMs can tackle real freelance engineering work. Built from over 1,400 Upwork tasks valued at $1 million in payouts—spanning $50 bug fixes to $32,000 feature builds—it found that today's top models still fail at most tasks.