OpenAI gives GPT models function calling superpowers

Getting an LLM to reliably talk to software used to be an exercise in frustration. You would craft lengthy prompts begging the model to return raw JSON without conversational filler, only for it to happily reply with "Sure! Here is your data:", instantly crashing your backend parser.

OpenAI rolled out a major API update that tackles this problem head-on: native function calling for gpt-4-0613 and gpt-3.5-turbo-0613. Along with it came a 16k context window for GPT-3.5, substantial price cuts, and a deprecation timeline for legacy models.

Teaching GPT to speak JSON natively

Function calling allows developers to describe programming functions using JSON Schema directly inside the Chat Completions API.

When a user sends a prompt, the fine-tuned models can intelligently decide whether an external tool is required. If so, instead of returning prose, the model outputs a structured JSON object populated with the necessary function arguments.

This streamlines three major developer use cases. First, chatbots can query external APIs on the fly, turning "Email Anya about coffee next Friday" into a clean send_email call. Second, natural language can be converted into database queries or backend requests, like transforming "Who are my top clients?" into an internal API call or SQL query. Third, it allows reliable data extraction from messy text, such as pulling structured names and dates out of long articles.

OpenAI noted that tool integrations still carry safety risks. Untrusted data returned from a tool could potentially instruct the model to perform unintended actions. The company advises developers to restrict calls to trusted APIs and require human confirmation before triggering high-stakes actions like sending messages or making payments.

Longer context and aggressive price cuts

Beyond function calling, OpenAI introduced gpt-3.5-turbo-16k, quadrupling the standard 4k context window to roughly 20 pages of text in a single request. It is priced at $0.003 per 1K input tokens and $0.004 per 1K output tokens.

Meanwhile, running costs across the platform dropped significantly. Input tokens for the standard gpt-3.5-turbo saw a 25% price reduction down to $0.0015 per 1K tokens, which works out to roughly 700 pages per dollar. The popular text-embedding-ada-002 model received an even steeper 75% cut, dropping to $0.0001 per 1K tokens.

For existing integrations, default model names automatically pointed to the 0613 versions starting June 27. While original endpoints like gpt-4-0314 were initially slated for a September retirement, OpenAI extended support through at least June 13, 2024, following community feedback.

Why it matters

We see function calling as the moment language models transitioned from conversational novelties into genuine software components. Trying to parse unstructured model output with regex was fragile and messy. By baking function signatures and argument extraction directly into the model's fine-tuning, OpenAI made building automated workflows and agents far more reliable for developers.

It also shakes up the tooling ecosystem. Frameworks like LangChain gained massive popularity partly because they provided the glue code needed to force models into calling tools. When the underlying model handles tool selection and formatting natively, much of that scaffolding becomes redundant.

OpenAI's pricing move is equally sharp. While GPT-4 remains pricey and waitlisted for many, a 16k GPT-3.5 at a fraction of the cost makes processing large documents practical for everyday projects. The looming concern, however, is safety: as models gain the hands to interact with databases and send emails, guarding against prompt injections hidden in external tool outputs is now a major burden developers must shoulder.