OpenAI brings fine-tuning to GPT-3.5 Turbo

For months, developers have been stuck in an expensive dilemma: pay up for GPT-4 to get reliable outputs, or use the cheap, fast GPT-3.5 Turbo and pray it actually follows your formatting rules.

OpenAI just broke that trade-off. Fine-tuning for GPT-3.5 Turbo is officially live, with GPT-4 fine-tuning slated to roll out later this fall.

According to OpenAI, early tests show that a fine-tuned GPT-3.5 Turbo can match, or even outperform, base GPT-4 capabilities on certain narrow tasks.

Cutting prompt size by 90%

Fine-tuning here is not really about teaching the AI new facts. It is about making it strictly obedient.

OpenAI highlighted three main wins from early beta testers: better steerability, rock-solid output formatting like clean JSON snippets, and custom brand tone. Instead of cramming dozens of examples into every single prompt just to keep the AI on track, developers can bake those behaviors directly into the model.

In fact, early testers managed to shrink their prompt sizes by up to 90%. That translates directly into faster API response times and significantly lower per-request costs.

The updated model handles a 4k token context window, double the previous fine-tuned models. Support for 16k context and function calling is promised for later this fall.

The pricing is surprisingly accessible. Training runs at $0.008 per 1K tokens, while inference costs $0.012 per 1K tokens for input and $0.016 per 1K tokens for output. A training dataset of 100,000 tokens run across three epochs costs roughly $2.40.

OpenAI also made sure to address enterprise privacy fears: any data sent into the fine-tuning API belongs entirely to the customer and will never be used to train future OpenAI models. To keep guardrails intact, training files pass through a GPT-4 powered moderation system.

Alongside this update, OpenAI introduced babbage-002 and davinci-002 as replacements for legacy GPT-3 models, which are set to shut down on January 4, 2024.

Why it matters

We think this launch takes serious air out of the open-source hype train. When Meta released Llama 2, the biggest selling point was giving teams a cheap, customizable alternative to OpenAI's walled garden. By letting developers fine-tune GPT-3.5 Turbo for literally a couple of dollars, OpenAI just removed the biggest friction point for small engineering teams.

If you run an application where GPT-4's latency or cost is killing your margins, you should test this immediately. Dialing in a custom JSON-generating engine on your own data will likely pay for itself in hours.

That said, fine-tuning is not magic dust. It adjusts behavior and formatting, not reasoning limits. Fine-tuned inference also costs noticeably more than the vanilla GPT-3.5 Turbo endpoint. Teams will still need to pair it with prompt engineering and retrieval systems rather than treating fine-tuning as an easy shortcut for clean data.