ChatGPT might actually be slacking off for winter break

ChatGPT might actually be slacking off because it thinks it is holiday season.

After weeks of users speculating that GPT-4 had become lazy in December, developer Rob Lynch decided to put the "AI Winter Break Hypothesis" to a statistical test. Wharton professor Ethan Mollick flagged the results, noting that the data actually backs it up.

May vs. December in the system prompt

Lynch used the gpt-4-turbo API to run an identical code completion prompt: implement a machine learning task without using external libraries.

The only difference between the two test groups was the date specified in the system prompt. One set told the model it was May, while the other told it it was December.

Across 477 completions for each group, the results showed a noticeable gap. The May prompt yielded a mean completion length of 4,298, while the December prompt dropped to 4,086. With a t-test p-value below 2.28e-07, the difference is statistically significant.

Why it matters

If you have noticed GPT-4 giving shorter code snippets or feeling more terse lately, your easiest workaround might literally be telling the API it is spring.

Beyond the humor, this highlights just how weird and sensitive frontier LLMs are. Because models train on human-generated text, GPT-4 appears to have picked up the subtle human tendency to do less work ahead of the winter holidays.

We think this serves as a cautionary tale for developers building agent workflows. Many production pipelines routinely inject real-time timestamps into system prompts without second thought, but this test shows that even an innocent date parameter can accidentally degrade model performance.