Former OpenAI and Anthropic researchers built a 1930s AI

What happens when you build a modern large language model, but forbid it from knowing anything that happened after December 31, 1930?

That was the question tackled by Nick Levine, former Anthropic researcher David Duvenaud, and former OpenAI researcher Alec Radford. The trio just released Talkie, a 13B vintage language model trained entirely on 260 billion tokens of pre-1931 public-domain books, newspapers, journals, patents, and legal rulings.

Talkie is fun to chat with, but it wasn't built as a novelty bot. It was designed to solve one of the biggest headaches in AI research: benchmark contamination.

Coding without knowing computers exist

Because Talkie’s worldview stops before digital computing arrived, it offers a rare, sterile environment to measure true generalization.

To see if the model could genuinely reason rather than regurgitate web data, the team gave Talkie a few-shot Python coding prompt. The model had never seen code, let alone Python. Yet, given a couple of short examples, it successfully wrote working one-line programs.

In one standout test, the researchers gave Talkie an encoding function for a rotation cipher and asked for the decoding function. The model swapped an addition sign for a subtraction sign and solved the problem. It was a single-character edit, but it signaled a genuine, uncontaminated grasp of inverse functions.

The researchers also tested Talkie's predictive abilities by feeding it post-1930 historical events. The model took pre-1931 history in stride, but its 'surprisingness' score shot up dramatically for events from the 1950s and 1960s before plateauing, offering a clean baseline for how language models forecast the unknown.

Beating bad OCR and modern chat habits

Training a 1930s AI came with messy, era-specific hurdles.

First was data quality. Because natively digital text did not exist in 1930, everything had to be scanned from physical print. Classic OCR systems struggled so badly that models trained on raw scans hit just 30% of the performance of those trained on clean human transcripts. Regex cleaning boosted that to 70%, and the team is now building custom vintage OCR tools to close the gap.

Second was post-training. Feeding the base model off-the-shelf instruction datasets would instantly inject modern slang and contemporary attitudes. Instead, the team built training pairs from vintage etiquette manuals, letter collections, and cookbooks, using Claude Sonnet 4.6 as an automated judge to steer multi-turn conversation quality.

Even with an n-gram anachronism filter, total isolation proved tricky. An early 7B version knew about FDR's New Deal, and the 13B model still harbors faint glimpses of World War II and the United Nations.

Why it matters

Every frontier model today feels familiar because they all digest the same modern web scrapings. They also suffer from severe benchmark contamination: when a model scores high on an evaluation, researchers can rarely be sure if it reasoned its way through or simply memorized the answer during pre-training.

Talkie provides a genuinely contamination-free reference point. By stripping away nearly a century of digital noise, it allows researchers to probe how raw language shapes reasoning, whether models can rediscover scientific principles from first principles, and how vocabulary constrains AI personas.

When paired against an identical modern twin trained on FineWeb, Talkie lagged on modern trivia but held its own on core linguistic comprehension and basic numeracy. That tells us foundational reasoning depends far more on the underlying grammar and structure of language than on modern web content.

You can chat with Talkie right now in your browser. Ask it about TikTok, and it will likely assume you mean a mechanical clockwork device from the 1800s. We think this approach is brilliant: as the team gears up for a GPT-3-level vintage model trained on over a trillion tokens this summer, vintage AI could become one of the cleanest tools we have to measure what machines actually know.