OpenAI taps the public to help train its models

Who gets to decide what an AI should and shouldn't say? For the longest time, the answer was basically a handful of engineers in San Francisco. OpenAI wants to change that narrative—or at least look like it is.

OpenAI shared an update on its "Democratic Inputs to AI" grant program alongside the code created by its grantees. Back in May, the company pledged $100,000 each to 10 teams picked from roughly 1,000 applicants across 113 countries. The challenge: build working prototypes that let everyday humans use democratic methods to govern AI behavior. Now, with the experiments wrapped, OpenAI is setting up an in-house "Collective Alignment" team to turn those community-built ideas into actual model-training pipelines.

Shifting moods and the digital divide

The 10 selected teams covered 12 countries and pulled expertise from law, journalism, machine learning, and peace-building. When they actually put normal humans in front of governance tools, things got messy fast.

For one, public opinion turned out to be wildly fickle. Teams found that participants' views shifted on a day-to-day basis. If public sentiment swings that fast, it poses a headache for AI training: how often do you survey the public without letting fleeting social media moods overwrite core human values?

Then there's the participation bias. Participants recruited online naturally leaned far more pro-AI than the general public. Meanwhile, bridging the digital divide ran into basic technical walls. Philippine news organization Rappler found that speech tools like Whisper struggled badly with local languages like Tagalog, Binisaya, and Hiligaynon during focus groups, making it nearly impossible to capture non-English deliberation cleanly. In Africa, the Ubuntu-AI team had to build direct financial compensation into their platform just to get local creators to participate.

Finding consensus wasn't impossible, though. While vocal minorities consistently pushed back against any censorship whatsoever, structured deliberative methods yielded surprising common ground. The Collective Dialogues team managed to craft policy recommendations on vaccine information—a topic that usually starts a shouting match—that secured over 72% support across Democrats, Republicans, and Independents alike.

Enter the Collective Alignment team

Rather than shelving the papers, OpenAI announced it is forming a dedicated "Collective Alignment" engineering team. Their job is to build the end-to-end infrastructure that takes democratic input, encodes it, and actually feeds it into model behavior fine-tuning.

Grantees also noticed that participants were rightfully uneasy about letting AI write its own rules. The most trusted setups were hybrid: using AI to synthesize messy transcripts or draft policy language, but keeping final decisions firmly in the hands of human panels or majority votes. Strikingly, about 85% of participants across groups rallied behind a single core rule: whatever policies are chosen, they must be continuously expanded and updated as AI evolves.

Why it matters

For developers and everyday users, this hints at a future where AI guardrails aren't just dictated by a tech giant's safety committee behind closed doors. That said, don't expect a ballot box inside ChatGPT next week. The immediate practical utility for consumers is zero; this is long-term foundational infrastructure.

For the wider AI industry, this raises the bar on governance. While Anthropic has leaned into "Constitutional AI" using predefined expert principles, OpenAI is attempting something closer to citizen assemblies. If this works, every frontier lab will face uncomfortable questions from regulators and users if they continue relying on opaque, proprietary RLHF guidelines to decide worldviews.

We think this is a clever strategic play by OpenAI. Funding external academics allows them to outsource the messiest philosophical debates of our era while deflecting regulatory heat over bias and censorship. The real risk, however, is tokenism. As the grantees themselves showed, online participation systematically excludes marginalized voices, and speech tools still favor wealthy English speakers. If OpenAI simply turns these flawed inputs into fine-tuning weights, it risks slapping a false "democratic mandate" onto policies that are anything but universally representative.