OpenAI drops GPT-4o System Card detailing voice and persuasion risks

If you have talked with GPT-4o, you know how unnervingly natural it feels. The voice pauses, chuckles, and responds in an average of 320 milliseconds. But according to OpenAI’s own safety assessment, its most dangerous capability is not hacking firewalls or engineering toxins. It is how good it has become at convincing you.

OpenAI finally dropped the full GPT-4o System Card, detailing months of safety stress-testing before wider voice deployment. Under its Preparedness Framework, GPT-4o scored "Low" risk in cybersecurity, biological threats, and model autonomy. However, it scored a borderline "Medium" in persuasion. Because the highest individual score dictates the overall rating, GPT-4o is officially ranked as a Medium risk model.

A hundred red teamers and a very persuasive bot

To probe the model's new capabilities, OpenAI engaged more than 100 external red teamers across 29 countries and 45 languages between March and late June 2024. Testing spanned four phases, culminating in trials inside the live iOS interface. External groups like METR and Apollo Research were also brought in to inspect autonomous capabilities.

The real headache for researchers was the speech-to-speech architecture. Because GPT-4o processes text, audio, and vision within a single neural network, it opens up novel hazards: unauthorized voice cloning, copyrighted audio leakage, and emotional anthropomorphism. OpenAI even converted existing text benchmarks into audio using its Voice Engine to measure speech safety.

Yet the most striking finding was in persuasion. While GPT-4o was not universally more convincing than humans, researchers found specific writing samples where the model swayed reader opinions more effectively than human-written text.

Perfect timing, imperfect optics

The report lands at a tense moment. GPT-4o debuted in May, yet this safety card arrived months later, right in the shadow of the upcoming US presidential election.

Just minutes before the report went live, Senator Elizabeth Warren and Representative Lori Trahan released an open letter pressing OpenAI on whistleblower protections and internal safety protocols. With former safety staff claiming safeguards took a backseat to shiny products, and California lawmakers pushing hard on safety legislation, outside scrutiny has never been higher. As The Verge pointed out, despite third-party testers, the entire rubric is still OpenAI grading its own homework.

Why it matters

For users and developers, GPT-4o represents a massive leap in human-computer interaction, but one that requires guardrails. Its voice responsiveness makes it unmatched for live translation, tutoring, and customer service. However, the borderline medium persuasion rating is a warning. If an AI sounds human, reacts like a human, and writes more convincingly than a human, the potential for social engineering and subtle bias manipulation is enormous.

For the industry, the battleground has officially shifted. Text moderation is no longer enough when neural networks stream raw audio and vision in real time. OpenAI's approach of using Voice Engine to translate text evals into speech evals is clever, but it also reveals how underdeveloped multimodal safety benchmarks actually are.

In our view, OpenAI deserves credit for openly labeling its own flagship model as medium risk due to persuasion. But self-policing only goes so far. As models become more emotionally resonant and legislation like California's AI bill looms, proprietary safety scorecards are unlikely to satisfy regulators for much longer.