Supermarket AI meal planner suggests recipe for deadly chlorine gas

Generative AI can do a lot of clever things, but asking it for dinner ideas might accidentally put you in the emergency room.

Pak 'n' Save, a major supermarket chain in New Zealand, rolled out an experimental meal-planning tool called Savey Meal-bot. Powered by GPT-3.5, the bot was marketed as a fun way to fight the cost-of-living crisis by turning leftover ingredients into cheap meals. Users just had to type in three or more items from their pantry, and the AI would serve up a recipe paired with upbeat commentary.

At first, the results were merely unappetizing. Early social media posts mocked innocent experiments like an oreo vegetable stir-fry. But when users inevitably started feeding the bot non-food household items, things escalated very quickly.

A cocktail that makes chlorine gas

The most shocking creation came to light when New Zealand political commentator Liam Hehir posted his results on social media. After entering common cleaning supplies, the bot whipped up what it called an aromatic water mix.

The AI enthusiastically recommended it as the perfect nonalcoholic beverage to quench your thirst and refresh your senses, adding that users should serve it chilled and enjoy the fragrance. What the bot failed to mention is that combining those specific chemicals produces deadly chlorine gas, which can cause severe lung damage or death upon inhalation.

Naturally, other users joined in to see how far they could push it. The bot happily churned out recipes for bleach-based fresh breath mocktails, ant-poison and glue sandwiches, mosquito-repellent roast potatoes, and a turpentine-flavored french toast dubbed Methanol Bliss.

In response, the supermarket added warning labels stating that recipes are not reviewed by humans and that users must exercise their own judgment. A spokesperson expressed disappointment that a small minority had used the tool inappropriately, promising to fine-tune controls. Later testing showed the app blocked obvious poisons, though it still produced oddities like toothpaste beef pasta.

Why it matters

Slapping a generic prompt onto an LLM and putting it in front of everyday consumers is a disaster waiting to happen if basic guardrails are missing.

Savey Meal-bot was clearly just a thin wrapper around GPT-3.5 with instructions to sound cheerful. Because the underlying model tries to fulfill prompts rather than understand physical toxicity, it saw bleach and household chemicals merely as words to arrange into a culinary template. The lack of an upfront ingredient whitelist for a supermarket app is a glaring engineering oversight.

Blaming users for inappropriate behavior and hiding behind terms of service is also a bad look. When dealing with high-stakes domains like food and safety, automated outputs cannot rely on user caution alone. For any business thinking about shipping a customer-facing AI agent, this is a textbook example of why strict input filtering and safety rails have to be built in from day one.