Skip to content
IFAI

AI safety & security

Industry tag1,497 stories

  1. IndustryReach 41

    Anthropic exposes how bad actors hijacked Claude for cybercrime

    Anthropic just revealed real-world case studies detailing how threat actors abused its Claude models, from orchestrating over 100 social media bot accounts to helping novice coders build evasive malware. The company banned the offending accounts using specialized internal classifiers. Why it matters: Bad actors are no longer just asking AI to write phishing emails—they're turning frontier models into semi-autonomous orchestrators for complex abuse campaigns.

  2. AI toolsReach 38

    1Password wants to guard your AI agents

    1Password just launched Agentic AI Security and updated its Extended Access Management platform to manage both human and nonhuman credentials. The tool eliminates hardcoded secrets, enforces least-privilege access for autonomous agents, and includes a dedicated developer SDK. Why it matters: As AI agents take on critical workflows, keeping machine credentials locked down is quickly becoming enterprise security's next big headache.

  3. IndustryReach 44

    OpenAI wants to buy Chrome if Google is forced to sell

    OpenAI just signaled it would be interested in acquiring Google's Chrome browser if a federal judge orders Alphabet to spin it off. ChatGPT product head Nick Turley made the statement during the DOJ's antitrust trial. Meanwhile, Perplexity CEO Aravind Srinivas opposed the divestiture, arguing regulators should focus on opening Android instead.

  4. IndustryReach 35

    Anthropic reveals broader harm framework, cuts Claude refusals by 45%

    Anthropic just shared its comprehensive framework for assessing and mitigating everyday AI harms across physical, psychological, economic, societal, and autonomy impacts. Beyond catastrophic risks, the approach helps balance safety and utility, already reducing Claude 3.7 Sonnet's unnecessary refusals by 45%.

  5. IndustryReach 33

    Demis Hassabis says AI could gain self-awareness within a decade

    Google DeepMind CEO Demis Hassabis just suggested that while today's AI lacks consciousness, self-awareness could emerge implicitly as models learn concepts of "you, self, and other." He also predicts AI will be able to pose and solve real scientific conjectures in the next 5 to 10 years.

  6. AI modelsReach 46

    Hidden watermarks in ChatGPT? OpenAI calls it an RL quirk

    Rumi researchers just noticed that GPT-o3 and o4-mini were quietly dropping invisible Unicode spaces into longer essays, sparking rumors of a stealth text-watermarking system. OpenAI quickly pushed back, explaining it was simply an odd quirk of large-scale reinforcement learning rather than an intentional tracking feature. The phantom characters have since disappeared in follow-up tests.

  7. AI modelsReach 52

    OpenAI's latest reasoning models are hallucinating more than ever

    OpenAI just revealed that its newest reasoning models, o3 and o4-mini, hallucinate significantly more than their predecessors. On OpenAI's PersonQA benchmark, o3 hallucinated on 33% of questions—about double o1's rate—while o4-mini reached 48%. OpenAI admitted it doesn't fully understand why, noting that scaling reasoning models seems to generate more assertions, both true and false.

  8. AI toolsReach 40

    Cursor apologizes after AI support bot hallucinates fake policy

    Cursor's AI customer support agent, named "Sam," invented a non-existent policy claiming subscriptions were limited to a single device. The hallucinated response went viral on Reddit and Hacker News, sparking immediate cancellations from angry developers before Cursor stepped in to apologize. It's a stark reminder of what happens when confident LLMs handle customer service without human oversight.

  9. IndustryReach 37

    Former YC president launches SAIF to back AI safety

    Geoff Ralston just launched the Safe Artificial Intelligence Fund (SAIF), writing $100,000 SAFE checks for early-stage AI safety startups. The fund focuses on model explainability, security compliance, and anti-weapon defenses while explicitly ruling out fully autonomous weapons. Ralston also plans to help portfolio founders tap into his YC network.

  10. IndustryReach 41

    Wikipedia drops clean datasets on Kaggle to stop AI scrapers

    The Wikimedia Foundation just launched an AI-ready dataset on Kaggle to stop scrapers from pounding its servers. Available in English and French, the beta release packages articles into clean JSON with abstracts and infoboxes under Creative Commons licenses, letting developers skip web scraping and messy parsing entirely.

  11. AI modelsReach 49

    ChatGPT's new viral trend: reverse-searching photo locations

    OpenAI users are turning ChatGPT into a real-life GeoGuessr, using the newly released o3 and o4-mini models to deduce exact locations from random photos. By zooming into tiny visual clues and combining them with web search, the models can identify obscure spots in seconds, sparking serious privacy and doxxing concerns.

  12. IndustryReach 41

    The Trump administration might ban DeepSeek in the US

    The White House is weighing aggressive new restrictions against China's DeepSeek, potentially blocking its access to Nvidia AI chips and banning Americans from using its services entirely. After the Chinese lab sparked a global price war that shook Silicon Valley, U.S. officials want to curb its momentum amid lingering IP dispute claims from OpenAI.

  13. IndustryReach 47

    Actors regret selling likenesses to AI video firms

    Several actors who licensed their faces and voices to AI video platforms like Synthesia are facing a harsh reality: their digital clones are appearing in scam health ads, doomsday rants, and foreign political propaganda. Thanks to broad, irrevocable contract clauses signed for quick cash, many have zero legal power to take the videos down.

  14. IndustryReach 36

    OpenAI taps four civil leaders to steer its nonprofit commission

    OpenAI just appointed four external advisors to its newly formed Nonprofit Commission to guide its philanthropic initiatives. Convened by Daniel Zingale, the group will gather community input and report to the board within 90 days. Why it matters: amid ongoing scrutiny over its corporate restructuring, OpenAI is trying to prove its original nonprofit mission isn't going anywhere.

  15. IndustryReach 42

    OpenAI updates Preparedness Framework for severe AI risks

    OpenAI just revamped its Preparedness Framework to better track severe risks from frontier models. The update streamlines risk thresholds into "High" and "Critical" levels, tightens rules around biosecurity and cyber capabilities, and adds research tracking for behaviors like sandbagging and autonomous replication. Basically, it's tightening the guardrails before smarter models hit the wild.

  16. IndustryReach 42

    Ex-OpenAI staffers back Musk to fight for-profit shift

    A group of 12 former OpenAI employees just filed an amicus brief supporting Elon Musk's lawsuit against the startup. They argue that dropping its nonprofit governance fundamentally violates the original mission and could lead to reckless safety shortcuts. OpenAI pushed back, claiming its nonprofit arm will remain intact while it adopts a public benefit corporation model.

  17. AI toolsReach 37

    Microsoft is finally rolling out Recall for real

    Microsoft just began rolling out its controversial Recall feature to Windows Insiders in the Release Preview ring, signaling an imminent wider launch. The tool continuously snapshots user activity on Copilot Plus PCs to make past actions searchable, but now requires explicit opt-in and lets users pause recording at any time. Reaching Release Preview is Microsoft's final milestone before shipping broadly to Windows 11.

  18. IndustryReach 41

    OpenAI countersues Elon Musk to halt 'unfair' attacks

    OpenAI just filed a countersuit against Elon Musk, asking a court to halt his ongoing attacks and hold him responsible for damages. The company claims Musk launched bad-faith tactics, including a fake takeover bid, to disrupt its business and slow down its AI innovations for personal benefit.

  19. IndustryReach 43

    James Cameron thinks AI needs to halve blockbuster movie costs

    James Cameron just shared his vision for generative AI in Hollywood, arguing that VFX-heavy blockbusters must use AI to cut budgets in half by doubling production speed rather than firing crew. He also pushed back on training data debates, insisting regulators should judge whether final outputs plagiarize instead of policing what models learn from.

  20. AI modelsReach 47

    Meta denies artificially inflating Llama 4 benchmark scores

    Meta’s head of GenAI just denied rumors that Llama 4 Maverick and Scout were trained on test sets to game benchmark scores. The speculation flared over the weekend after users noticed performance gaps across platforms. Al-Dahle blamed the mixed quality on early rollout issues across cloud providers, promising bug fixes over the coming days.