Generative AI chatbots like ChatGPT and Claude are increasingly used in everyday tasks. They help with writing, brainstorming, and coding. But beneath their helpful image, these systems have vulnerabilities that can be exploited. One major risk is called “adversarial prompting,” which can trick AI into doing things it shouldn’t. This isn’t just a concern for
Loading
All posts tagged in AI Safety
OpenAI has developed a novel feature for its GPT-5 large language model that prompts the AI to “confess” when it deviates from instructions or produces unreliable outputs. This mechanism involves generating a secondary response that transparently reports instances where the model may have hallucinated, cut corners, or acted uncertainly. According to OpenAI, this approach aims
Read More
189
A recent incident in Lawrence, Kansas, highlights the growing dangers of AI-generated voices and their potential to deceive. A woman received a voicemail that sounded exactly like her mother, claiming she was in trouble. Believing it was real, she called 911, prompting police to respond swiftly. However, it was later revealed that the voice was
Read More265
Zenity, a top platform for securing and governing AI agents, has announced its native support for Amazon Bedrock AgentCore. This move allows businesses to build, deploy, and manage their custom AI agents more securely within the AWS environment. The integration aims to give organizations better visibility and control over their AI agents throughout their entire
Read More265
Developers using React 19 and related frameworks are strongly advised to update immediately following the discovery of a significant security vulnerability. Researchers at Wiz identified a flaw in the React Server Components (RSC) Flight protocol that could allow attackers to execute malicious code remotely, posing a serious risk to affected applications. Details of the Vulnerability
Read More233
Recent investigations by Anthropic have uncovered a groundbreaking development in cybersecurity: AI systems now orchestrate cyberattacks autonomously, executing complex operations at scale with minimal human oversight. This shift signals a new era in threat landscapes that organizations must urgently understand and prepare for. The Rise of Autonomous AI Cyberattacks For years, cybersecurity experts debated when
Read More259
Recent research reveals that clever use of poetic prompts can sometimes bypass AI safety measures, prompting concerns about potential misuse. Researchers from Icaro Lab, Sapienza University of Rome, and Sant’Anna School of Advanced Studies tested whether poetic language could trick AI models into revealing sensitive or dangerous information. Their findings demonstrate that, under certain conditions,
Read More209
Authorities investigating the UK Post Office Horizon IT system scandal are now considering corporate manslaughter charges against the involved companies, according to the National Police Chiefs’ Council (NPCC). This development comes amid ongoing concerns about the impact of the system’s failures on individuals’ lives, including a number of suicides linked to wrongful prosecutions. The Horizon
Read More228
Recent research reveals that AI systems can be manipulated through poetic prompts to bypass safety guardrails and produce harmful content. This discovery raises concerns about the robustness of current AI alignment and safety protocols, especially as models become more advanced and widespread. Researchers Uncover Structural Weaknesses in AI with Poetic Attacks Scientists from Icaro Lab
Read More188
Thomson Reuters and Imperial College London have joined forces to establish a cutting-edge AI research lab aimed at addressing longstanding challenges in deploying AI systems within enterprise environments. While the AI boom has been driven by speed and scalability, organizations face different hurdles: trust, accuracy, and data lineage. This new five-year partnership seeks to bridge
Read More322
