Microsoft Introduces New Azure AI Tools to Ensure Security and Trust in Generative AI Applications
- Written by Elijah Ugoh Cybersecurity & Tech Writer
- Fact-Checked by Justyn Newman Former Lead Cybersecurity Editor
Microsoft has unveiled new tools in Azure AI to enhance the security and reliability of generative AI applications. The tools will help generative AI app developers using Azure AI Studio to prevent prompt injection attacks and hallucinations in model outputs.
The new tools are now available , while more will be coming soon to Azure AI Studio for generative AI app developers, says Sarah Bird, Chief Product Officer of Responsible AI at Microsoft. Bird acknowledged that “Prompt injection attacks have emerged as a significant challenge, where malicious actors try to manipulate an AI system into doing something outside its intended purpose, such as producing harmful content or exfiltrating confidential data.”
The announcement, which was made on March 28, added that these tools will be useful for business leaders who are “trying to strike the right balance between innovation and risk management” and “want to ensure that their AI systems are not generating errors or adding information” that can erode user trust.
First, it will address the issue of prompt injection. Prompt injection happens when malicious actors try to exploit vulnerabilities in AI systems and manipulate them to produce undesirable outcomes. Prompt injection attacks have emerged as a significant threat to the safety and security of AI applications, especially chatbots.
Bird explains, “Prompt injection attacks, both direct attacks, known as jailbreaks, and indirect attacks, are emerging as significant threats to foundation model safety and security. Successful attacks that bypass an AI system’s safety mitigations can have severe consequences, such as personally identifiable information (PII) and intellectual property (IP) leakage.”
Microsoft has introduced Prompt Shields , a feature designed to detect and block suspicious inputs in real-time, thereby safeguarding the integrity of large language model (LLM) systems. Safety Evaluations is another essential feature now available on the Azure platform. It’ll help developers assess an application’s vulnerability to jailbreak attacks and generate content risks. However, this feature is currently available in preview mode.
The tech giant will also introduce Groundedness detection , aimed at identifying and mitigating instances of ‘hallucinations’ in model outputs. These hallucinations occur when AI models generate outputs that lack factual grounding or common sense, posing a risk to the reliability of AI-generated content. Safety System Messages is another feature Bird says will be available soon. It will guide the behavior of AI models toward safe and responsible outputs, along with
Finally, the Risk and Safety Monitoring tool will enable organizations to gain insights into model inputs, outputs, and end-user interactions, helping them make informed decision-making and risk mitigation strategies.
Microsoft remains at the forefront of developments in the generative AI landscape. The company has also reaffirmed its commitment to advancing safe, secure, and trustworthy AI , as well as empowering developers to build safe and reliable AI applications.
NYC Will Test Gun Detectors on the Subway
- Written by Deep Shikha Content Writer
- Fact-Checked by Justyn Newman Former Lead Cybersecurity Editor
New York City plans to test a new system that uses artificial intelligence (AI) to detect guns and other weapons on the Subway, Mayor Eric Adams announced on March 28 . This move follows a recent incident where a man was shot with his own gun during a conflict with another passenger at a Brooklyn subway station.
Mayor Adams said that New York City is currently working with Evolv, a company from Massachusetts that makes weapons detection systems used in schools and other places across the US. However, Evolv has been criticized for how well its detectors work. It’s also been investigated by the government twice and is facing a lawsuit from its investors.
When New York City set up Evolv’s scanning machines at a Bronx hospital in 2022, they often mistakenly identified objects as weapons, a report revealed. Of 194,000 scans, 50,000 triggered an alarm, but 85% were false positives. Police officers mostly caused the few true alarms, and only 295 were due to civilians carrying knives, guns, or other types of weapons.
The test will begin in 90 days. It will follow the POST Act rules that allow the New York City Police Department to share details about its surveillance tools and their effects before using new ones. Mayor Adams mentioned that during this 90-day period, the city will also evaluate other suppliers.
A city spokeswoman clarified that New York City hadn’t signed a deal with Evolv. The announcement was actually an invitation for any company with similar technology to come forward, correcting what Mayor Adams had previously stated.
The Legal Aid Society argued that these systems often make mistakes by incorrectly signaling there’s a gun when there isn’t. They mention that, despite what the mayor says, New York City shouldn’t be used as a place for companies to test surveillance technology without people’s agreement.
New York’s state and local officials are working to make the subway safer. Governor Kathy Hochul has sent the National Guard and State Police to do random bag checks . Also, Caban recently took subway trips to listen to what passengers had to say. Authorities report that crimes in the transit system have increased by 4% this year compared to last and by 8% compared to 14 years ago.
On March 25, the New York Police Department also revealed a plan to deploy 800 extra officers for a one-week effort to target people skipping subway fares. In addition, the city is bringing on more mental health professionals to assist those dealing with mental illness.
