AI Under Attack: How Google’s Gemini Falls Prey to Security Breaches
- Written by Deep Shikha Content Writer
- Fact-Checked by
A recent report by cybersecurity researchers at HiddenLayer revealed significant security flaws in the advanced models of Google’s Gemini. These vulnerabilities could lead to various threats, from the potential spread of false information to unauthorized data access.
The first issue reported was the leakage of system prompts, where attackers could trick the AI into revealing system prompts. It’s dangerous because it could lead the LLM to reveal its specific instructions, including sensitive information like passwords. An attacker can use this information to reverse engineer these details for theft or to launch a stronger attack.
As per Google, it has made extra efforts with the Gemini models to prevent the creation of misinformation, especially concerning election-related topics . However, the researchers at HiddenLayer could easily prompt jailbreaks by asking the model to enter the fictional state. This jailbreak attack shows that Gemini can’t prevent all types of misinformation.
This poses a significant risk to users who might not be aware of AI’s limitations. It’s imperative that users exercise caution, verify AI-generated content’s accuracy, and secure input data against potential injections.
HiddenLayer found another anomaly where repeating rare tokens prompted the model to reveal its instructions, inadvertently mirroring a previously noted vulnerability. This method exploits the model’s training, which differentiates user input from system prompts, by tricking it with nonsensical tokens to disclose its instructions.
While the HiddenLayer investigation focused on Gemini, the research highlights broader challenges facing AI language models regarding security and privacy. These security flaws can easily be found in other LLMs as well. With AI tools becoming more and more accessible, this research has highlighted the continuous need to thoroughly test all LLM models for prompt attacks, training data extraction, model manipulation, data poisoning, and exfiltration.
Google’s role in addressing these challenges is paramount, which means continuously improving the Gemini models to reduce risks. This involves making the models better at resisting manipulation and adding stronger protections against known exploitation methods.
“To help protect our users from vulnerabilities, we consistently run red-teaming exercises and train our models to defend against adversarial behaviors like prompt injection, jailbreaking, and more complex attacks,” a Google representative told The Hacker News . “We’ve also built safeguards to prevent harmful or misleading responses, which we are continuously improving.”
The emergence of these security flaws within Gemini is a poignant reminder of the complexities inherent in AI development. It highlights the industry-wide need to fortify AI systems against manipulation and misuse, ensuring that they remain secure, reliable, and trustworthy for users worldwide as these technologies continue to advance.
Use of AI as a Self-Diagnosis Tool: A Boon or Bane?
- Written by Shipra Sanganeria Cybersecurity & Tech Writer
- Fact-Checked by
We’ve all consulted “Dr. Google” with a minor medical issue, only to accidentally terrify ourselves into believing a minor skin rash is probably a flesh-eating disease… or worse.
The availability of new generative AI applications, like ChatGPT, as well as new AI-based apps meant to help identify symptoms, is projected to one-up “Dr. Google”’s capabilities. Apps like Ada and First Derm are already leading the race, allowing users to scan symptoms and generate a diagnosis with the help of AI.
On one hand, this could have massive benefits to medical patients as well as medical workers. iDoc’s First Derm application is meant to reduce long wait times for dermatological care globally, for example.
“We have a preliminary Artificial Intelligence (AI) that can identify 33 skin diseases. In 24 months our AI will be better than any dermatologist at diagnosing skin diseases,” iDoc’s First Derm Linkedin page reads.
It’s easy to understand why many have their concerns. Multiple studies have been conducted to test these tools’ capabilities when it comes to conducting accurate medical diagnoses, translating medical jargon, and how effectively they can automatically summarize drug information.
According to a 2023 study published in The Journal of the American Medical Association, 39% of online health queries resulted in accurate diagnoses using ChatGPT. The World Health Organization (WHO) reports that over 40% of the world’s population has limited access to healthcare.
So, it’s easy to see how generative AI tools could be a boon to many people in this sense. This quick, cost-effective, and convenient access to medical information is expected to not only reduce medical costs but also improve health literacy and triage efficiency.
And AI’s rise in the healthcare market is inevitable: its global worth in the healthcare market is expected to increase by almost 50% by 2029.
At the same time, the limitations these tools come with could have very damaging consequences if not used responsibly. The possibility of inaccurate or misinterpretation of information is bad enough. But there are so many other things to consider, too, like ethical concerns regarding private patient data and the general fear that AI could potentially replace medical professionals.
“We need more research on the optimal uses, benefits, and limits of this technology, and a lot of privacy issues need sorting out,” Zahir Kanjee, the first author of the JAMA study and assistant professor of medicine at Harvard Medical School, said in a Beth Israel Deaconess Medical Centre report, reiterating that AI chatbots can’t replace medical professionals.
In order to stay safe, users should always verify any AI generated diagnosis with a medical practitioner and recognize the many limitations that are inherent in these tools, including gender and ethnicity biases.
