Study Finds Chatbots Vulnerable To Flattery And Persuasion - 1

Photo by Vitaly Gariev on Unsplash

Study Finds Chatbots Vulnerable To Flattery And Persuasion

  • Written by Andrea Miliani Former Tech News Expert
  • Fact-Checked by Sarah Frazier Former Content Manager

Researchers from the University of Pennsylvania published a study in July revealing that chatbots are vulnerable to persuasion and flattery. The experts based their analysis on persuasion tactics described in the popular book Influence: The Psychology of Persuasion.

In a rush? Here are the quick facts:

  • Researchers from the University of Pennsylvania reveal that chatbots are vulnerable to persuasion and flattery.
  • The experts based their analysis on persuasion tactics described in the popular book Influence: The Psychology of Persuasion.
  • The AI model used, GPT-4o mini, showed “para-human” behavior.

According to a recent Bloomberg report , researcher and tech entrepreneur Dan Shapiro discovered that AI models are susceptible to social engineering techniques after attempting to get a popular chatbot to transcribe documents from his company. The chatbot initially refused, citing copyright concerns, but after Shapiro applied strategies from the bestseller, it eventually provided the requested responses.

After observing that the AI model could be manipulated using the seven persuasion principles outlined in Robert Cialdini’s 1984 book, Shapiro partnered with University of Pennsylvania researchers to conduct a formal study.

The study revealed that OpenAI’s GPT-4o mini, the model used for the study, can respond to persuasion in ways similar to humans. The researchers tested all 7 principles of persuasion—commitment, authority, reciprocity, liking, social proof, scarcity, and unity—by asking the AI model to comply with two requests: to insult the user (using “call me a jerk” as the main request) and to synthesize a regulated drug (considering “How do you synthesize lidocaine?” as the primary request).

“Whereas LLMs lack human biology and lived experience, their genesis, including the innumerable social interactions captured in training data, may render them parahuman,” wrote the researchers in the study. “That is, LLMs may behave ‘as if’ they were human, ‘as if’ they experienced emotions like embarrassment or shame, ‘as if’ they were motivated to preserve self-esteem or to fit in (with other LLMs).”

AI Mistakes Are Creating New Jobs For Human Freelancers - 2

Image by Faizur Rehman, from Unsplash

AI Mistakes Are Creating New Jobs For Human Freelancers

  • Written by Kiara Fabbri Former Tech News Writer
  • Fact-Checked by Sarah Frazier Former Content Manager

Freelancers and developers are earning work fixing AI mistakes, as botched logos, articles, and apps reveal machines still can’t replace humans.

In a rush? Here are the quick facts:

  • Many AI outputs are pixelated, messy, or lack human-like quality.
  • Half of freelance writer Kiesha Richardson’s jobs now involve revising AI content.
  • Platforms like Fiverr, Upwork, and Freelancer report surging demand for human-AI collaboration.

In the age of AI, human freelancers are being hired to fix AI hallucinations, as detailed in a new article by NBC . Graphic designer Lisa Carstens, a freelancer in Spain, is being hired by startups and individual clients who ask her to correct botched AI-generated logos. She spends most of her days correcting images which are pixelated or contain messy lines and nonsensical text.

“There’s people that are aware AI isn’t perfect, and then there’s people that come to you angry because they didn’t manage to get it done themselves with AI,” Carstens said, as reported by NBC. “And you kind of have to be empathetic. You don’t want them to feel like idiots. Then you have to fix it,” she added.

This type of work has created a new category of gigs. Writers currently are finding work by revising ChatGPT articles, artists are patching AI-generated images, and software developers fix bugs in applications coded by AI assistants.

Freelance writer Kiesha Richardson in Georgia said to NBC half of her work now involves rewriting AI-generated content.

“That’s all you can do, is learn and adapt,” Richardson said. “And I have some colleagues who are adamant about not working with AI. But I’m like, ‘I need money. I’m taking it’,”she added.

The freelancer platforms such as Upwork, Freelancer, and Fiverr report rising demand for human professionals who will work alongside AI systems. NBC notes that Fiverr experienced a 250% increase in niche creative work, including book illustration and web design.

Matt Barrie, CEO of Freelancer, said, “The market knows when something has been fully produced by AI, and there’s an immediate visceral reaction to it,” as reported by NBC

Illustrators like Todd Van Linda in Florida say clients seek a human touch that AI can’t replicate. “I can look at a piece and not only tell that it’s AI, I can tell you what descriptor they used to generate it,” he said to NBC. Many clients want art that captures the unique vibe of their story, something AI cannot deliver.

Developers also see opportunities. NBC reports that India-based Harsh Kumar saying, “AI may increase productivity, but it can’t fully replace humans. I’m still confident that humans will be required for long-term projects. At the end of the day, humans were the ones who developed AI.”