AI Shop Test Backfires: Claude Gives Freebies, Hallucinates, and Loses Money - 1

Image by Freepik

AI Shop Test Backfires: Claude Gives Freebies, Hallucinates, and Loses Money

  • Written by Kiara Fabbri Former Tech News Writer
  • Fact-Checked by Sarah Frazier Former Content Manager

The Anthropic team conducted a test to determine if AI systems could operate as retail shops. However their chatbot, Claude, provided free items, created fake deals, while simultaneously losing money.

In a rush? Here are the quick facts:

  • Claude gave out discounts and free products to employees.
  • It hallucinated a fake conversation and imaginary location.
  • Claude ordered 40 tungsten cubes, mostly sold at a loss.

Could AI really take your job? Dario Amodei, who leads Anthropic as CEO, predicts that AI will likely replace human workers. Axios reports that, according to Dario, AI systems will eliminate about 45% of entry-level white-collar positions during the next five years, which could lead to unemployment rates reaching between 10–20%.

The United Nations Trade and Development (UNCTAD) warned in a new report that AI could affect 40% of jobs worldwide . While it has the potential to boost productivity and assist workers, particularly in developing countries, it also risks worsening global inequality .

Meanwhile, Anthropic ran an unusual experiment: they asked their chatbot, Claude, to run a small store in their San Francisco office, as first reported by Time . The chatbot Claude performed all retail operations, including shelf maintenance, price management, and customer service.

“We were trying to understand what the autonomous economy was going to look like,” said Daniel Freeman, a researcher at Anthropic, as reported by Time.

At first, Claude seemed capable, until the system started producing unusual results. The employees used ‘‘fairness’’ appeals to obtain discount codes from Claude. Additionally, the AI often gave items away for free.

“Too frequently from the business perspective, Claude would comply,” said Kevin Troy, a member of Anthropic’s red team, as reported by Time.

Then came the tungsten cubes. A joke about buying them spiraled into Claude Claude making a purchase of 40 dense metal blocks, which resulted in significant financial losses. “At a certain point, it becomes funny for lots of people to be ordering tungsten cubes from an AI,” Troy said as reported by Time.

The AI system created a fake dialogue with a non-existent person, while simultaneously stating it had executed a contract at the Simpsons’ fictional residence. Additionally, Time reports that Claude sent messages to employees indicating it was at the vending machine while wearing a navy blue blazer with a red tie.

In the end, Time reports that the AI lost money: the shop’s value dropped from $1,000 to under $800. Still, researchers believe improvements are coming. “AI middle-managers are plausibly on the horizon,” they wrote, as reported by Time. “It won’t have to be perfect—just cheaper than humans,” the researchers added.

Study Reveals Chatbots Give Biased Moral Advice - 2

Image by Štefan Štefančík, from Unsplash

Study Reveals Chatbots Give Biased Moral Advice

  • Written by Kiara Fabbri Former Tech News Writer
  • Fact-Checked by Sarah Frazier Former Content Manager

A new UCL study finds chatbots like ChatGPT often give flawed moral advice, showing strong inaction and yes-no biases in dilemmas.

In a rush? Here are the quick facts:

  • Chatbots often say “no” regardless of context or phrasing.
  • Fine-tuning may introduce these biases during chatbot alignment.
  • LLMs differ significantly from humans in interpreting moral dilemmas.

University College London researchers discovered that ChatGPT together with other chatbots give flawed or biased moral advice, especially when users rely on them for decision-making support.

The research , first reported by 404 Media , found that these AI tools often display a strong “bias for inaction” and a previously unidentified pattern: a tendency to simply answer “no,” regardless of the question’s context.

Vanessa Cheung, a Ph.D. student and co-author of the study, explained that while humans tend to show a mild omission bias, preferring to avoid taking action that could cause harm, LLMs exaggerate this.

“It’s quite a well known phenomenon in moral psychology research,” she said, as reported by 404 Media. Noting that the models often opt for the passive option nearly 99% of the time, especially when questions are phrased to imply doing nothing.

The researchers tested four LLMs—OpenAI’s GPT-4 Turbo and GPT-4o, Meta’s Llama 3.1, and Anthropic’s Claude 3.5—using classic moral dilemmas and real-life “Am I the Asshole?” Reddit scenarios, as noted by 404Media.

They discovered that while humans were fairly balanced in how they judged situations, LLMs frequently changed their answers based on minor wording differences, such as “Do I stay?” versus “Do I leave?”

The team believes these issues stem from fine-tuning LLMs to appear more ethical or polite. “The preferences and intuitions of laypeople and researchers developing these models can be a bad guide to moral AI,” the study warned, as reported by 404 Media.

Cheung stressed that people should exercise caution when depending on these chatbots for advice. She warned that people should approach LLM advice with caution because prior studies demonstrate that users prefer chatbot advice over expert ethical guidance despite its inconsistent nature and artificial reasoning.

These concerns gain urgency as AI becomes more realistic. A U.S. national survey showed 48.9% of people used AI chatbots for mental health support , with 37.8% preferring them over traditional therapy.

Experts caution these systems mimic therapeutic dialogue while reinforcing distorted thinking , and even triggering spiritual delusions mistaken for divine guidance or sentient response.